JP6073649B2 - Automatic voice recognition / conversion system - Google Patents

Description

  The present invention relates to an automatic speech recognition / speech conversion system. For example, the present invention relates to a speech automatic recognition / speech conversion system suitable for provision to a call center.

  As background art of this technical field, there are JP-A-2005-12833 (Patent Document 1) and JP-A-2011-9902 (Patent Document 2).

  In the former publication, “in a voice response service device that performs processing in a dialogue format with a user by voice response, an attribute storage unit that stores the attribute of the user, and a user's stored in the attribute storage unit” A user sound quality changing unit that changes the sound quality of the voice that responds to the user based on the attribute information, and the sound quality is adjusted according to the attributes such as the gender and age classification of the telephone service user and the operation environment of the other party "Voice response service device that provides a highly comfortable service by changing". (See summary).

  Further, the latter publication states that “a first voice acquisition unit that acquires a voice of a telephone call from a customer to a store, an emotion recognition unit that recognizes an emotion from the voice acquired by the first voice acquisition unit, and the emotion recognition Based on whether or not the emotion type recognized by the means represents at least one of “anger” and “excitement”, a complaint determination means for determining whether or not the voice content is a complaint, and a person in charge of handling the complaint A first notification for notifying the contact stored in the contact storage means when the voice content is determined to be a complaint by the contact storage means for storing the contact information of the person and the complaint determination means And a customer response device for providing a customer response device capable of automatically discriminating complaints from customers and notifying an appropriate customer. (See summary).

JP 2005-12833 A JP 2011-9902 A

  In Patent Document 1, the volume and sound speed are changed in accordance with attributes such as sex and age classification of the telephone service user and the operation environment of the other party. For example, when the user makes an input error on a push button, A voice response service device that increases the sound speed and slows down the sound speed is described. However, the voice response service device of Patent Document 1 is effective in a telephone service that performs voice conversion of pre-registered narration, but converts the conversation voice of a person in charge who needs complex support in real time. For example, it cannot be used in a call center.

  Further, in Patent Document 2, a customer who recognizes emotions from telephone voices of customers who are responding to telephone calls, automatically discriminates complaints, analyzes voiceprints, and assigns a complaint processing person who is appropriate for the customer's age and gender. A corresponding device is described. However, no particular consideration is given to the response after assigning the person in charge.

  In other words, in each patent document, if changes in the customer's (partner's) emotion or the like can be felt during the conversation, the person in charge of the conversation (customer response person) during the conversation seems to be appropriate for the change in the customer's emotion or the like. It doesn't even consider changing to voice in real time and trying to reduce customer stress.

  Therefore, the present invention provides an automatic speech recognition / speech conversion system that can reduce the stress of a customer by changing the voice of a customer service representative in real time according to the emotional state of the customer during conversation.

  In order to solve the above-described problem, the present invention provides a voice that changes a person-in-charge voice in real time in response to an emotional change of a customer when the emotional change is perceived from the voice of the customer during a conversation after the start of telephone correspondence. Voice conversion means for performing conversion tuning (reconstruction of voice conversion) is provided.

An automatic voice recognition / conversion system used in a service providing system for carrying out voice conversations with a call partner via voice communication means,
The voice automatic recognition / speech conversion system includes:
Voiceprint recognition means having a function of recognizing the voiceprint of the input voice of the other party;
Emotion recognition means having a function of recognizing the other party's emotion from the input voice;
A customer DB that stores customer information indicating the voiceprint, age, and gender of the other party;
A person-in-charge DB that accumulates voice parameters of persons in charge corresponding to the other party;
A voice DB that stores voice parameters corresponding to age, gender, and emotion;
Voice analysis means for receiving a voiceprint recognition result of the voiceprint recognition means and an emotion recognition analysis result of the emotion recognition means, calculating a difference between the voice parameter of the person in charge DB and the parameter of the voice DB, and outputting the difference;
Voice conversion means for receiving the difference extracted by the voice analysis means, converting the voice of the person in charge based on the difference, and outputting the voice to the communication means;
Have
In a call state between the call partner and the person in charge, the emotion recognition unit detects a change in the emotion of the call partner, and the voice analysis unit calculates a voice parameter of the voice DB and a voice parameter of the person in charge. When the difference is calculated, the voice conversion means converts the voice of the person in charge on the basis of the difference in real time. An automatic voice recognition / voice conversion system.

The automatic speech recognition / speech conversion system,
The service providing system includes a call center, and the call center includes display means for displaying the analysis result of the voice analysis means.

The automatic speech recognition / speech conversion system,
The voice automatic recognition / speech conversion system includes:
Furthermore, it has keyword recognition means for recognizing a keyword related to re-listening from the input voice of the other party,
When the keyword recognition unit recognizes the keyword and the voice analysis unit detects the keyword, the voice conversion unit changes a parameter value of each attribute according to the keyword, and performs the charge according to the parameter. A voice automatic recognition / conversion system characterized by converting the volume and / or speed of a person's voice.

  According to the present invention, there is provided an automatic speech recognition / speech conversion system capable of generating a person-in-charge voice corresponding to a customer (customer) emotion in real time and, as a result, reducing a customer stress. be able to.

It is a block diagram which shows the whole structure when it applies to the service provision apparatus (call center apparatus) using the voice automatic recognition and the voice conversion system of this invention. It is a flowchart explaining the process of a voice automatic recognition and a voice conversion system. It is a figure which shows the flow of the audio | voice between a customer and a call center, and the schematic processing example by the call center side. It is a figure which shows the example of tuning when an audio | voice automatic recognition / audio | voice conversion system detects an emotional change. It is a figure which shows the example of tuning when the audio | voice automatic recognition and the audio | voice conversion system detect the keyword of re-listening.

Hereinafter, examples will be described with reference to the drawings.
The contact center or call center reception system (hereinafter referred to as the call center) and the quality of the telephone response greatly depend on the target product, service, and the impression of the company itself. Therefore, it is required to have a voice response that is easy to understand for customers (hereinafter referred to as customers) and that does not give stress.
However, the current situation is that there are variations in the correspondence between telephones depending on the person in charge. There is a risk that the product, service, and the company's own impression will be worsened by the poor telephone response, and there will be a large loss.

  In this embodiment, an example of reducing the above-described risk in the call center will be described.

  FIG. 1 is a configuration diagram showing the overall configuration when the automatic speech recognition / speech conversion system of the present invention is applied to a call center device.

  The call center 100 includes a communication unit (communication unit) 110, a display unit (display unit) 120, and an automatic speech recognition / speech conversion system 130.

  The communication unit 110 transmits / receives voice to / from a communication terminal (communication unit) 200 such as a telephone on the customer side. The received customer voice (input voice) is supplied to the display means 120 and the automatic voice recognition / speech conversion system 130.

  An automatic speech recognition / speech conversion system 130 includes a voiceprint recognition unit (voiceprint recognition unit) 1301, an emotion recognition unit (emotion recognition unit) 1302, a keyword recognition unit (keyword recognition unit) 1303, a voice analysis unit (speech analysis unit) 1304, A voice conversion unit (voice conversion means) 1305, a customer DB 1306, a person-in-charge DB 1307, and a voice DB 1308 are provided.

The voiceprint recognition unit 1301 receives the telephone voice (input voice) from the communication unit 110 and recognizes the voiceprint of the voice. From this voiceprint recognition, age, gender, and cautionary points (if there is a history) are determined.
This voiceprint recognition is performed by collating with a voiceprint registered in the customer DB 1306, for example. If not registered in the customer DB 1306, the voiceprint recognition unit 1301 determines the age, sex, and the like from the voiceprint of the input voice, and registers the result in the customer DB 1306.

  The emotion recognizing means 1302 discriminates emotions such as “feeling emotional” of the target customer from the tone color (voice color) of the input voice.

  The keyword recognizing unit 1303 detects a keyword as a keyword, for example, a keyword such as “low voice” related to re-listening, from the customer's voice.

The voice analysis unit 1304 searches the voice DB 1308 and the person in charge DB 1307 based on the data indicating the age and sex by the voiceprint recognition unit 1301 and the data indicating the emotion by the emotion recognition unit 1302.
Then, voice information (speech parameters such as volume, voice speed, pitch, sound quality, etc.) that can be easily heard by the customer (partner) is extracted from the voice DB.

  Also, the voice analysis unit 1304 extracts voice information (voice parameters such as volume, voice speed, pitch, and sound quality) of the person in charge from the person-in-charge DB 1307. Further, when the keyword recognition unit 1303 detects a keyword such as “voice is low” or “cannot be heard” from a customer, for example, the corresponding voice information (voice parameters such as volume and voice speed) is obtained from the voice DB 1308. Extract.

  In short, the voice analysis means 1304 extracts voice information (attribute parameters) that can be converted into a volume, voice speed, pitch, and sound quality that are easy for the customer to hear according to the state, age, gender, etc. of the target customer. Are displayed on the display device 120.

The voice conversion means 1305 converts the voice of the person in charge (output voice) into voice (volume, voice speed, pitch, sound quality) that is easy for the customer (partner) to hear based on the voice analysis information from the voice analysis means 1304. Also, real-time voice conversion tuning such as “increase the volume” and “decrease the voice speed” from the conversation contents such as “speech”, “quick”, etc. are performed in real time.
In other words, the sound is tuned according to the state of the customer 20 according to the attribute parameter value from the sound analysis means. This converted voice is transmitted to the communication means 200 on the customer side via the communication means 110.

  The customer DB 1306 stores personal information of the customer (customer), as well as information indicating the voiceprint, the age and sex determined from the voiceprint.

  The person-in-charge DB 1307 has parameters such as “volume: 50, voice speed: 70. pitch 20, sound quality: −40” and person B as “voice volume”, “person A”, and “volume: 30, Parameters such as “voice speed: 30. pitch 70, sound quality: +20” are accumulated.

  The voice DB 1308 stores voice parameters corresponding to age, gender, and emotion, for example, “volume: 65, voice speed: 45, pitch: 60, sound quality:“ age 50 years old, gender: male, emotion: normal ”. +30 ”,“ age 20 years old, gender: female, emotion: anger ”,“ sound volume: 45, voice speed: 55, pitch: 40, sound quality: +10 ”etc. Accumulate parameter combinations that result in speech that seems to be suitable.

  FIG. 2 is a diagram showing a processing flow in the automatic speech recognition / speech conversion system of the present invention.

In the same figure, telephone correspondence, voice conversion, update of changes, etc. will be described.
First, in step S13001, the voice parameter of the person in charge of the voice conversion source is registered in advance in the person in charge DB 1307.

  Next, in step S13002, the voice 31 from the communication means 200 on the customer (hereinafter referred to as customer) 20 side is received by the communication means 110 on the service provider side 100.

At this time, the automatic speech recognition / speech conversion system 100 executes processing according to the following steps.
In step S13003, the voiceprint recognition unit 1301 acquires voiceprint and voice color information from the voice 31 of the customer 20.

  In step S13004, the voiceprint recognition unit 1301 searches for a voiceprint in the customer DB 1306. In step S1305, the voiceprint recognition unit 1301 determines whether a voiceprint matching the voiceprint has been registered in the customer DB 1306. That is, it is determined whether or not the customer voice 31 is an existing user registered in the customer DB 1306.

  If the determination result shows that it is not registered in the customer DB 1306 (No), the voiceprint recognition means 1301 determines the age and sex from the voiceprint in step S13006, and the determination result is registered in the customer DB1306 in step S13007. To do.

  If the determination result shows that the voiceprint is registered in the customer DB 1306 (Yes), the voiceprint recognition unit 1301 identifies an individual (customer) from the customer DB 1306 and extracts age, gender, cautions, etc. in step S13008. .

  Next, the emotion recognizing means 1302 determines the customer's emotion, that is, “healthy emotion” from the voice color in step S13009.

  In step S13010, the voice analysis unit 1304 searches the voice DB 1308 based on the data of age, gender, and emotion, and then in step S13011, the voice that seems to be optimal according to the age, sex, and emotion of the other party. Extract parameters. Also, in step S13012, the voice parameters of the person in charge (information on volume, voice speed, pitch, sound quality, etc.) are extracted from the person in charge DB 1307.

  In step S13013, the voice analysis unit 1304 takes the difference between the voice parameter of the person in charge 10 and the optimum voice parameter, and extracts the difference (change value).

  In step S13014, the voice conversion unit 1305 converts the voice of the person in charge 10 according to the parameter value of the attribute received from the voice analysis unit 1304.

Next, in step S13015, the communication unit 110 transmits the converted voice 33 converted by the voice conversion unit 1305 to the communication unit 200 on the customer 20 side.
In step S13016, information such as the age, sex, emotions, and caution points (when there is a history) of the customer 20 is displayed on the display unit 120.

  In the state in which the above steps are executed, the emotion recognition means 1302 monitors the customer's emotional change during the conversation with the customer 20 in step S13017. In step S1308, it is determined whether there is an emotional change.

  If there is emotion recognition as a result of the monitoring (Yes), emotion change is detected in step S13019, and points to be noted are displayed on the display means 120. For example, when the customer 20 changes from a normal state to an angry state that suddenly begins to anger, a message indicating that fact is displayed on the display means 120 and the person in charge 10 is urged to pay attention to customer service.

  In this case, the emotion recognizing means 1302 re-searches the voice DB 1308 based on the age, gender, and changed emotion in step S13020, and again extracts a voice parameter that seems to be optimal in this emotion change state.

  In step S13021, the voice analysis unit 1304 takes the difference between the voice parameter of the person in charge and the optimum voice parameter, and extracts the difference (change value).

  In step S13022, the voice conversion unit 1305 converts the voice of the person in charge 10 in accordance with the attribute parameter value received from the voice analysis unit 1304. This converted voice 33 is transmitted to the communication means 200 on the customer 20 side via the communication means 110 in step S13023.

  In addition, the voice analysis unit 1304 reflects this change in the customer DB 1306 in step S13024.

  If there is no change in emotion in step S13018, the keyword recognition means 1303 in the next stage monitors the keyword in conversation in step S13025. In step S13026, it is determined whether there is a keyword to be heard again.

  If there is a keyword to be re-listed as a result of the determination (Yes), the keyword recognizing unit 1303 detects the re-listening keyword in step S13027, and points to be noted, for example, if the voice is low, to that effect. Display on the display means 120.

In step S13028, the voice analysis unit 1304 changes the parameter value of each attribute corresponding to the keyword extracted by the keyword recognition unit 1303.
For example, when the voice of the person in charge 10 is low, the volume is increased (+10). If the mouth is fast, tuning such as lowering the voice speed (-15) is performed.

  In step S13029, the voice conversion unit 1305 converts the voice of the person in charge 10 according to the parameter value of the attribute received from the voice analysis unit 1304.

  The converted voice 33 is transmitted to the communication means 200 on the customer 20 side via the communication means 110 in step S13030.

  In addition, the voice analysis unit 1304 reflects this change in the customer DB 1306 in step S13031.

Finally, in step S13032, it is determined whether or not the telephone correspondence is finished. If not finished (No), the process returns to step 13017.
If the telephone correspondence is terminated (Yes), the telephone correspondence is terminated, that is, the telephone is disconnected in step S13033. Next, in step S13034, the customer DB 1306 is updated and a history is added.

  3 to 5 are diagrams schematically illustrating an example of voice communication (exchange) between a customer (customer terminal) and a person in charge (call center).

First, the state before an emotion change is demonstrated using FIG. FIG. 3 schematically shows the response by the person in charge when it is assumed that there is a problem with the operation management software provided to the customer by the service provider (call center) 100 and the customer contacts the call center. FIG.
When a customer inquires about a product by telephone, the automatic speech recognition / speech conversion system 130 determines that the customer's voice at this time, for example, a voice print such as “Hello”, is registered in the customer DB 1306. Recognize by referring to the information. It also performs emotion recognition from the customer's voice.

  These recognition results are displayed on the PC monitor of the display device 120 on the call center side, for example. In this example, “X Trading YY Contract: J Software Age: 50, Gender: Male, Emotion: Normal” is shown. This monitor allows call center personnel to be seen.

  Next, based on this information, that is, “X Trading YY Contract: J Software Age: 50, Gender: Male, Emotion: Normal” The parameter of “pitch: 60: sound quality: 30” is extracted from the speech DB 1308. In addition, parameters of the “person in charge A volume: 50, voice speed: 70, pitch: 20, sound quality: 40” that are the person in charge values of the person in charge are extracted from the person in charge DB 1307.

  Then, the difference between these parameters is calculated. In this example, “volume: +15, voice speed: −25, sound quality: +40, sound quality +70”. Based on this difference, the voice of the person in charge A is converted so as to be a voice suitable for the customer. The customer responds to this voice-converted voice, for example, "Yay Y of X Shoji. Thank you all the time. Are you inquiring about J Software?"

Next, a case where a change in customer emotion is detected will be described with reference to FIG. FIG. 4 is an example in which emotion recognition is assumed, for example, when the volume of the input voice is 3% higher than normal and the pitch is 5% higher than normal.
In the conversation between the two parties, when a change in emotion is felt from the customer's voice, for example, “normal” to “anger”, the voice DB 1308 reads “ The parameters of volume: 55, voice speed: 35, pitch: 40: sound quality: +10 are extracted, and the difference from the parameter of the person in charge A is “volume: +5, voice speed: -35, pitch: +20: sound quality: +50” Calculate Then, the voice of the person in charge A is reconstructed (tuned) based on this difference. According to this example, fine tuning of voice conversion is also possible.
Note that a simple determination method may be adopted as a determination method of emotion recognition.

Next, the case of re-listening will be described with reference to FIG.
If the customer says "I couldn't hear you, please ask again. My voice is a little loud.", The keyword recognition function, that is, "speech" is detected. The volume of the person in charge A is, for example, “volume: +10”.

  The customer's emotional change and keyword voice at this time are displayed on the PC monitor so that the person in charge A can detect the situation from the PC monitor.

According to the embodiment described above, the following effects can be expected.
(1)
Can talk with the best voice (volume, voice speed, pitch, sound quality) according to the customer's age, gender, and emotion, and can improve the impression of telephone support (products, services, company itself) Can improve the impression). In addition, telephone support troubles (claims) can be reduced by stress-free telephone support.
(2)
All the persons in charge can respond to the above, and it is possible to correct the dispersion of the persons in charge.
(3)
It can accurately detect customer emotion changes that cannot be detected by human ears, and can tune voice conversion and raise the awareness of the person in charge.
(4)
For customers who have an elderly handicap, the voice of the person in charge can be converted into an optimal voice so that they can provide a service that does not stress the customer.
(5)
In addition, as an incidental effect, it is possible to identify an individual with a voice print. By monitoring information related to the individual, for example, in support services, it is easy to confirm the contract and reduce the trouble of confirming the contract on the part of the customer. it can. For example, the person in charge can respond, such as “Dr. XX from XX. Thank you all the time. △△ Are you inquiring about the product?” You can expect a reply such as
(6)
Further, as an accompanying effect, it is possible to identify an individual with a voiceprint, so that it is possible to prevent so-called “spoofing” in which contract information is illegally obtained and a service is used.

In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described.
Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment.
Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit.
Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor.
Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

  The present invention is applicable to any customer service system using voice without specifying a call center.

100 Service provision system (call center)
110 Communication means (call center side)
120 Display means 130 Automatic voice recognition / conversion system 1301 Voiceprint recognition means 1302 Emotion recognition means 1303 Keyword recognition means 1304 Voice analysis means 1305 Voice conversion means 1306 Customer DB
1307 Person in charge DB
1308 Voice DB
200 Communication means (customer side)

Claims (3)

An automatic voice recognition / conversion system used in a service providing system for carrying out voice conversations with a call partner via voice communication means,
The voice automatic recognition / speech conversion system includes:
Voiceprint recognition means having a function of recognizing the voiceprint of the input voice of the other party;
Emotion recognition means having a function of recognizing the other party's emotion from the input voice;
A customer DB that stores customer information indicating the voiceprint, age, and gender of the other party;
A person-in-charge DB that accumulates voice parameters of persons in charge corresponding to the other party;
A voice DB that stores voice parameters corresponding to age, gender, and emotion;
Voice analysis means for receiving a voiceprint recognition result of the voiceprint recognition means and an emotion recognition analysis result of the emotion recognition means, calculating a difference between the voice parameter of the person in charge DB and the parameter of the voice DB, and outputting the difference;
Voice conversion means for receiving the difference extracted by the voice analysis means, converting the voice of the person in charge based on the difference, and outputting the voice to the communication means;
Have
In a call state between the call partner and the person in charge, the emotion recognition unit detects a change in the emotion of the call partner, and the voice analysis unit calculates a voice parameter of the voice DB and a voice parameter of the person in charge. When the difference is calculated, the voice conversion means converts the voice of the person in charge on the basis of the difference in real time. An automatic voice recognition / voice conversion system. The automatic speech recognition / speech conversion system according to claim 1,
The service providing system includes a call center, and the call center includes display means for displaying the analysis result of the voice analysis means. The automatic speech recognition / speech conversion system according to claim 1 or 2,
The voice automatic recognition / speech conversion system includes:
Furthermore, it has keyword recognition means for recognizing a keyword related to re-listening from the input voice of the other party,
When the keyword recognition unit recognizes the keyword and the voice analysis unit detects the keyword, the voice conversion unit changes a parameter value of each attribute according to the keyword, and performs the charge according to the parameter. A voice automatic recognition / conversion system characterized by converting the volume and / or speed of a person's voice.

Images