JP5099218B2 - Problem solving time estimation processing program, processing apparatus, and processing method - Google Patents

Description

  According to the present invention, there is provided a computer for executing a process for estimating a time required for solving a problem inquired by a customer using voice dialogue data in which a dialogue between the customer and an operator who handles the telephone is recorded. It relates to problem solving time estimation processing.

  The contents of customer inquiries to the call center vary from those that can be answered immediately by the responding operator to those that require the operator to call back because it takes time to present the survey results and solutions. There is something of a degree.

  When a call center manager or operator recognizes an inquiry from a customer as a problem, the call center manager grasps the time required to resolve this problem (problem resolution time), and improves the efficiency of customer service and customer satisfaction. There is a demand to plan.

  Conventionally, when there is a customer's incoming call, in addition to the voice dialogue data, the response history information that records the response processing of the operator's response, reception time, investigation, transfer to the secondary line (separate department), etc. A response history management system was established to record callback history information for customer inquiries, and record callback information in association with response history information.

  On the other hand, in the call center, voice conversation data recording all conversation contents is accumulated so that the conversation contents between the customer and the operator can be heard later. The accumulated large amount of spoken dialogue data can be used not only as confirmation data for dialogue content but also as analysis material for various purposes. However, it is necessary to identify the necessary parts according to the purpose of use. is there.

In order to make it possible to identify and reproduce the part of the spoken dialogue data that contains the inquiry content, which is the core part of the response between the operator and the customer, the keywords extracted by voice recognition processing, the operation information on the operator's terminal screen, etc. There is a conventional method in which the voice conversation recording data is given as an index so that it can be used to specify the playback start position when the voice conversation data is played back (for example, see Patent Document 1).
Japanese Patent Laid-Open No. 11-25112

  If the customer's inquiry content is recognized as a problem and the time required to resolve this problem is known, if the customer can respond within the response time for incoming calls, the start time and end time of the response can be set. The response time can be regarded as the time required for resolution by recording.

  However, depending on the problem, there is a case where the operator has to call back to the customer after completing the response and getting a solution to the problem. When answering with this callback, a system that can estimate the time required from problem recognition to answer presentation has not been realized.

  Conventional response history management systems simply manage response history information in a database, and the callback time is recorded along with information such as the contents of each response, reception time, end time, person in charge, etc. Operation and processing to record the information about the callback in the history information.

  In addition, there is no technology for estimating the problem solving time from the voice conversation data because the voice conversation data recorded and stored has no correspondence between the customer's incoming call dialogue and the operator's callback dialogue.

  An object of the present invention is to provide a processing method capable of estimating a problem solving time only from voice conversation data with a recording time.

  First, the principle of the present invention will be described.

  In the spoken dialogue data, a problem appears as a part where a customer utters a question, and a solution to this problem appears as a part where an operator who receives the customer's utterance utters an answer. Therefore, the time from the recognition of the problem to the completion of the presentation of the answer is regarded as the problem solving time from the start of the dialogue in which the customer asks the question until the end of the dialogue in which the operator answers the customer. Can do.

  By the way, in general, a speaker who speaks predominantly among speakers during a conversation tends to continuously utter with a certain amount of speech, compared to a speaker. In the dialogue between the customer and the operator, when the customer makes an inquiry, it is assumed that the customer is leading and speaking in advance. In the dialogue in which the operator calls back to the customer and answers, it is assumed that the operator is leading and speaking after the dialogue including the customer's question.

  In addition, when there is only a dialogue that the customer leads and speaks first, and there is no dialogue that the operator leads, the response is answered during the previous customer's dialogue. The situation is assumed.

  Therefore, in the present invention, voice conversation data that can be regarded as a dialogue with the same customer and operator is extracted from a set of voice dialogue data between the customer and the operator, and the type of dialogue is determined by specifying the leading speaker of each dialogue. Determine whether “question” or “non-question”. Furthermore, voice dialogue data having a correspondence relationship such that there is a “non-question” dialogue following the “question” dialogue is extracted, and it is judged as “question and answer”. Estimate the time until the end of the dialogue as the problem solving time.

  Specifically, the program disclosed discloses a computer as a set of voice interaction data with recording time, which is composed of 1) a first channel in which operator's voice is recorded and a second channel in which customer's voice is recorded. A processing unit for input; and 2) for each of the voice conversation data, identify a lead utterance section in which one speaker continuously speaks for a predetermined length or longer with a louder voice than the other speaker; , Identify the earliest initiative utterance section as the preceding initiative utterance section, and if the preceding initiative utterance section is included in the second channel, the type of the voice dialogue data is set as a question, and the preceding initiative utterance section A processing unit that sets the type of the voice conversation data as non-question when the first channel is included in the first channel, and 3) for each of the voice conversation data, speaker characteristics of the voice data of each channel are determined in advance. A processing unit that learns by the machine learning processing method, and 4) a voice in which the speaker characteristics of the first channel and the second channel of the voice interaction data are within a certain similar range based on the speaker characteristics. A processing unit that collects dialogue data and sets it as a similar speaker set; and 5) arranges the voice dialogue data according to the recording time for each similar speaker set, takes out one voice dialogue data whose type is a question, A processing unit that associates the voice dialogue data that is a question with the voice dialogue data that is temporally succeeding and is a non-question type, and 6) the earliest recording start time of the pair of the voice dialogue data that is associated It is for functioning as a processing unit that calculates the time from the recording end time to the last recording end time and sets it as the problem solving time.

  The computer on which the disclosed program is installed and executed first inputs a set of voice interactive data with recording time, which is composed of a first channel in which the operator's voice is recorded and a second channel in which the customer's voice is recorded. To do. For each input voice dialogue data, one speaker speaks continuously for a predetermined length or longer from the voice data of the first and second channels. Identify the interval. In addition, when the earliest initiative utterance section is identified as the preceding initiative utterance section, and when the preceding initiative utterance section is included in the second channel, the type of the voice conversation data is set as a question, and when it is included in the first channel Set the type as non-question.

  Further, for each voice conversation data, the speaker characteristics of the voice data of each channel are learned by a predetermined machine learning processing method, and based on the learned speaker characteristics, the first channel and the second channel of the voice conversation data are learned. Speech dialogue data in which the speaker characteristics of the two channels are within a certain similar range is collected to form a similar speaker dialogue set.

  Furthermore, for each of similar speaker conversation sets, the voice conversation data constituting the set is arranged according to the recording time, one voice conversation data whose type is a question is taken out, and the type of temporal conversation that follows this voice conversation data is Correlate with non-questional voice dialogue data.

  Further, the time from the earliest recording start time to the last recording end time of the set of the spoken dialogue data associated is calculated as the problem solving time.

  As a result, voice dialogue data with recording time is also available for cases in which the operator does not answer the customer's question while responding to the incoming customer, and the operator calls back and answers after conducting a survey. The problem solving time can be calculated using only

  Further, the disclosed processing apparatus is a processing apparatus having processing units for realizing processing that the program causes the computer to execute.

  Further, the disclosed processing method is a processing method configured by each processing step executed by the computer by the program.

  According to the present invention, it is possible to automatically estimate the time required for solving a problem inquired by a customer from voice dialogue data with recording time information accumulated for response confirmation.

  In particular, even when a response is made by a callback, work / processing for associating the callback information with the customer's incoming call information is not required, and the problem solving time can be estimated easily.

  Based on the measured problem solving time, it is possible to estimate the problem solving time at the individual level of response, and it can be used as data for call center business analysis and planning.

  In addition, the problem solving time for each question content is analyzed by combining the processing method for estimating customer inquiry tendency from voice dialogue data for another application made by the same person with the disclosed problem solving time estimation process. It can be used as various analysis materials.

It is a figure which shows the structural example of a problem solution time estimation processing apparatus. It is a figure which shows the processing flow of the outline | summary process of a problem solution time estimation processing apparatus. It is a figure which shows the example of the content of the operator's and customer's utterance used as voice interaction data. It is a figure which shows the data structural example of voice interaction data. It is a figure which shows the more detailed process flow of the process of step S2. It is a figure which shows the more detailed process flow of the process of step S21. It is a figure which shows the audio | voice power information example of audio | voice dialog data (recording 1). It is explanatory drawing of total reception time. It is a figure which shows the example of total response time of each voice dialogue data (recording 1, 2, ...). It is explanatory drawing of a preceding speaker (preceding speech channel). It is a figure which shows the example of the preceding speaker (preceding speech channel) of each voice dialogue data (recording 1, 2, ...). It is explanatory drawing of a prior | preceding initiative speech person (preceding initiative speech channel). It is a processing flow figure (the 1) which asks for a leader initiative utterer and a precedence initiative utterance time. It is a processing flowchart (the 2) which calculates | requires a leading initiative utterer and precedence leading utterance time. It is a figure which shows the example of a calculation result of the prior | preceding initiative speech time of audio | voice dialog data (recording 1). It is a processing flowchart of the process which determines a question speech part by a rule base. It is a figure which shows the example of the data input. It is a figure which shows the example of a rule. It is a figure which shows the processing flow of the learning data extraction in the process of step S3. It is a figure which shows the processing flow of the learning process in the process of step S3. It is a figure which shows the processing flow of the similarity calculation process in the process of step S3. It is a figure which shows the example of reliability information. It is a figure which shows the calculated similarity matrix example. It is a figure which shows the more detailed process flow of the process of step S4. It is a figure which shows the example of a setting of "indefinite" to a similarity matrix. It is a figure which shows the example of a setting of "no similarity" to a similarity matrix. It is a figure which shows the example of a correction coefficient and a normalization similarity. It is a figure which shows the calculated normalization similarity matrix example. It is a figure which shows the calculated average similarity matrix example. It is a figure which shows the example of determination of the audio | voice data made into a similar speaker group. It is a more detailed process flowchart of the process of step S5. It is a figure which shows the example of the data input into the same speaker process. It is a figure which shows the example of matching of audio | voice data. It is explanatory drawing of problem solution time.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Problem solution time estimation processing apparatus 11 Data input part 13 Dialog type estimation part 15 Similar speaker set calculation part 17 Dialog data matching part 19 Problem solution time calculation part 3 Voice dialog data (with time information)
4 Voice power information 5 Problem solving time

  FIG. 1 is a diagram illustrating a configuration example of a problem solving time estimation processing apparatus.

  The problem solving time estimation processing device 1 solves a problem presented by a customer from voice dialogue data recorded by an operator and the customer's dialogue recorded at a call center that receives and answers a customer's inquiry. The problem solving time required is an estimation device.

  The problem solving time estimation processing device 1 includes a data input unit 11, a dialogue type estimating unit 13, a similar speaker set calculating unit 15, a dialogue data associating unit 17, and a problem solving time calculating unit 19.

  The data input unit 11 inputs a set of voice conversation data 3.

  The dialogue type estimation unit 13 is a section in which one speaker continuously speaks with a louder voice than the other speaker for a predetermined length or longer from each of the inputted voice conversation data 3 (hereinafter referred to as a lead utterance section). ), And the leading utterance interval that appears first is defined as the preceding initiative utterance interval. Then, it is determined whether the preceding initiative utterance section is on the speaker side of the customer or the operator. If the speaker in the preceding initiative section (preceding initiative speaker) is a customer (second channel), the type of the voice conversation data is set to “question”. If the preceding initiative speaker is an operator (first channel), the type of the voice interaction data 3 is set to “non-question”.

  The similar speaker set calculation unit 15 learns the speaker characteristics of each of the customer and the operator for each of the inputted voice conversation data 3 by a predetermined machine learning processing method. Further, based on the learned speaker characteristics, the voice conversation data 3 in which the speaker characteristics of both the customer and the operator are within a certain similar range are collected and classified, and the classification of the voice conversation data 3 is similar to the similar conversation. A set of people.

  The dialogue data associating unit 17 arranges the voice dialogue data 3 constituting the similar speaker dialogue set according to the reception time (recording time) for each of the similar speaker dialogue sets, and applies the corresponding correspondence conditions. Correlate speech dialogue data.

  For example, the following conditions are prepared.

  ・ Associating with the voice conversation data of type “question” as the starting point.

  • The types “question” and “non-question” are consecutive in time order.

  • The types “non-question” and “non-question” are consecutive in time order.

  ・ Sets of types other than the above are not consecutive.

  Under this condition, one voice conversation data 3x whose type is “Question” is extracted, and it is checked whether there is voice dialog data 3y whose type is “Non-Question” that is subsequent to the extracted voice conversation data 3x. . If there is the corresponding voice dialogue data 3y, the voice dialogue data 3x and 3y are associated with each other. If there is no corresponding voice dialogue data, no association is performed.

  The problem solving time calculation unit 19 calculates the time from the earliest recording start time to the last recording end time of the associated voice conversation data 3x, 3y as the problem solving time 5, and outputs the problem solving time 5 To do.

  FIG. 2 is a diagram showing a processing flow of the problem solving time estimation processing apparatus 1.

  Step S1: The data input unit 11 of the problem solving time estimation processing device 1 inputs a set 3 of spoken dialogue data.

  FIG. 3 shows an example of the contents of the speech of the operator and the customer as the voice dialogue data 3, and FIG. 4 shows the data structure of the voice dialogue data 3.

  The voice dialogue data 3 is voice data obtained by recording the voice of the dialogue between the operator and the customer as shown in FIG. 3 using a known recording device. The voice interaction data 3 is composed of two channels. Voice data of the operator is recorded on the first channel (for example, L channel), and voice data of the customer is recorded on the second channel (for example, R channel) independently.

  At the beginning of the voice dialogue data 3, as data index, data identification information (recording 1), operator name (Yamada), recording date (05/10/11), recording start time (15:25:20) The recording end time (15:31:32) is stored.

  The recording start time / end time is used as the response start time / end time.

  Step S2: The dialogue type estimation unit 13 estimates the dialogue type of the input voice dialogue data 3.

  Here, as the dialogue type, “question” indicating a dialogue in which a customer asks a question in the dialogue and “other (non-question)” indicating a dialogue that is not a customer question are set in advance.

  Specifically, the conversation type estimation unit 13 continuously utters the voice conversation data 3 from the voice data of each of the L / R channels with a louder voice than the other speaker for a predetermined length or longer. The channel including the preceding initiative utterance section, which is the leading utterance section that appears first, is determined, and the channel is set as the preceding initiative utterer (preceding initiative utterance channel).

  More specifically, the conversation type estimation unit 13 calculates a sound power value for each predetermined unit section for each channel of the voice conversation data 3, and generates voice power information in which the power values are arranged in time series. . Then, the audio power information of each channel is compared in time series from the beginning, and in each predetermined determination unit section, a channel whose sum or percentage of the determination unit section of the power value is a larger value is determined. It is determined that the speaker is the lead speaker in the unit section.

  Further, the leading speaker in the determination unit section closer to the head in time series is identified as the preceding leading speaker. The determination unit section of the leading speaker that is continuous from the determination unit section of the preceding initiative speaker and that is the same as the preceding initiative speaker is defined as the preceding initiative section.

  Thereafter, when the preceding initiative utterance section is included in the R channel, the type of the voice conversation data 3 is set to “question”, and when the preceding initiative utterance section is included in the L channel, the voice conversation data 3 is set. Set the type of "Non-question".

  Details of the dialogue type estimation unit 13 process will be described later.

  Step S3: The similar speaker set calculation unit 15 learns the speaker characteristics of the voice data of each channel for each of the voice conversation data 3 by a predetermined processing method.

  Step S4: Further, the similar speaker set calculation unit 15 obtains a similarity relationship between the voice conversation data 3 for each of the L channel and the R channel of the voice conversation data 3 based on the speaker characteristics obtained by the learning process. , Voice dialogue data 3 having speaker characteristics within a certain similar range for both the L channel and the R channel are collected and classified to form a similar speaker dialogue set.

  As a result, the set of voice dialogue data 3 is classified into dialogues between the same customer and the operator.

  Step S5: The dialogue data associating unit 17 arranges the voice dialogue data 3 according to the recording time (for example, the recording start time) for each similar speaker dialogue set, and extracts one voice dialogue data whose type is “question”. . Then, voice dialogue data satisfying a predetermined condition is extracted and associated. For example, if there is voice dialogue data of the type “non-question” that follows in time, it is extracted and associated.

  As a result, among a plurality of dialogues between the same customer and the operator, a dialogue asked by the customer is associated with a dialogue of the operator's answer to the question.

  Step S6: The problem solving time calculation unit 19 calculates the time from the recording start time of the voice dialogue data of the previous dialogue of the associated voice dialogue data to the recording end time of the subsequent voice dialogue data to calculate the problem solving time. And

  In addition, the problem solving time calculation unit 19 takes out the voice conversation data 3 that is not associated and the type is a question, calculates the time from the recording start time to the recording end time of the voice conversation data, and calculates the problem solving time. And

  As a result, it is possible to estimate the problem solving time of the case where the customer's question is called back and answered and the case where the question was answered during the answering.

Hereinafter, each processing of the processing flow of FIG. 2 performed by the problem solving time estimation processing device 1 will be described in more detail.
[Dialogue type estimation process]
FIG. 5 is a diagram showing a more detailed processing flow of the processing in step S2.

  Step S20: Divide the voice interaction data 3 into predetermined unit intervals. The unit interval is, for example, a value of 1 to 2 seconds.

  Step S21: The average of the power values of the voices in each unit section is obtained and converted to voice power information 4 that is a continuation of time-series power values.

  The audio power information 4 is bit string information obtained by converting an average value (power) of audio data of each channel in a predetermined unit section into a bit string using a predetermined threshold th and arranging them in time series. Therefore, if the voice power of the utterance is greater than or equal to a certain threshold th, “1” is stored in the bit, otherwise it remains “0”.

  FIG. 6 shows a processing flow of the generation processing of the audio power information 4 in step S21.

  A Fourier transform process is applied to each channel of the voice interaction data 3 to obtain a column of [power, pitch] (step S111). Further, a unit section m which is the minimum time unit of the power train is determined (step S112). As the voice power information 4, an average power value is obtained for each unit section m from the beginning of the voice conversation data 3. If the average power value is equal to or greater than the threshold th, “1” is indicated. If the average power value is less than the threshold th, “0” is indicated. A bit string to which is added is output (step S113).

  FIG. 7 is a diagram showing the voice power information 4 of the voice conversation data (recording 1) 3. The voice power information 4 shown in FIG. 7 represents a bit string to which a value “1” is assigned between the utterance start time and the utterance end time in the format of “utterance start: utterance end”. For example, in the case of the unit interval m = 1 second, [utterance start = 0: utterance end = 3] is the interval in which the value “1” is given from the start 0 second to 3, ie, the threshold th It means the time when there was an utterance with the above size.

  Step S22: From the converted voice power information 4, the total response time, the preceding speech channel, the preceding initiative speaker (channel), and the precedence initiative speech time are acquired as attribute information.

  The total response time indicates the total actual dialogue time of the voice dialogue data 3. As shown in FIG. 8, it is obtained by the difference between the recording start time and the end time of the dialogue of the index information of the voice dialogue data. FIG. 9 is a diagram showing an example of the total response time of each of the voice conversation data (recordings 1, 2,...) 3.

  The preceding utterance channel indicates a channel in which an utterance precedes in the dialogue between the customer and the operator. In the bit string of the power value of the audio power information 4, the channel having the earliest unit section in which “1” is assigned to the bit is defined as the preceding speech channel. The values of the preceding speech channel are “L”, “R”, and “LR”.

  In the voice dialogue data 3 recorded at the call center, generally, the recipient of the telephone call starts a dialogue, that is, speaks first. Therefore, in the case of a customer-side call at the time of a normal inquiry, the first utterance is an operator. Conversely, when the operator calls back to the customer, the operator calls and the first utterance is the customer. In general, callback conversations rarely include customer questions, so by identifying which channel the operator or customer's voice was recorded on corresponds to the preceding speech channel, the operator's call You can specify the back-up dialogue.

  In the bit sequence of the audio power information 4 shown in FIG. 10, the unit interval in which “1” is assigned to the bit sequence in the L channel = 0, and the unit interval in which “1” is assigned to the bit sequence in the R channel = 3. Speech channel = L. FIG. 11 is a diagram showing the preceding speech channel of each of the voice conversation data (recordings 1, 2,...) 3.

  The leading initiative utterer (preceding initiative utterance channel) is the initiative utterer (channel) in the determination unit interval closest to the head among the initiative utterers in the predetermined determination unit interval.

  The conversation type estimation unit 13 sets a channel having a large total number (or a high ratio) of unit sections in which a power value bit of the voice power information 4 is “1” within a predetermined determination unit section as a leading speaker. judge. Then, the leading utterer in the determination unit section closest to the head (the first determination unit section in the time series) is specified as the preceding initiative utterance.

  Furthermore, in the voice power information 4 of the channel set as the preceding speech channel, from the unit interval in which “1” is first added to the power value, the unit determination interval in which the preceding initiative speech channel is determined as the lead speaker The continuation is the lead-led utterance time.

  FIG. 12 is a diagram for explaining the preceding initiative utterer and the precedence initiative utterance time.

  The dialogue type estimation unit 13 performs a determination process by shifting a window indicating the range of a unit section that is a target of a predetermined determination process by a predetermined movement unit. When the power value unit interval m = 1 second, the window size n of the processing corresponding to the unit determination time is 15 seconds (unit interval), and the window shift unit k is 3 seconds (unit interval). The number of unit sections to which “1” is assigned as the power value for each channel is calculated, and a channel with a large number of unit sections is determined as the leading speaker. Furthermore, within the window size n shifted by the movement unit (size) k = 3 seconds, similarly, a channel having a large number of unit sections of “1” is determined as the lead speaker.

  In FIG. 12, “R channel” is determined as the leading speaker in the first to fifth determination processes, “L channel” is determined in the sixth determination process, and “LR” is determined in the seventh determination process. Yes. Therefore, the “R channel” determined as the leading speaker in the earliest determination unit section is determined as the preceding leading speaker (preceding leading utterance channel).

  Next, in the L channel specified as the preceding speaker channel, the unit in which the preceding initiative utterance channel is determined as the initiative speaker from the first unit determination section in which “1” is assigned to the power value bit. The continuous section of the determination section is set as the preceding initiative utterance time.

  Here, when the lead speaker changes from the R channel to the L channel, a continuous section up to a unit section plus half of the window size n at that time is calculated as the preceding lead speech period.

  FIG. 13 and FIG. 14 are processing flowcharts for obtaining the preceding initiative utterer and the precedence initiative utterance time.

  The conversation type estimation unit 13 selects the L channel specified as the preceding speech channel (step S131). A window size n is set (step S132), and a pointer is set at the head of the bit string of the audio power information (step S133).

  The number of unit sections in which the bit on the L channel side is “1” in the window is calculated as a value A (step S134). Further, the number of unit sections in which the bit on the R channel side is “1” in the window is calculated as a value B (step S135).

  It is determined whether the value A is greater than the value B (step S136). If the value A is greater than the value B, the lead speaker = L channel is set (step S137). If the value A is not greater than the value B, it is further determined whether or not the value A is equal to the value B (step S138). If the value A is equal to the value B, the lead speaker = LR channel is set (step S139). . If the value A is not equal to the value B, the lead speaker = R channel is set (step S1310).

  Then, a set of [pointer position, initiative speaker value] is output (step S1311).

  Next, the window is shifted by the movement unit k (step S1312), and if the window has reached the end of the bit string of the audio power information 4 (FIG. 14: step S1313), the process proceeds to step S1314. If the end of the bit string of information 4 has not been reached, the process returns to step S134. In the process of step S1314, the initiative speaker value whose pointer position is “0” is set as the value of the preceding initiative speaker.

  Then, a unit interval range (L) in which the values of the preceding initiative speaker and the initiative speaker continuously take the same value is obtained (step S1315). The section from the pointer position = 0 to the pointer position = L is converted into the utterance time and is set as the preceding initiative utterance time (step S1316).

  FIG. 15 is a diagram illustrating a calculation result of the preceding initiative utterance time of the voice conversation data (recording 1) 3. In FIG. 15, the start second indicates the start position of the window, and the window size indicates the window size n. The lead channel is the channel determined to be the lead speaker, and the L ratio and the R ratio indicate the number of unit sections to which “1” is assigned in the window.

  In the voice conversation data (recording 1) 3, the preceding initiative speaker (channel) = R channel, and the precedence initiative speech time = 55.5 seconds.

  Step S23: The dialogue type estimation unit 13 determines the question utterance unit from the preceding initiative utterer (channel) and the precedence initiative utterance time. For example, when the preceding initiative utterance channel is the R channel, that is, the channel in which the customer's voice is recorded, the time corresponding to the precedence initiative utterance time is specified as the question utterance section.

  FIG. 16 is a process flow diagram for determining a question utterance unit based on a rule base.

  The dialogue type estimation unit 13 sets a set of [preceding utterer (channel), preceding led utterer (channel), preceding led utterance time, total response time] for the speech target data to be determined as shown in FIG. Input (step S141).

  Then, based on the rule base shown in FIG. 18, the determination processing in steps S142 to S147 is performed.

  In the rule base of FIG. 18, the following determination conditions are defined.

Rule 1: If the preceding speaker is a leading initiative speaker, “reject”;
Rule 2: If the preceding speaker = LR, “reject”;
Rule 3: If “rejection-led” speaker = L or “first-led” speaker = LR, “reject”;
Rule 4: If the total response time is 1/3 or less of the average response time, “reject”;
Rule 5: If the lead-led utterance time is 5 seconds or less, “reject”;
Initial value: If none of rule 1 to rule 5, it is “accept”.
Here, “reject” = no question utterance part exists, and “accept” = preceding initiative utterance part is a question utterance part.

  It is determined whether the input of step S141 corresponds to rule 1 (step S142). If it corresponds to rule 1, it is further determined whether it corresponds to rule 2 (step S143). If it corresponds to rule 2, Further, it is determined whether it corresponds to rule 3 (step S144). If it corresponds to rule 3, it is further determined whether it corresponds to rule 4 (step S145). (Step S146), and if it falls under rule 5, it is determined that there is no question utterance part (reject) (step S147). On the other hand, if none of the rules 1 to 5 is applicable, it is determined that the question utterance part is included (step S148).

  By this determination processing, it is determined that, for example, the voice dialogue data of recording 1 and recording 2 includes the question utterance part (accept) among the voice dialogue data of FIG. It is determined that the question utterance part is not included (reject).

  Step S24: The dialogue type estimation unit 13 determines whether or not a question utterance unit exists in the voice dialogue data 3.

  Step S25: When it is determined as “accept” in the process of Step S24, that is, when a question utterance part exists in the voice dialogue data 3 (YES in Step S24), the dialogue type of the voice dialogue data 3 is set to “ “Question”.

Step S26: When it is determined as “reject” in the process of Step S24, that is, when there is no question utterance part in the voice dialogue data 3 (NO in Step S24), the dialogue type of the voice dialogue data 3 is “ Other ”.
[Speaker feature learning process]
The similar speaker set calculation unit 15 extracts voice data for each channel recorded with a certain power and length from the voice dialogue data as learning data used for the speaker feature learning process, and each of the extracted voice data The speaker feature is learned about and the similarity of the speaker feature is calculated for all speech data. The similarity is calculated by the brute force method of audio data.

  FIG. 19 is a diagram showing a process flow of learning data extraction in the process of step S3.

  Step S300: The similar speaker set calculation unit 15 applies a Fourier transform process to each channel of the voice conversation data 3 to obtain a column of [power, pitch].

  Step S301: A unit interval m which is the minimum time unit of the power train is determined.

  Step S302: For the voice data of each channel of the voice interaction data 3, an average power value is obtained for each unit interval m from the head, and a portion where the average power value is equal to or greater than the threshold th2 is output. Audio data corresponding to the output location is stored in the audio set A.

  Step S303: The total recording time of the output audio data is recorded in association with each audio data.

  FIG. 20 is a diagram showing a processing flow of the learning process in the process of step S3. The following processing is performed for each piece of voice data in the voice set A.

As long as the learning processing is known, any learning processing method may be used. For example, Mahalanobis distance determination is used. The decision process based on Mahalanobis distance is detailed in the reference (PC Mahalanobis, "On the generalized distance in statistics", Proceedings of the National Institute of Science of India, 12 (1936) 49-55, 1936).
Step S310: The similar speaker set calculation unit 15 extracts one piece of voice data from the voice set A.

  Step S311: The speaker features are learned using the extracted voice data, and the learning results (speaker feature data set) are stored in the learning set B.

  Step S312: The used voice data is removed from the voice set A.

  Step S313: If there is remaining voice data in the voice set A, the process returns to step S310, and if all voice data is used, the process is terminated.

  FIG. 21 is a diagram illustrating a processing flow of similarity calculation processing in step S3. The following processing is performed for each piece of voice data in the voice set A.

  Step S320: The similar speaker set calculation unit 15 extracts one speaker feature data set from the learning set B.

  Step S321: The similarity to all the audio data of the audio set A is calculated.

  When calculating similarity, the total learning target time and average similarity of each voice data is calculated as reliability information. FIG. 22 shows the total learning target time and average similarity of speech data (A, B, C,...) Constituting speech set A.

  The total learning target time is the recording time of voice data used to learn speaker characteristics. The average similarity is an average value of similarities when outputting similarities with other audio data.

  Step S322: The speaker feature data set used is removed from the learning set B.

  Step S323: If there is any remaining speaker feature data set in the learning set B, the process returns to step S320, and if all speaker feature data sets are used, the process is terminated.

  FIG. 23 is a diagram illustrating an example of the calculated similarity matrix. In this similarity matrix, the similarity is “0” as the maximum (minimum distance), and the greater the value, the less similar.

  The column A of the similarity matrix shown in FIG. 23 is the similarity of the other audio data (B, C,...) With respect to the audio data A, and the audio data A includes the audio data A, B, C, D, E,..., And “0, 30, 1500, 25000, 230,.

[Similar speaker set calculation]
The similar speaker set calculation unit 15 calculates speaker authentication, that is, a set of voice data that can be regarded as the same speaker (similar speaker set) based on the similarity of each voice data. Here, in order to improve the determination accuracy, a similarity correction process is performed.

  FIG. 24 is a diagram showing a more detailed processing flow of the processing in step S4.

  Step S40: The similar speaker set calculation unit 15 determines a threshold th3 for determining speech data regarded as a similar speaker candidate. For example, the threshold th3 is set to “similarity = 1000”.

  Step S41: The average similarity of all audio data is obtained.

  Step S42: “Undetermined” is set for the audio data lower than a certain level with respect to the average similarity, and is excluded. When the average similarity is “23000000”, “undefined” is set to the audio data “1/4 or less of the average similarity” as shown in FIG.

  Step S43: Among the voice data, “no similarity” is set for two voice data whose speaker characteristics on the L channel side (operator) of the corresponding voice dialogue data are not regarded as the same speaker.

  Assuming that the L channel audio data of the audio data B and the audio data D are not in a certain similar range, “no similarity (−)” is set in the corresponding item as in the similarity matrix shown in FIG.

  Step S44: The similarity of the audio data is corrected according to the corresponding average similarity value, and the correction coefficient and the normalized similarity are calculated as shown in FIG.

Correction coefficient = (average similarity of each voice conversation data / average similarity of the whole voice conversation data) squared normalized similarity = original similarity * correction coefficient Thus, as shown in FIG. The correction coefficient of each voice interaction data is calculated, and the normalized similarity matrix of each voice data shown in FIG. 28 is obtained.

  Step S45: The pair similarity between the audio data to be processed and other audio data is calculated as an average value of the normalized similarity of each audio data.

  That is, the similarity between the audio data A and B is the average of the similarity between the audio data A and the audio data B (A → B similarity) and the similarity between the audio data B and the audio data A (B → A similarity). And

  Further, when one of “A → B similarity” and “B → A similarity” is “indefinite”, it is determined as the non-indeterminate similarity. If both “A → B similarity” and “B → A similarity” are “undefined”, it is determined that they are not the same speaker.

  FIG. 29 shows an example of the average similarity matrix of each voice interaction data.

  Step S46: Using the average similarity matrix of the voice conversation data, it is determined whether or not they are the same speaker. If the average similarity of the voice data is equal to or greater than the threshold th3 (= 1000), the speaker is not the same speaker (1), and if it is less than the threshold th3, the speaker is the same speaker (0).

  As shown in FIG. 30, for the voice data A, the voice data B, C, E can be regarded as the same speaker, and a similar speaker set {A, B, C, E} is calculated.

Similarly, for voice data B, voice data E can be regarded as the same speaker, for similar speaker set {B, E}, for voice data C, voice data D can be regarded as the same speaker. A similar speaker set {C, D} is calculated.
[Interaction data mapping process]
The dialogue data association unit 17 sets the following conditions in advance as conditions for associating the voice dialogue data.

  ・ Association is started from the data of the dialogue type “question”.

  • The dialogue types “Question” and “Others” continue in chronological order.

  • The dialogue types “Other” and “Other” continue in chronological order.

  -Other than the above types of dialogue types are not consecutive.

  FIG. 31 is a more detailed process flow diagram of the process of step S5.

  Step S50: The dialogue data associating unit 17 leaves the voice data that satisfies the condition for the voice data constituting the similar speaker set based on the dialogue type of the voice data.

  Assume that the association processing is performed based on the determination result of the same speaker shown in FIG.

  If the similar speaker set {A, B, C, E} is targeted, since the voice data A is a “question” from the conversation type shown in FIG. 32B, this is the starting point. Since the voice data B and C are “questions” and do not satisfy the condition, they are excluded. The voice data E is “other” and satisfies the condition, and is left, and {A, E} is left in this similar speaker set.

  For the similar speaker set {B, E}, since the voice data B is “question”, the voice data E is “other” and satisfies the condition. However, since the voice data E has a higher similarity to the voice data A, the voice data E is excluded and {B} is left in the similar speaker set.

  For the similar speaker set {C, D}, since the voice data C is a “question”, the voice data D is “other” and satisfies the condition since it is the starting point. Therefore, {C, D} is left in this similar speaker set.

  Step S51: Output voice dialogue data corresponding to the voice data left in each similar speaker set in time order.

  From the processing result of step S50, when the recording time of each voice conversation data is in the order shown in FIG. 32C, “voice data A → E”, “voice data B”, “voice data” shown in FIG. Three associations “C → D” are output.

[Problem solving time calculation process]
The problem solving time calculation unit 19 obtains the output of the dialogue data association unit 17 and calculates the problem solving time based on the recording start time and the recording end time recorded in the header of each voice dialogue data.

  As shown in FIG. 34A, for “voice data A → E”, the time t1 from the recording start time of the voice dialogue data A to the recording end time of the voice dialogue data E is set as the problem solving time. Similarly, as shown in FIG. 34 (B), for “voice dialogue data C → D”, the time t2 from the recording start time of the voice dialogue data C to the recording end time of the voice dialogue data D is shown in FIG. As shown in (2), for “voice dialogue data B”, the time t3 from the recording start time to the recording end time of the voice dialogue data B is set as the problem solving time.

  Although the present invention has been described above with reference to the embodiments, it is obvious that the present invention can be variously modified within the scope of the gist thereof.

  The problem solving time estimation processing apparatus 1 can be realized as a computer program, and this program can be stored in a suitable recording medium such as a portable medium memory, a semiconductor memory, or a hard disk that can be read by a computer. It can be provided by being recorded on these recording media, or can be provided by transmission / reception using various communication networks via a communication interface.

Claims (9)

A problem solving time estimation processing program for estimating a problem solving time required for solving a problem presented by a customer from voice dialogue data in which a dialogue between an operator and a customer is recorded.
Computer
A processing unit for inputting a set of voice conversation data with recording time, which is composed of a first channel in which the voice of the operator is recorded and a second channel in which the voice of the customer is recorded;
For each of the voice dialogue data, a leading utterance section in which one speaker continuously speaks for a predetermined length or longer with a louder voice than the other speaker is identified, and the leading utterance existing first When the section is identified as a leading initiative utterance section, and when the preceding initiative utterance section is included in the second channel, the type of the voice interaction data is set as a question, and the preceding initiative utterance section is included in the first channel A processing unit for setting the type of the voice interaction data as non-question,
For each of the voice interaction data, a processing unit that learns speaker characteristics of the voice data of each channel by a predetermined machine learning processing method;
Based on the speaker characteristics, a processing unit that collects voice conversation data in which the speaker characteristics of the first channel and the second channel of the voice conversation data are within a certain similar range to form a similar speaker set;
For each set of similar speakers, the voice dialogue data is arranged according to the recording time, one voice dialogue data whose type is a question is taken out, and the voice dialogue data which is the question is temporally succeeding data. Is a processing unit for associating with non-question voice dialogue data;
A processing unit that calculates a time from the earliest recording start time to the last recording end time of the set of the spoken dialogue data associated with each other and sets it as a problem solving time;
A problem solving time estimation processing program characterized in that it functions as: The processing unit for associating the voice dialogue data is:
When there is a plurality of voice dialogue data whose type is a question before the voice dialogue data which is the question, the voice dialogue data having the highest degree of speaker similarity is selected and dealt with. The problem solving time estimation processing program according to claim 1, wherein the processing is performed. The processing unit for calculating the problem solving time is:
The voice dialogue data that is not associated with the voice dialogue data, the type of which is a question, the voice dialogue data is taken out, and the time from the recording start time to the recording end time of the voice dialogue data is calculated. The problem solving time estimation processing program according to claim 1, wherein processing for solving time is performed. A processing device for estimating problem solving time required to solve a problem presented by a customer from voice dialogue data in which a dialogue between an operator and a customer is recorded.
A data input unit for inputting a set of voice conversation data with recording time, which is composed of a first channel in which the operator's voice is recorded and a second channel in which the customer's voice is recorded;
For each of the voice dialogue data, a leading utterance section in which one speaker continuously speaks for a predetermined length or longer with a louder voice than the other speaker is identified, and the leading utterance existing first When the section is identified as a leading initiative utterance section, and when the preceding initiative utterance section is included in the second channel, the type of the voice interaction data is set as a question, and the preceding initiative utterance section is included in the first channel A dialogue type estimation unit that sets the type of the voice dialogue data as non-question,
A speaker feature learning unit that learns speaker features of voice data of each channel for each of the voice conversation data by a predetermined machine learning processing method;
Based on the speaker features, a similar speaker set is obtained by collecting voice dialogue data in which the speaker features of the first channel and the second channel of the voice dialogue data are within a certain similar range. A calculation unit;
For each set of similar speakers, the voice dialogue data is arranged according to the recording time, one voice dialogue data whose type is a question is taken out, and the voice dialogue data which is the question is temporally succeeding data. A dialog data mapping unit that maps non-question voice dialog data,
A problem solving time calculation unit for calculating a time from the earliest recording start time to the last recording end time of the set of the spoken dialogue data set as the problem solving time;
A problem solving time estimation processing device characterized by comprising: The dialogue data associating unit has the highest degree of speaker similarity when there is a plurality of voice dialogue data of which the type is a question before the voice dialogue data which is the question and the temporally subsequent data. The problem solving time estimation processing apparatus according to claim 4, wherein processing for selecting and correlating voice dialogue data is performed. The problem solving time calculation unit extracts voice dialogue data that is not associated with the voice dialogue data and whose type is a question, and records a recording end time from a recording start time of the voice dialogue data. The problem solving time estimation processing device according to claim 4, wherein a process for calculating a time until a problem solving time is calculated. A processing method executed by a computer in order to estimate a problem solving time required for solving a problem presented by a customer from voice conversation data in which a dialogue between an operator and a customer is recorded.
Processing step of inputting a set of voice dialogue data with recording time, which is composed of a first channel in which the voice of the operator is recorded and a second channel in which the voice of the customer is recorded;
For each of the voice dialogue data, a leading utterance section in which one speaker continuously speaks for a predetermined length or longer with a louder voice than the other speaker is identified, and the leading utterance existing first When the section is identified as a leading initiative utterance section, and when the preceding initiative utterance section is included in the second channel, the type of the voice interaction data is set as a question, and the preceding initiative utterance section is included in the first channel A processing step for setting the type of the voice interaction data as non-question,
A processing step of learning speaker characteristics of the voice data of each channel for each of the voice conversation data by a predetermined machine learning processing method;
Based on the speaker characteristics, processing steps for collecting voice conversation data in which the speaker characteristics of the first channel and the second channel of the voice conversation data are within a certain similar range to form a similar speaker set;
For each set of similar speakers, the voice dialogue data is arranged according to the recording time, one voice dialogue data whose type is a question is taken out, and the voice dialogue data which is the question is temporally succeeding data. Processing steps for associating with non-questional voice interaction data;
A processing step of calculating a time from the earliest recording start time to the last recording end time of the associated voice dialogue data set as a problem solving time;
A problem solving time estimation processing method provided. In the processing step of associating the voice dialogue data, if there is a plurality of voice dialogue data whose type is a question before the subsequent dialogue, the voice dialogue data having the highest degree of speaker similarity is selected and dealt with. The problem solving time estimation processing method according to claim 7, wherein the processing is performed. In the processing step of calculating the problem solving time, voice dialogue data that is not associated with the voice dialogue data and whose type is a question is extracted, and from the recording start time of the voice dialogue data The problem solving time estimation processing method according to claim 7 or 8, characterized in that a process up to a recording end time is calculated to obtain a problem solving time.

Images