JP5385677B2 - Dialog state dividing apparatus and method, program and recording medium - Google Patents

Description

  The present invention relates to an apparatus and method, a program, and a recording medium for classifying voice data under a situation in which two speakers interact with each other according to the state of the conversation.

  In recent years, a mining technique for acquiring knowledge from a large amount of accumulated data has attracted attention. For example, a technique such as text mining is used for the purpose of investigating the general reputation and trends of products from blogs written by unspecified speakers on the Web and free description questionnaires for products.

  One of the text mining techniques includes word ranking and topic classification. For example, when collecting multiple documents written in text, such as free-form questionnaires and blog articles on products, and examining their topic trends, the document frequency ( Word ranking by Document Frequency (hereinafter referred to as DF) is used. DF is a value representing how many documents contain a certain word.

  Such mining technology has been attracting attention in the field of CRM (Customer Relationship Management), and attempts have been made to analyze customer records and develop customer needs and improve CS (Customer Satisfactin). As CRM analysis data, there is a recording of a telephone reception between an operator and a customer in a call center (hereinafter referred to as reception voice).

  Therefore, when analyzing the response speech using voice recognition or manually transcribed documents using word ranking etc., even if the analysis is performed for the entire range from the start to the end of the response speech, the intended call It is difficult to get the contents. Even if it is said to be a response voice in one word, the customer can explain to the operator what he has called, confirm the identity of the customer, and the operator can explain the customer's requirements. A conversation can be divided into several states. Therefore, it is expected that the accuracy of data mining can be improved by dividing the conversation into several states and analyzing each divided state. In other words, it is possible to divide the dialogue state as to whether the customer is in the state of the business, the operator is listening to information from the customer, or the operator is answering. Improvement of data analysis accuracy can be expected.

  As a related prior art, for example, a text segmentation technique for dividing text from the appearance tendency of words disclosed in Non-Patent Document 1 is known. Text segmentation technology is a technology that divides documents such as newspaper articles and novels into groups of meanings. However, no technology has been reported so far that divides and classifies the correspondence state of conversations between two parties such as reception voice.

Marti A. Hearst. Multi-Paragraph Segmentation of Expository Text. 32nd Annual Meeting of the Association for Computational Linguistics. Pp. 9-16. 1994

  When using a conventional text segmentation technique, it is necessary to transcribe the response voice into text once. It takes a lot of cost to manually write the response voice. In addition, when the response voice is recognized and converted into text automatically, there is a concern that the division accuracy may deteriorate due to the influence of misrecognition included in the recognition result.

  The present invention has been made in view of the above points, and is a dialog state dividing apparatus and method capable of reducing the cost of transcription of text and avoiding a reduction in division accuracy due to erroneous recognition. An object of the present invention is to provide a program and a recording medium.

The dialog state dividing device of the present invention includes an utterance section detecting unit, a frame extracting unit, an intra-frame utterance time ratio calculating unit, a frame representative score calculating unit, and a dialog state classifying unit. The utterance section detection unit detects the respective utterance sections of the two parties, using as input the voice data of the conversation between the two parties of the speaker A and the speaker B. The frame extraction unit arranges the respective utterance sections in order of elapsed time and outputs a predetermined number of utterance sections as one frame. The intra-frame utterance time ratio calculation unit calculates the total utterance time of the utterer A or the utterer B in the frame by the total value of the total utterance time of the utterer A and the total utterance time of the utterer B in the frame. An intra-frame speech time ratio R _j that is the divided value is calculated for each frame. The frame representative score calculation unit determines the intra-frame speech time ratio R _j or the smoothed intra-frame speech time ratio R _j as a representative score in the one frame. The dialogue state classification unit classifies each frame into at least three dialogue states by comparing the time ratio of the utterance period with at least two threshold values.

  According to the dialogue state dividing apparatus of the present invention, at least the dialogue state without using the text information is “a state in which the customer is describing the business”, “a state in which the operator is hearing information from the customer”, and “an operator Can be divided into three states: “state that is answering”. Since the conversation state is divided by using the conversation time ratio of the talker, there is no cost for converting the conversation information into text information. In addition, it is not affected by recognition result errors when speech recognition is used for transcription of text.

The figure which shows an example of the conversation of the customer correspondence state in a call center. The figure which shows the function structural example of the dialog state division | segmentation apparatus 100 of this invention. The figure which shows the operation | movement flow of the dialog state division | segmentation apparatus 100. FIG. The figure which shows the function structural example of the utterance area detection part. The figure which shows the function structural example of the speech separation part 20. FIG. The figure which shows the function structural example of the frame extraction parts 11 and 11 '. The figure which shows an example which arranged the utterance area of two persons in order of utterance time. The figure which shows the function structural example of the utterance time ratio calculation part 12 in a flame | frame. The figure which shows notionally an example of the output signal of the frame representative score calculation part 13. FIG. The figure which shows the operation | movement flow of frame representative score calculation part 13 '. The figure which shows notionally an example of the output signal of frame representative score calculation part 13 '. The figure which shows the operation | movement flow of the dialog state division | segmentation part 14. FIG. The figure which shows an example of the result as which the dialog state division | segmentation part 14 operate | moved. The figure which shows the example of the flame | frame which consists of an utterance area of a conflict.

Before describing the embodiments of the present invention, the idea of the present invention will be described.
[Concept of this invention]
The dialog state dividing method of the present invention is an idea of dividing the dialog state by paying attention to which of the two speakers has the initiative of the conversation. FIG. 1 shows an example of a conversation state between a customer and an operator in a call center, for example. The horizontal direction in FIG. 1 is the elapsed time, and the operator's speech section is shown above the center line representing the elapsed time, and the customer's speech section is shown below.

The flow of conversation customers and operators in call centers, the state customer is stated requirements: U R _{(hereinafter,} referred to as state U _R), the state operator is elicit customer information: O H _{(hereinafter,} the state O _H ), and the state in which the operator has answered: O _A (hereinafter referred to as state O _A ). This series of conversation flows can be divided by paying attention to which speaker is talking how much.

Dialog state division method of the present invention determines the ratio of the respective speaker speech time, by the ratio, the state a section customer is speaking longer than the operator U _R, talk time originated extent operators and customers same Is divided into three states, state O _H , and a section in which the operator speaks longer than the customer, state O _A. According to this method, it is not necessary to convert conversation information into text information. Therefore, the cost of converting to text information is unnecessary, and the dialog state can be divided without being affected by the conversion error when converting to text information.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

  FIG. 2 shows a functional configuration example of the dialog state dividing apparatus 100 of the present invention. The operation flow is shown in FIG. The dialog state dividing apparatus 100 includes an utterance section detecting unit 10, a frame extracting unit 11, an intra-frame utterance time ratio calculating unit 12, a frame representative score calculating unit 13, a dialog state dividing unit 14, and a control unit 15. It has. The dialog state dividing apparatus 100 is realized by a predetermined program being read into a computer composed of, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

  The utterance section detection unit 10 receives voice data of conversation between the two parties, and detects the respective utterance sections of the two parties (step S10). The frame extraction unit 11 inputs the utterance interval of one speaker and the utterance interval of the other speaker output from the utterance interval detection unit 10, arranges the respective utterance intervals in order of elapsed time, and collects a predetermined number of utterance intervals. Output as one frame (step S11).

The intra-frame utterance time ratio calculation unit 12 calculates the time ratio of each utterance section in the frame for each frame (step S12). The frame representative score calculation unit 13 determines a representative score _Rj ^ (notation of ^ is correct in the figure) in one frame from the time ratio of the utterance interval in units of utterance intervals (step S13). The dialogue state dividing unit 14 classifies each frame into three dialogue states by comparing the representative score R _j ^ with two threshold values (step S14). The control unit 15 controls the operation of each unit so as to repeat the processing of steps S10 to S14 until all the frames are classified (step S15).

According to the dialog state dividing apparatus 100, a predetermined number of two utterance sections are arranged in order of elapsed time to form one frame, and the time ratio of the two utterance sections is obtained for each frame. Then, the representative score R _j ^ of each frame is determined from the time ratio of the utterance interval of each frame in units of the utterance interval. By comparing the values of the representative score R _{j ^} and two thresholds, it is possible to divide the audio data two parties to talk to the three dialog state condition U _R and State O _H and a state O _A.

  It is not necessary to use text information as in the prior art, so the cost is low, and it is possible to divide and categorize dialog states without being affected by conversion errors that occur when converting voice data to text information. To do.

  Hereinafter, the functional configuration example of each part of the dialog state dividing apparatus 100 will be shown and the operation will be described in more detail.

[Speech section detector]
FIG. 4 shows a functional configuration example of the utterance section detection unit 10. The example shown in FIG. 4 is an example in which the voice data with which the two parties talk is given as a two-channel (stereo signal) signal separated from each other.

  The utterance section detection unit 10 includes power calculation means 101a and 101b and voice section detection means 102a and 102b. The power calculation means 101a and 101b calculate the respective voice powers by using the speech data of one speaker and the speech data of the other speaker as inputs. The voice section detecting means 102a and 102b receive the voice power of each utterance, compare the voice power with a predetermined threshold value, and output a section where the voice power continues for a predetermined time or more as each utterance section. The utterance section detection unit 10 can be realized with the same configuration as what is called a so-called voice switch.

  When voice data for conversation between two parties is given by one channel, it is necessary to separate the utterances of the two parties. FIG. 5 shows a functional configuration example of the utterance separating unit 20 for separating two utterances. The utterance separating unit 20 classifies speakers by recognizing voice data.

  The utterance separation unit 20 includes an AD conversion unit 21, a feature amount extraction unit 22, a speaker classification unit 23, a model parameter recording unit 24, and DA conversion units 25 and 26. The AD conversion means 21 converts voice data, which is an analog signal of one channel with which the two parties talk, into a digital signal. The feature quantity extraction unit 22 extracts the feature quantity of the voice data by converting the voice data converted into the digital signal into a frequency domain signal by, for example, short-time Fourier transform.

  The speaker classification unit 23 recognizes the speaker by comparing the feature quantity with the acoustic model and the language model recorded in the model parameter recording unit 24, and separates the speech. The separated voice data is converted into an analog signal for each speaker by the DA conversion means 25 and 26.

  The operation after the voice data for each speaker is converted into an analog signal is the same as the operation of the utterance section detection unit 10 (FIG. 4). In addition, although the speech separation unit 20 has been described as an example of recognizing a conversation between two parties, the acoustic features of the speaker's voice may be classified using only an acoustic model. Further, if there is a certain difference in the frequency of the voice of the speaker, it is possible to classify the speaker with a simple frequency filter.

[Frame Extraction Unit]
FIG. 6 shows a functional configuration example of the frame extraction unit 11. The frame extraction unit 11 includes an utterance time order arrangement unit 111 and a frame generation unit 112. The utterance time order arrangement unit 111 arranges the utterance sections of the two parties in the order of the utterance start time.

  FIG. 7 shows an example in which two utterance sections are arranged in the order of utterance time. OPE1 enclosed by an ellipse in FIG. 7 means the first utterance section of the operator. Similarly, USR1 means the customer's first utterance section. The speech utterances of the conversation starting with “What is your business today?” Shown in FIG. 7 are arranged in order of elapsed time by the operations of the utterance duration detection unit 10 and the utterance time order arrangement means 111.

The frame generation unit 112 collects the utterance sections arranged in the order of elapsed time and outputs the frames as a single frame F _j (1 ≦ j ≦ N−k + 1), for example, by k = 3, for example. Here, N is the total of the number of utterances of the operator and the number of utterances of the customer in the conversation. The frame generation means 112 generates a frame for the entire call by sliding N at regular intervals, for example, for each utterance. In the example shown in FIG. 7, a frame is formed by sharing two utterance sections. By overlapping frames in this way, an effect of stabilizing the value of the time ratio in the utterance interval can be expected. Note that the frames may be configured such that the speech sections do not overlap at all.

[Intra-frame utterance time ratio calculator]
FIG. 8 shows a functional configuration example of the intra-frame speech time ratio calculation unit 12. The intra-frame utterance time ratio calculation unit 12 includes a utterer utterance time counting unit 121 and an utterance time ratio calculation unit 122. The speaker utterance time totaling unit 121 totals the utterance time for each speaker. In the example of frames _{F 1} shown in FIG. 7, by calculating the speech time OPE1 + OPE2 operator performs an operation to isolate the customer speech time USR1.

Speech time ratio calculating means 122 calculates the frame speech time ratio R _j is an operator and speech time ratio of the customer contained in the frame for each frame F _j in equation (1).

In the example of the frame F ₁ shown in FIG. 7, R _j = (OPE1 + OPE2) / (OPE1 + US)
R1 + OPE2).

If the intra-frame speech time ratio R _j is a value close to 1, it means that the operator has been talking to the customer for a long time within the frame. A value close to 0 means that the customer has been talking to the operator for a long time. A value close to 0.5 means that the operator and the customer are talking for almost the same time.

[Frame Representative Score Calculator]
The frame representative score calculation unit 13 determines a representative score in one frame in units of utterance sections. FIG. 9 conceptually shows an example of the output signal of the frame representative score calculation unit 13. In the horizontal direction of FIG. 9, the speech sections U _i are arranged in the order of elapsed time, and in the vertical direction, the frames F _j are arranged in the order of elapsed time. FIG. 9 shows an example in which one frame is composed of three utterance sections.

  In FIG. 9, the representative score of one frame is determined as the value of the utterance section in the middle of the frame. Which utterance section in the frame is used as the representative score is arbitrary. The frame representative score calculation unit 13 may determine the first utterance section or the last utterance section of the frame as the representative score.

  Further, an average value of time ratios of speech sections of a plurality of frames may be obtained for the purpose of suppressing the variation of the representative score, and the average value may be used as the representative score. FIG. 10 shows an operation flow of the frame representative score calculation unit 13 ′ using the average value of the time ratios of the utterance sections of a plurality of frames as a representative score.

  The frame representative score calculation unit 13 ′ includes storage means for storing, for example, time ratios of three utterance sections in order of elapsed time. This storage means erases the time ratio of the oldest utterance interval every time a new utterance interval time ratio is input. Therefore, the time ratio of the latest three utterance sections is always stored. Since this storage means can be constituted by a general memory circuit, a description of the functional configuration example is omitted.

  In step S130, it is determined whether or not the time ratio for three frames is stored in the storage means. While the time ratio for three frames is not yet stored (N in step S130), the time ratio for each frame is determined as the representative score (step S131).

  When the time ratios for three frames are stored in the storage means (Y in step S130), the average value of the three time ratios is calculated (step S132). The average value is determined as the representative score of the middle frame (step S133). The operation from step S130 to S133 is performed for all frames (N in step S150).

As a result of the above operation, the representative score of each frame shown in FIG. 9 changes as shown in FIG. The frame F _{j + 1} is changed from 0.2 to 0.3, the frame F _{j + 2} is changed from 0.4 to 0.3, and the frame F _{j + 3} is changed from 0.3 to 0.4. The value of the frame F _{j + 4 is} a value before moving average because of the notation in the figure. In this way, the time ratio of the utterance sections of a plurality of frames may be subjected to a moving average to obtain a representative score. By making the representative score an average value, local variation of the representative score can be suppressed.

[Dialogue State Division]
FIG. 12 shows an operation flow of the dialog state dividing unit 14. The dialog state dividing unit 14 classifies each frame into at least three dialog states by comparing the representative score output by the frame representative score calculating unit 13 with at least two threshold values X and Y.

The dialog state dividing unit 14 first compares the representative score R _j ^ for each frame output by the frame representative score calculating unit 13 with the threshold value X. The threshold value X is a predetermined value such as 0.4, which is smaller than 0.5. Representative score _{R j} ^ is is less than 0.4 (Y in step S140), the frame state customer is stated requirements: fall into _{U R} (step S141).

If the representative score R _j ^ is greater than the threshold X, then the representative score R _j ^ is compared with the threshold Y. The threshold Y is a value larger than 0.5, for example, 0.6. If the representative score R _j ^ is larger than the threshold Y (Y in step S142), the frame is classified into a state that the operator has answered: O _A (step S143).

Representative score R _{j ^} is smaller than the threshold value Y (N in step S142), the frame state operator is elicit customer information: is classified into O _H. Operation of classifying the frame can be a method of storing a respective frame in the memory circuits, each state U _R in each _frame, O _A, may be operating subjecting the labels O _H.

As described above, the dialogue state dividing unit 14 operates, whereby each frame can be divided into three dialogue states in units of utterance sections. FIG. 13 shows an example of the division. The horizontal axis represents utterance intervals U _i arranged in the order of elapsed time, and the vertical axis represents the representative score R _j ^ of each frame.

Speech segment _{U 5} until the condition _{U R,} to the speech segment _U 6 _{~U 11} state _{O H,} is the speech segment _{U 12} after being divided into state _{O A.} As described above, since the representative score of each frame is determined in units of utterance sections, the conversation state can be divided in units of utterance sections. Speech segment U ₅ changes from state U _R state O _H, when one frame consists of three speech period corresponds to the speech segment in the middle of the two first frames from the origin in FIG. 13. Which utterance section in this frame is used as the representative score is arbitrary as described above.

  As described above, according to the dialog state dividing device 100, it is possible to divide the dialog state without converting the voice data of the conversation between the two into text information.

[Modification 1]
In the dialog state dividing apparatus 100, the frame extraction unit 11 arranges a predetermined number of the utterance sections detected by the utterance section detection unit 10 in order of elapsed time to be one frame. As the utterance section, “Yes”, “Eh” and the like are also included.

  FIG. 14 shows an example of a frame made up of utterance intervals. The horizontal direction of FIG. 14 represents elapsed time. It starts with the customer's "I moved the other day, so I changed the address." It shows the conversation part where the operator returns a mutual inquiry. If the utterances are arranged in the order of elapsed time, the utterance section of OPE1, OPE2, and OPE3, which is an operator's relationship, is regarded as one frame, and it may be determined that the operator is speaking for a long time. If such a frame is generated, an erroneous determination may be caused, so that the utterance interval of the conflict may be deleted.

  FIG. 6 shows an example of a functional configuration of the frame extraction unit 11 ′ in which the conflict is deleted by a broken line. The frame generation unit 112 ′ of the frame extraction unit 11 ′ includes a conflicting utterance section deletion unit 1120.

  The conflicting utterance section deleting means 1120 deletes, for example, a short utterance section as a conflict. For example, an utterance period of less than 1 second may be regarded as a conflict. In addition, an utterance section in which one side utters while the other starts uttering and ends in a short time may be regarded as a conflict. Further, “Yes” or “Eh” may be recognized as a voice and the utterance section may be deleted.

As described above, according to the dialog state dividing apparatus 100 of the present invention, at least the dialog state without using the text information, “the state in which the customer is describing the business”, “the operator hears the information from the customer”. It is possible to divide the state into three states, that is, a state where the operator is answering and a state where the operator is answering. Since the conversation state is divided by using the conversation time ratio of the talker, there is no cost for converting the conversation information into text information. In addition, there is an effect that the recognition result error is not affected when speech recognition is used for text transcription.

  The method and apparatus of the present invention are not limited to the above-described embodiments, and can be appropriately changed without departing from the spirit of the present invention. For example, the dialog state dividing unit 14 has been described as an example in which the representative score is divided into three dialog states by comparing the representative score with two threshold values, but the state to be divided is set to N threshold values to be compared with the representative score. The number may be divided into N + 1 states.

  Note that the processes described in the above method and apparatus are not only executed in time series according to the order of description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Good.

  Further, when the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (6)

A speech section detection unit that detects speech sections of the two parties by inputting voice data of conversation between the two parties of the speaker A and the speaker B , and
A frame extracting unit that arranges the respective utterance sections in order of elapsed time and outputs a predetermined number of utterance sections as one frame;
The total utterance time of the speaker A or the speaker B in the frame is divided by the total value of the total utterance time of the speaker A and the total utterance time of the speaker B in the frame. An intra-frame speech time ratio calculation unit for calculating the ratio R _j for each frame,
A frame representative score calculation unit that determines the intra-frame speech time ratio R _j or the smoothed intra-frame speech time ratio R _j as a representative score in the one frame;
A dialogue state dividing unit for classifying each frame into at least three dialogue states by comparing the representative score with at least two threshold values;
A dialog state dividing apparatus comprising: In the dialog state dividing device according to claim 1,
The frame extraction unit
Further comprising a compatible utterance unit deleting means for deleting the utterance section that is less than the predetermined time width,
A dialog state dividing device characterized by the above. In the dialog state dividing device according to claim 1 or 2,
One of the two thresholds of the dialog state splitting unit is less than 0.5 and the other threshold is greater than 0.5;
A dialog state dividing device characterized by the above. An utterance interval detection process in which an utterance interval detection unit detects voice data of conversation between the two parties of the utterer A and the utterer B and detects the respective utterance intervals of the two parties;
A frame extraction process in which the frame extraction unit arranges the respective utterance sections in order of elapsed time and outputs a predetermined number of utterance sections as one frame;
The intra-frame utterance time ratio calculation unit calculates the total utterance time of the utterer A or the utterer B within the frame by the total value of the total utterance time of the utterer A and the total utterance time of the utterer B within the frame. An intra-frame speech time ratio calculation process for calculating the intra-frame speech time ratio R _j , which is the divided value , for each frame,
A frame representative score calculation unit in which the frame representative score calculation unit determines the intra-frame speech time ratio R _j or the smoothed intra-frame speech time ratio R _j as a representative score in the one frame;
A dialogue state classification process in which the dialogue state classification unit classifies each frame into at least three dialogue states by comparing the representative score with at least two threshold values;
Dialog state splitting method including   A program for causing a computer to function as the dialog state dividing device according to any one of claims 1 to 3.   A computer-readable recording medium on which any one of the programs according to claim 5 is recorded.