JP5325176B2 - 2-channel speech recognition method, apparatus and program thereof - Google Patents

Description

  The present invention relates to a voice recognition method, apparatus, and program for recognizing two-channel voice consisting of customer-side voice and operator-side voice, for example, at a call center.

  Conventionally, when recognizing two-channel voices of customer-side voice (hereinafter referred to as “sending-side voice”) and operator-side voice (hereinafter referred to as “receiving-side voice”), each channel has been separately recognized. And the recognition reliability showing the certainty of a speech recognition result was provided to each speech recognition result.

As a conventional technique for calculating the recognition reliability, for example, Patent Document 1 discloses the recognition reliability of the word w _{1 with} the highest score of N best candidates up to the top N in the search result of the speech recognition processing. believed to be normalized value of the score difference between w ₁ and different words w ₂ less than 2-position ₁ of the score and the score in duration of words w ₁ is shown.

In addition, as another method for calculating the recognition reliability, the strength of the relationship between each word in the speech recognition result is measured, and a high recognition reliability is given to a word strongly related to the surrounding word, There is a method of giving a low recognition reliability to a weakly related word (Non-Patent Document 1). In this method, n word sets N (w) of the word w and k words immediately before the word w and one word immediately after the word w are acquired from the speech recognition result. Then, the mutual information MI (w _i , w _j ) calculated in advance on the learning corpus is used for all combinations (w _i , w _j ) of two words included in the word set N (w). Then, the strength S (w _i , w _j ) of the association between words is calculated. Further, the average value of the relation strength S (t, w _i ) for all the words t in the word set N (w) is calculated as the context consistency measure SC (t).

JP 2005-148342 A

D. Inkpen, A. Desilets, “Semantic Similarity for Detecting Recognition Errors in Automatic Speech Transcripts,” Proceedings of HLT / EMNLP, pp.49-56, October 2005.

  In general, since the voice on the transmission side is transmitted in various acoustic environments, the voice quality varies greatly. On the contrary, the voice on the receiving side is a relatively quiet office conversation, so the voice quality is good. As described above, when the voices having different voice qualities are separately recognized and given the recognition reliability, the voice recognition reliability on the transmission side may be evaluated worse than the actual reliability. .

  This invention is made in view of such a subject, and provides the speech recognition method of the 2-channel speech which can assign | provide the speech recognition reliability of a transmission side appropriately, its apparatus, and a program With the goal.

  The speech recognition method for two-channel speech according to the present invention includes a speech recognition process and a recognition reliability calculation process. In the speech recognition process, the transmitting side speech recognition result and the receiving side speech recognition result are output by giving the word recognition reliability to each word obtained by performing speech recognition processing on the respective speeches by inputting the transmitting side speech and the receiving side speech. The recognition reliability calculation process is performed by referring to a word relevance table indicating the relevance between two words of a combination of all words of the speech recognition result with the sending side speech recognition result and the receiving side speech recognition result as inputs. The intra-channel context consistency measure of the speech recognition result and the inter-channel context consistency measure between the sending side speech recognition result and the receiving side speech recognition result. The weighted sum of the inter-context consistency measure is calculated and output as the sender recognition confidence.

  For example, in the reception voice exchanged at a call center or the like, there are many cases where a common word or related word is included in the transmission side voice and the reception side voice. The inter-channel context consistency measure of the present invention, which focuses on word co-occurrence between the transmitting voice and the receiving voice, shows a large value when the relationship between the two voices is strong. Therefore, the speech recognition method for two-channel speech according to the present invention, which calculates the weighted sum of the inter-channel context consistency measure and the intra-sender context consistency measure as the sender recognition reliability, The degree can be given appropriately.

The figure which shows the function structural example of the speech recognition apparatus 100 of this invention. The figure which shows the operation | movement flow of the speech recognition apparatus 100. The figure explaining N best candidate and word recognition reliability. The figure which shows the function structural example of the word related degree table creation apparatus 150. FIG. The figure which shows a word set notionally. The figure which shows an example of a word related degree table. The figure which shows the function structural example of the recognition reliability calculation part 30. FIG. The figure which shows the function structural example of the speech recognition apparatus 200 of this invention. The figure which shows the function structural example of the weight calculation part 80. FIG.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

  FIG. 1 shows a functional configuration example of the speech recognition apparatus 100 of the present invention. The operation flow is shown in FIG. The speech recognition apparatus 100 includes a speech recognition unit 20, a recognition reliability calculation unit 30, and a word association degree table 40. The function of each part of the speech recognition apparatus 100 is realized by reading a predetermined program into a computer composed of, for example, a ROM, a RAM, a CPU, etc., and executing the program by the CPU.

  For example, the speech recognition unit 20 is a speech that has been given a word recognition reliability for each word that has been subjected to speech recognition processing on a response speech composed of a transmission side speech and a reception side speech exchanged between a customer and an operator at a call center or the like. The side speech recognition result and the receiving side speech recognition result are output (step S20). The speech recognition unit 20 analyzes the transmitting side speech and the receiving side speech into LPC cepstrum, MFCC, and other acoustic feature parameter series in units called frames of several tens of msec by an internal acoustic analysis unit (not shown). Then, a recognition result candidate for the input speech is searched for the acoustic feature parameter series using a dictionary and a language model. As a result of the search, N best candidates up to the top N are output as a speech recognition result together with the word recognition reliability. In the case where the answering voice is an integration of the sending voice and the receiving voice, a voice channel dividing unit 10 is provided for dividing the answering voice into two channels of the sending voice and the receiving voice.

  Here, N best candidates and word recognition reliability will be described with reference to FIG. Note that N best candidates and word recognition reliability are conventional techniques. The word recognition reliability is described in Patent Document 1, for example.

  The horizontal axis in FIG. 3 represents elapsed time and is represented by a frame. The vertical axis represents N best candidates in which word string candidates searched for in units of frames are arranged in descending order of scores. The score is the likelihood at the time of search.

Word recognition reliability, if the word w _** in frame t _* (* is an arbitrary integer) is different word present in the N-best candidate words w _** frame t score paired candidacy word frames in _* Given by the score difference between the next ranking score at t _* . In the example shown in FIG. 3, the word recognition of the first candidate word w ₁₁ (11 represents the first word of the first candidate) searched for in the acoustic feature parameter series of frames t _{1 to} t _4. Since the opposite words are the third candidate word w ₃₁ and the second candidate word w ₂₁ , the value obtained by dividing the total score difference (●) by the number of frames is the word recognition reliability. It becomes. For the word w ₁₃ the conflict candidate does not exist, the word recognition reliability by predetermined fixed value (○) is used. This word recognition reliability is accumulated for each candidate and becomes the word string recognition reliability.

  The recognition reliability calculation unit 30 refers to the word association degree table 40 indicating the degree of association between two words of the combination between all words of the speech recognition result with the transmission side speech recognition result and the reception side speech recognition result as inputs. Determine the in-channel context consistency measure between the sending and receiving speech recognition results and the inter-channel context consistency measure between the sending and receiving speech recognition results. A weighted sum of the in-channel context consistency measure and the inter-channel context consistency measure is calculated and output as the transmitting side recognition reliability (step S30).

  Since the response voice is a dialogue between the customer and the operator, the customer's utterance is often related to the utterance content of the operator. Therefore, when a word related to the voice recognition result on the operator side is included in the voice recognition result on the customer side, the recognition result may be considered to be correct. The speech recognition method for two-channel speech according to the present invention uses the relationship of word co-occurrence seen when the relevance between two utterances is strong, and has the effect of increasing the accuracy of the transmission side recognition reliability.

Here, the word association degree table creation device 150 that creates the word association degree table will be described.
[Word relevance table creation device]
FIG. 4 shows a functional configuration example of the word association degree table creation device 150. The word association degree table creation device 150 includes a learning corpus 151, a morpheme analysis unit 152, a learning corpus word set acquisition unit 153, a word list 154, a word count unit 155, a word association degree calculation unit 156, and a table array. Part 157.

  The learning corpus 151 is a collection of voice documents on a large scale. The morpheme analysis unit 152 performs a well-known morpheme analysis process that reads a speech document from the learning corpus 151 and divides it into words. Output corpus. The morphological analysis processing is well known and is described in, for example, a reference document “Japanese Patent No. 3379643”.

The learning corpus word set acquisition unit 153 performs windowing with the window width n words and the window shift amount m words from the beginning to the end of the word-boundary learning corpus output by the morpheme analysis unit 152, and the word list included in each window The words described in 154 are combined into a word set, and a word set for each window is output. The word list 154 describes all words that can appear in the speech recognition result, and is created in advance. FIG. 5 conceptually shows the word set. The horizontal direction is the passage of time, and the word set is indicated by N _{1 to} N _h . m is the window shift amount, and n is the window width. Adjacent word sets share a relationship of nm words.

The word count unit 155 receives the word set output from the learning corpus word set acquisition unit 153 as an input, and the number of occurrences C (w) of each word in the word set and the number of occurrences C (w _i , w _{j of} each word pair) ), Count and output the total number of word sets. The number of occurrences C (w) of the word w is the number of word sets including the word w. The number of occurrences C (w _i , w _j ) of the word pair (w _i , w _j ) is the number of word sets including both w _i and w _j .

The word relevance calculation unit 86 calculates the relevance S (w _i , w _j ) of each word pair (w _i , w _j ) using, for example, equation (1).

N is the total number of word sets, C (w) is the number of single occurrences of word w, and C (w _i , w _j ) is the number of co-occurrence of words w _i and w _j . When the value of the relevance S (w _i , w _j ) is large, it means that the relevance between these words is high. For the relevance S (w _i , w _j ), for example, a Jaccard coefficient (formula (2)) may be used in addition to the formula (1).

  A Dice coefficient (Equation 3) or a Simpson coefficient (Equation 4) can also be used.

The table arrangement unit 157 arranges the relevance S (w _i , w _j ) calculated from the words w _i and w _j in a table format that can be referred to. FIG. 6 shows an example of the word association degree table 40. The uppermost column and the leftmost column are words w _{1 to} w _N , and the relevance S (w _i , w _j ) of each word is arranged in a column where each row and each column intersect.

  FIG. 7 shows a more specific functional configuration example of the recognition reliability calculation unit 30 and will be described in more detail. The recognition reliability calculation unit 30 includes a reception recognition result word set acquisition unit 31, a transmission recognition result word set acquisition unit 32, an in-channel context consistency measure calculation unit 33, and an inter-channel context consistency measure calculation unit 34. , Receiving side context consistency integration means 35, transmission side context consistency integration means 36, and word list 154.

  Similarly to the learning corpus word set acquisition unit 153 of the word association degree table creation device 150, the reception recognition result word set acquisition unit 31 has a window width n word and a window shift amount m from the beginning to the end of the reception side speech recognition result. Windowing is performed with words, and the words described in the word list 154 included in each window are grouped into a word set, and a word set with time information is output for each window. The word list 154 is the same as the word association degree table creation device 150. The transmission side recognition result word set acquisition means 32 also outputs a word set with time information for each window by using the transmission side speech recognition result as an input.

The in-channel context consistency measure calculating means 33 receives the word sets NU _i and NO _i with time information on the transmitting side and the receiving side as inputs, and calculates the context consistency measure of each word set as an in-channel context consistency measure. To do. The in-channel context consistency measure is the relevance S obtained by referring to the word relevance table 40 for all word pairs (w _i , w _j ) included in the word set (where w _i ≠ w _j ). It is an average value of (w _i , w _j ).

The in-channel context consistency measure calculation means 33 performs the above calculation for each of the word sets NU _i and NO _i with time information on the transmitting side and the receiving side, and the in-channel context consistency measure SC _in ( NU _i ) and receiver's in-channel context consistency measure SC _in (NO _i ). The in-channel context consistency scales SC _in (NU _i ) and SC _in (NO _i ), which represent the strength of relevance of word set units, are indices that represent the consistency of contexts in the utterance units on the sending and receiving sides. It is.

The inter-channel context consistency measure calculating means 34 calculates the _inter- channel context consistency measure SC _inter (NU _i ) with the word sets NU _i and NO _i with time information on the transmitting side and the receiving side as inputs. The _inter- channel context consistency measure SC _inter (NU _i ) is calculated by the following procedure.

First, the receiver side word set NO _i to which the time immediately before the time TU _i given to the transmitter side word set NU _i is given is acquired. If there is no receiver-side word set NO _{i at} the time immediately before time TU _i , the receiver-side word set assigned the earliest time is used.

Next, the average value of the degree of association between the words included in the receiving-side word set NO _i at the time immediately before the time TU _i and the words included in the focused transmitting-side word set is calculated as the inter-channel context. Calculated as the consistency measure SC _inter (NU _i ). The _inter- channel context consistency scale SC _inter (NU _i ) indicates the strength of the relationship between words for each word set on the sending side and the word set on the immediately preceding receiving side. This is an index representing the strength of the utterance relationship with the utterance side.

The sending-side context consistency integration means 36 includes an in-channel context consistency scale SC _in (NU _i ) output from the in-channel context consistency scale calculation means 33 and an inter-channel context consistency scale calculation means 34. The _inter- channel context consistency measure SC _inter (NU _i ) output by S is input and the transmission side recognition reliability CU is calculated by Equation (5) and output.

  h is the number of word sets, and α is a weight. The weight α is, for example, a recognition reliability obtained by weighting and adding a sender-side recognition reliability and a receiver-side recognition reliability obtained based on a development set in which an actual response voice and a manually created transcription text are paired. Any value (0 <α <1) set in advance so that the value of the correlation coefficient with the speech recognition accuracy becomes the highest. Usually, since the voice recognition accuracy on the receiving side is higher, the weight α should be set to a larger value. It is considered that this will more accurately evaluate the recognition reliability on the transmission side.

  As described above, the sending-side context consistency integrating unit 36 determines the strength of the relevance of the word-set unit of the sending-side speech recognition result, the word set of the sending-side speech recognition result, and the immediately preceding receiving-side speech recognition result. The weighted sum of the relevance to the word set is calculated as the transmission side recognition reliability averaged by the number of word sets of the transmission side speech recognition result. The transmission side recognition reliability CU is a large value when the relationship between the transmission side utterance and the reception side utterance is strong.

The receiver-side context consistency integration unit 35 averages the receiver-side in-channel context consistency measure SC _in (NO _i ) output from the in-channel context consistency measure calculation unit 33 by the number of word sets NO _i. The recognition reliability CO is calculated (formula (6)) and output.

  Further, the weight α may be calculated from the word recognition reliability.

  FIG. 8 shows a functional configuration example of the speech recognition apparatus 200 including the weight calculation unit 80 that obtains the weight α from the word recognition reliability. The speech recognition apparatus 200 differs from the speech recognition apparatus 100 only in that a weight calculation unit 80 is provided. The weight calculation unit 80 receives the transmission side speech recognition result and the reception side speech recognition result output from the speech recognition unit 20 as input, calculates the weight α, and the transmission side context consistency integration unit 36 of the recognition reliability calculation unit 30. Is given a weight α.

  FIG. 9 shows a functional configuration example of the weight calculation unit 80. The weight calculation unit 80 includes a reception side recognition score calculation unit 81, a transmission side recognition score calculation unit 82, and a sigmoid function calculation unit 83. The receiving side recognition score calculation means 81 receives the receiving side speech recognition result output from the speech recognition unit 20 as an input, and calculates the sum of the word recognition reliability assigned to each word of the speech recognition result as the duration length of each word. The receiver's speech recognition score PO divided by the sum is output.

  The transmission side recognition score calculation means 82 receives the received speech obtained by dividing the sum of the word recognition reliability given to each word of the transmission side speech recognition result output by the speech recognition unit 20 by the total duration of each word. Output side speech recognition score PU. The sigmoid function calculation means 83 calculates the weight α according to the equation (7) with the receiving side speech recognition score PO and the receiving side speech recognition score PU as inputs.

  g is a gain constant of the weight α, d is a shift constant, and is a preset value. For example, the shift constant d is set in a range of 0 <50000, and the gain constant is set in a range of 1000 to 5000.

  The weight calculation unit 80 uses the fact that the higher the speech recognition accuracy of the receiving channel than the transmitting channel is, the larger the difference between the receiving speech recognition score PO and the receiving speech recognition score PU is. A value obtained by converting the difference into a value in the range of 0 to 1 by the sigmoid function is output as the weight α.

  When the processing means in the above apparatus is realized by a computer, the processing contents of the functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (6)

A speech recognition process that outputs a speech recognition result and a speech recognition result on the transmission side and a speech recognition result on each word obtained by performing speech recognition processing on each speech by inputting the speech on the transmission side and the reception side speech; and
With reference to the word association degree table indicating the degree of association between two words in the combination of all the words in the speech recognition result using the above transmission side speech recognition result and receiver side speech recognition result as input, An intra-channel context consistency measure and an inter-channel context consistency measure between the transmitting side speech recognition result and the receiving side speech recognition result are obtained. A recognition reliability calculation process in which the weighted sum of the consistency measure is calculated and output as the sender recognition reliability, and
A speech recognition method for two-channel speech including The speech recognition method according to claim 1,
The recognition reliability calculation process is as follows:
Furthermore, the speech recognition method for two-channel speech, which is a process of calculating and outputting the average value of the context consistency measure in the receiver side channel as the receiver recognition reliability. The speech recognition method for two-channel speech according to claim 2,
The weight of the weighted sum is the receiving side obtained by dividing the sum of the word recognition reliability given to each word of the sending side speech recognition result and the receiving side speech recognition result by the sum of durations of the words. 2. A speech recognition method for two-channel speech, characterized in that it is a value obtained by a sigmoid function having a gain as a difference between a speech recognition score and a transmitting side speech recognition score. A speech recognition unit that outputs a speech recognition result and a speech recognition result on the transmission side, and a speech recognition result on the receiving side, to which a word recognition reliability is assigned to each word obtained by performing speech recognition processing on each speech by inputting the speech on the transmission side and the reception side speech
With reference to the word association degree table indicating the degree of association between two words in the combination of all the words in the speech recognition result using the above transmission side speech recognition result and receiver side speech recognition result as input, An intra-channel context consistency measure and an inter-channel context consistency measure between the transmitting side speech recognition result and the receiving side speech recognition result are obtained. A recognition reliability calculation unit for calculating and outputting the weighted sum of the consistency measure as the transmission side recognition reliability;
A two-channel voice recognition apparatus comprising: The speech recognition apparatus according to claim 4,
The recognition reliability calculation unit
Furthermore, a speech recognition apparatus for two-channel speech, characterized in that the average value of the context consistency measure in the receiver channel is calculated and output as the receiver recognition reliability.   A 2-channel speech recognition method program for causing a computer to execute the 2-channel speech recognition method according to any one of claims 1 to 3.

Images