JP2018031851A - Discourse function estimation device and computer program for the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a discourse function detection device for determining a discourse function with high accuracy.SOLUTION: A discourse function estimation device 44 includes: a morpheme analysis part 72 and a time series part-of-speech information storage part 74 for receiving text data of speech production 42, performing morpheme analysis and generating a first feature vector; an F0 extraction part 76, an F0 average storage part 78 and a speaker normalization part 80 for extracting F0 from the inside of a voice signal of 150 milliseconds just before a phase end of a voice signal corresponding to the speech production, and generating a second vector representing a change of F0 by normalizing F0; and a classifier 82 composed of learned SVM or DNN by previous machine learning to receive a feature vector composed of a first vector and the second vector as an input such that a discourse function of the speech production at a phase end is classified into any of a plurality of predetermined discourse functions.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a human-machine interaction system, and more particularly to a technique for detecting a turn end point of an utterance in order to enable natural interaction between a human and a robot.

  In the speech recognition task, it is important to appropriately detect the user's utterance end point. Further, it is required for the human-machine dialogue system to appropriately detect the utterance turn end point at the same time as the utterance end. Unlike a single-sentence one-answer conversation or a single voice command, a free conversation (discourse), which is a natural communication, often includes a plurality of spoken sentences in one turn of the speaker. Therefore, it is difficult to estimate the turn end point of utterance in free dialogue speech.

  When we talk to other people, the purpose is to understand the speaker's intention and respond appropriately to the speaker's utterance. Understanding what is intended by an utterance is as important as understanding all the words in the utterance and has a significant impact on smooth conversation.

  Accurate estimation of the utterance turn end point can be used to segment the utterance sentence, and also helps to understand the utterance content, leading to more natural interaction control. Furthermore, the relationship between the head movement and the utterance right change when a Japanese speaker speaks has been reported (Non-Patent Document 1). From the above, it is possible to improve the humanity of the robot by controlling the operation of the communication robot in accordance with the estimation result of the utterance end timing.

C. Liu, C. T. Ishi, H. Ishiguro, Proc. Of HRI 2012, pp. 285-292, 2012. R. Hariharan, J. Hakkinen, and K. Laurila, ICASSP 2001, vol. 1, pp. 249-252, May 2001. Q. Li, J. Zheng, Q. R. Zhou, and C. Lee, ICASSP 2001, vol. 1, pp. 233-236, May 2001. L. Huang and C. Yang, Proc. ICASSP 2000, vol. 3, pp. 1751-1754, 2000.

  In the conventional research, silence intervals (Non-Patent Document 2), zero-cross and entropy (Non-Patent Documents 3 and 4) are used to detect the turn end point of an utterance. However, the accuracy of sentence ending detection by these methods is easily influenced by the environment, and therefore there is a problem that the accuracy of detecting the turn end point is low. In addition, since natural conversation includes many overlaps and simultaneous utterances, it is difficult to accurately detect the turn end point with the conventional technique using a silent section, zero cross, or entropy.

  Moreover, the end of the utterance does not simply mean transferring the utterance turn to another person. In some cases, the previous speaker may hold the right to speak, or may request a question or response from the other party. That is, utterances have a function to advance conversations between speakers in a certain direction, not just to describe something. Such a function is called a discourse function here.

  In the human-machine dialogue system, there is a problem that a natural dialogue cannot be made unless the discourse function at the end of the utterance is detected with high accuracy. In the prior art, it is difficult to determine such a discourse function with high accuracy.

  Therefore, the present invention is to provide a discourse function detection device that determines a discourse function with high accuracy.

  The discourse function estimation apparatus according to the first aspect of the present invention receives a text data of an utterance, and generates a first feature vector for estimating a discourse function in the utterance by analyzing the text data. And generating a second vector representing a change in the fundamental frequency component by extracting the fundamental frequency component from the speech signal in a predetermined section immediately before the end of the phrase detected during the speech in the speech signal corresponding to the speech. A second vector generation unit that receives the feature vector composed of the first vector and the second vector as an input, and classifies the discourse function of the utterance at the end of the phrase as one of a plurality of predetermined discourse functions And classifying means that have been learned in advance by machine learning.

  Preferably, the second vector generating means divides the predetermined section immediately before the end of the phrase detected during speech into a plurality of divided sections, and the fundamental frequency of each divided section divided by the dividing means. Means for generating a second vector as an element.

  More preferably, the first vector generation unit receives the text data of the utterance, performs morphological analysis on the text data, and outputs the morpheme sequence output by the morpheme analysis unit. Using the morpheme sequence storage means for storing in series and at least the part of speech information obtained from each of the latest predetermined number of morphemes stored in the morpheme sequence storage means, a first vector is generated and used as a classifier Means for outputting.

  More preferably, the first vector generation means receives utterance text data, morphologically analyzes the text data, and outputs a morpheme string, and a set of words appearing in the morpheme string (BOW) BOW vector generating means for generating a BOW vector representing the element, and elements of the BOW vector generating means are normalized by the appearance frequency of each word in a predetermined data set and the appearance frequency of each word during utterance, and after normalization BOW vector normalizing means for outputting a BOW vector, and dimension reducing means for reducing the dimension of the normalized BOW vector output from the BOW vector normalizing means and outputting it as a first vector.

  The dimension reduction means may include means for reducing the dimension of the normalized BOW vector output from the BOW vector normalization means by a latent Dirichlet allocation method (LDA) to generate the first vector.

  The dimension reduction means is connected to receive the normalized BOW vector output from the BOW vector normalizing means, and has been learned in advance so that the input and the output are equal, and the normalized BOW vector. And a means for generating a first vector using the value output from each node of the bottleneck layer of the bottleneck neural network as an element.

  Preferably, the classification means includes a support vector machine that has received a feature vector as input and has learned to classify the discourse function of the utterance at the end of the phrase into one of a plurality of predetermined discourse functions.

  More preferably, the classification means receives as input a hidden Markov model that expresses a hidden state transition path corresponding to the speech discourse function, an output probability of each element of the feature vector in each state, and a feature vector. Deep neural network trained by machine learning in advance to output the probability that the state of the hidden Markov model transitions to each of the states after outputting the feature vector, and the feature vector, hidden Markov model, and deep neural network A maximum likelihood estimation means for estimating a maximum likelihood path as a transition path of an invisible state of speech, and a means for estimating a speech discourse function based on the path estimated by the maximum likelihood estimation means; including.

  More preferably, the second vector generation means extracts a fundamental frequency component from a speech signal in a predetermined section immediately before the end of a phrase detected during speech in a speech signal corresponding to speech, and stores it as a logarithmic fundamental frequency component. Fundamental frequency extraction means for storing, fundamental frequency average storage means for storing the logarithm average value of the fundamental frequency of the speech of the speaker who has been extracted in advance, and the fundamental from the logarithmic fundamental frequency component extracted by the fundamental frequency extraction means Means for normalizing the logarithmic fundamental frequency component by subtracting the average value stored in the frequency mean storage means, and generating a second vector using the normalized logarithmic fundamental frequency component as an element.

  Preferably, the discourse function estimation device includes a fundamental frequency calculation means for calculating a logarithm of a fundamental frequency of a speaker's voice in an utterance every predetermined time, and a fundamental frequency calculated by the fundamental frequency calculation means every predetermined time. Means for calculating an average value of the logarithm and storing it in the fundamental frequency average storage means.

  More preferably, the discourse function estimating device further includes a phrase end detection means for detecting a phrase end of the utterance and outputting a phrase end signal. The first vector generation means and the second vector generation means respectively generate and output the first vector and the second vector from the text data and the speech signal immediately before the phrase end detected by the phrase end detection means. .

  More preferably, the phrase end detection means should be treated as a phrase end from the speech recognition device that performs speech recognition on an utterance and outputs text data, and the phoneme information of the text data output by the speech recognition device immediately before the end of the phrase. A phrase end specifying means for specifying the phrase end section is included. The second vector generating means includes means for dividing the phrase end section into partial sections each having a predetermined length and extracting the logarithm of the fundamental frequency of each partial section; and each partial section extracted by the means for extracting And means for generating a second vector of fixed length based on the relationship between the logarithm of the fundamental frequencies.

  A computer program according to the second aspect of the present invention causes a computer to function as any one of the above-described discourse function estimation devices.

It is a figure which shows the structure of the human-machine dialogue system containing the discourse function estimation apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows schematic structure of the discourse function estimation apparatus shown in FIG. It is a block diagram which shows schematic structure of the discourse function estimation apparatus which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows schematic structure of the discourse function estimation apparatus which concerns on the 3rd Embodiment of this invention. It is a schematic diagram which shows schematic structure of the bottle net network used by 3rd Embodiment. It is a schematic diagram which shows the structure of the deep neural network used with the classifier of 3rd Embodiment. It is a schematic diagram for demonstrating the estimation process of the maximum likelihood series of a discourse function by the classifier of 3rd Embodiment. It is a block diagram which shows schematic structure of the discourse function estimation apparatus which concerns on the 4th Embodiment of this invention. In embodiment of this invention, it is a graph for demonstrating the effect at the time of using LDA as a classifier. In embodiment of this invention, it is a graph for demonstrating the effect at the time of using DNN as a classifier. It is an external view of the computer system for implement | achieving the discourse function estimation apparatus which concerns on each embodiment of this invention. It is a block diagram which shows the internal structure of the computer system which shows an external appearance in FIG.

  In the following description and drawings, the same parts are denoted by the same reference numerals. Therefore, detailed description thereof will not be repeated.

[First Embodiment]
<Outline>
With reference to FIG. 1, the discourse function estimation apparatus 44 according to the first embodiment of the present invention has a discourse function including not only language information obtained from the speech 42 uttered by the operator 40 but also its prosodic information. To detect. A natural dialogue between the operator 40 and the robot 48 is realized by controlling the head movement 46 of the robot 48 or controlling the response of the robot 48 by the detected discourse function.

<Constitution>
Referring to FIG. 2, the discourse function estimating device 44 according to the present embodiment receives the speech 42 of the speaker, recognizes the speech, and outputs the speech recognition result text data. The morpheme analysis is performed on the morpheme analysis unit 72 that outputs a morpheme sequence to which part-of-speech information is attached, and the time series of the morpheme information of the morpheme sequence output by the morpheme analysis unit 72 is stored. A time-sequential part-of-speech information storage unit 74 for outputting a vector to be output.

  The language handled in this embodiment is Japanese, and there are 11 parts of speech. Twelve types of part-of-speech information are used including “others”. A vector whose part of speech is an element is a fixed-length vector having the number of types of parts of speech (12). The value of the element corresponding to the corresponding part of speech is 1, and the values of the other elements are 0. is there.

  The discourse function estimation device 44 further includes the F0 extraction unit 76 for extracting the log scale fundamental frequency (F0) from the voice 42 every 10 milliseconds, and the voice 42 prepared in advance separately from the voice 42. The F0 average storage unit 78 for storing the average value of the F0 of the speech data in advance and the F0 extracted by the F0 extraction unit 76 for a predetermined time every 10 milliseconds (150 milliseconds in this embodiment). In response to the phrase boundary information 84, a vector whose elements are normalized values of F0 of the speaker's voice 42 by subtracting the average value of the speaker's F0 stored in the F0 average storage unit 78 is stored. A speaker normalizing unit 80 for outputting, a vector obtained by concatenating a vector output from the time-series part-of-speech information storage unit 74 and a vector output from the speaker normalizing unit 80 is received as a feature vector. 50 estimated by the containing and classifier 82 for outputting. The phrase boundaries for generating the phrase boundary information 84 are assumed to be known in the present embodiment and second and third embodiments described later. The reason that the information of F0 is used for 150 milliseconds is based on the fact that it corresponds to one Japanese mora is about 150 milliseconds. In addition, you may change this length suitably.

  The classifier 82 is an SVM in the present embodiment, and performs machine learning so as to estimate the discourse function 50 with respect to a feature vector in advance using learning data. Further, in the present embodiment, the discourse function 50 has been learned in advance to identify k (holding of turns), g (turning of turns), and o (others).

<Operation>
When the voice 42 of the speaker is input, the voice recognition device 70 performs voice recognition on the voice 42 and outputs text data corresponding to the content of the utterance. The morpheme analysis unit 72 receives this output, performs morpheme analysis with reference to an attached dictionary for morpheme analysis (not shown), and outputs a morpheme string with part-of-speech information. The time-series part-of-speech information storage unit 74 stores a predetermined number of part-of-speech information time series in the morpheme string.

  The F0 extraction unit 76 extracts F0 from the voice 42 every 10 milliseconds and gives it to the speaker normalization unit 80. The speaker normalization unit 80 subtracts and normalizes the F0 average value stored in the F0 average storage unit 78 from F0 given from the F0 extraction unit 76, stores the latest 150 milliseconds, In response to receiving the boundary information 84, a vector having the stored value of F0 for 150 milliseconds as an element is generated and given to the classifier 82.

  An input of the classifier 82 is a feature vector obtained by concatenating a vector from the time-series part-of-speech information storage unit 74 and a vector from the speaker normalization unit 80. The classifier 82 estimates and outputs an estimated value (one of k, g, or o) of the discourse function at the end of the utterance expressed by the speech 42 based on the feature vector. The head movement 46 shown in FIG. 1 can be controlled based on this discourse function.

  According to the experiment, the discourse function estimation device 44 according to this embodiment obtained a result of 69% as the accuracy of the discourse function recognition result.

[Second Embodiment]
<Constitution>
FIG. 3 shows a block diagram of the discourse function estimation apparatus 100 according to the second embodiment of the present invention. The discourse function estimation device 100 is different from the discourse function estimation device 44 according to the first embodiment in that it is based on the morpheme sequence output by the morpheme analysis unit 72 instead of the time-series part-of-speech information storage unit 74. A point including a vector generation unit 110 for generating a vector representing n-grams of the latest morpheme in a bag-of-words (BOW vector) expression, not a part-of-speech information, and a BOW vector output from the vector generation unit 110 The vector including the vector normalization unit 111 for normalizing the vector and the dimension of the BOW vector normalized by the vector normalization unit 111 is reduced by an LDA (Latent Dirichlet Association), thereby reducing the dimension. And a vector dimension reduction processing unit 11 instead of the classifier 82 of FIG. 2 obtained by connecting the vectors from 2 and the vectors output from the speaker normalization unit 80 as feature vectors, and the discourse function of the utterance represented by the speech 42 is k, g, q (question / response request), bc (interference) And a point including a classifier 114 made of SVM. The classifier 114 performs learning using learning data labeled in advance with the four tags described above and the normalized F0 output from the speaker normalization unit 80 with respect to the learning data.

  The BOW vector has as many elements as the number of words appearing in the entire data used for learning. The value of each element is the frequency at which each word appears in the last phrase as a result of speech recognition of the utterance data to be processed. Therefore, most elements of this vector are zero.

  The vector normalization unit 111 normalizes the BOW vector as follows. In this normalization, so-called tf-idf is used. That is, the appearance frequency of each word in the entire data used for learning is calculated in advance. Then, after dividing each element of the BOW vector by the appearance frequency of the word corresponding to the element in the entire data, normalization is performed so that the magnitude of the vector becomes 1.

  The normalized vector is processed by the vector dimension reduction processing unit 112 using LDA, thereby reducing the dimension of the vector. LDA is a probabilistic generation model for a large number of discrete data. This model is a hierarchical Bayesian model where each phrase is considered a finite mixture of a set of topics. Thus, each phrase can be expressed as a set of topic probabilities. Generally, the range of topics handled by LDA is about 100 to 300. By using this LDA, the vector size can be reduced from the number of vocabularies to the number of topics. In the experiment described later, the number of topics was set to 512, 1024, 1536, and 2048.

<Operation>
The discourse function estimation apparatus 100 according to the second embodiment operates as follows. The operations of the speech recognition device 70, the morphological analysis unit 72, the F0 extraction unit 76, the F0 average storage unit 78, and the speaker normalization unit 80 are the same as those in the first embodiment. The vector generation unit 110 generates a BOW vector of the last phrase based on the morpheme sequence output from the morpheme analysis unit 72 and supplies the BOW vector to the vector normalization unit 111. The vector normalization unit 111 normalizes the BOW vector according to the above-described procedure, and gives the vector to the vector dimension reduction processing unit 112. The vector dimension reduction processing unit 112 generates a vector with reduced dimensions by performing LDA processing on the normalized BOW vector.

  The classifier 114 receives a concatenation of the vector output from the vector dimension reduction processing unit 112 and the vector output from the speaker normalization unit 80 as a feature vector, and estimates the discourse function 102 according to parameters learned in advance. Then output.

  In this embodiment, SVM is used as the classifier 114. However, the present invention is not limited to such an embodiment, and any classifier having an identification function can be applied. For example, DNN can be used instead of SVM.

[Third Embodiment]
<Constitution>
FIG. 4 shows a schematic configuration of the discourse function estimation apparatus 130 according to the third embodiment. Referring to FIG. 4, the discourse function estimation device 130 is different from the discourse function estimation device 100 shown in FIG. 3 in that it includes a bottleneck neural network 140 instead of the vector dimension reduction processing unit 112 shown in FIG. In addition, in place of the classifier 114 using the SVM of FIG. 2, a classifier 142 that combines a deep neural network (DNN) and a drown Markov model (HMM) is included. In other respects, the discourse function estimation device 130 is the same as the discourse function estimation device 100. However, in this embodiment, the length of the section using the voice F0 is 150 milliseconds to 200 milliseconds, and an appropriate value is selected by a prior experiment.

  The classifier 142 according to this embodiment has a function of discriminating discourse functions of sil (silence), listening, k, g, i, and q.

  Similar to the vector dimension reduction processing unit 112 shown in FIG. 2, the bottleneck neural network 140 is for reducing vector dimensions. The bottleneck neural network 140 has a configuration as shown in FIG. 5, for example. This example is merely illustrative.

  Referring to FIG. 5, the bottleneck neural network 140 includes an input layer 152 that receives an input vector 150, an output layer 160 that has the same number of nodes as the input layer 152, and between the input layer 152 and the output layer 160. It includes a plurality of blister layers 154, 156 and 158 provided. Among the hidden layers 154, 156, and 158, the number of nodes of the drown layer 156 is smaller than that of the other layers. Therefore, the drown layer 156 is called a bottleneck layer. The learning of the bottleneck neural network 140 is performed by using a lot of learning data so that the output vector 162 output from the output layer 160 becomes equal to the input vector 150 when the input vector 150 is given to the input layer 152. Done. According to the bottleneck neural network 140 after learning, after the number of elements of the vector given to the input layer 152 is once reduced to the number of nodes in the bottleneck layer 156, the same vector as the input vector can be reproduced again. That is, the output of the bottleneck layer 156 is considered to have sufficient information to reproduce the contents of the input vector 150 with a small number. Therefore, by extracting the output of the bottleneck layer 156 as the bottleneck feature quantity 164, the dimension of the input vector can be reduced.

  FIG. 6 shows an outline of the configuration of the DNN 180 for determining the state transition probability of the drown Markov model in the classifier 142 of FIG. Referring to FIG. 6, DNN 180 is provided corresponding to silence, listening, k, g, i, and q, and input layer 190 having a plurality of nodes connected to receive feature vector 182. Provided between the input layer 190 and the output layer 194, and an output layer 194 having six nodes for outputting the probability that the state in which the Markov model is drowned immediately after the vector 182 is given is the transition state. A plurality of hidden layers 192 formed.

  The upper part of FIG. 7 illustrates the relationship between the state transition by the hidden Markov model and the output (feature) at that time. This example shows a state transition from silence to silence via states s1 and s2. When the state is silence, the feature sil is obtained from the output. Similarly, the feature f1 is obtained when the state is s1, and the feature f2 is obtained when the state is s2. The features f1 and f2 are vectors that are output according to the probability density function obtained by learning in advance for the states s1 and s2, respectively.

  When the feature vector 182 is given to the input layer 190 of the DNN 180, the DNN 180 outputs a probability vector indicating what state the drowning Markov model will next transition to.

  As shown in the lower part of FIG. 7, the classifier 142 selects the maximum likelihood sequence 218 having the highest probability of transition from the state 210 to the state 212 (maximum likelihood) by likelihood calculation. In the example shown in FIG. 7, the routes in the states 210, 214, 216, and 212 have a higher probability than the other routes, and are therefore selected as the most likely routes. In this case, if state 212 indicates the end of a phrase, it is estimated that the discourse function corresponding to state 216 is the discourse function at the end of the phrase.

<Operation>
The discourse function estimation apparatus 130 according to the third embodiment operates as follows. The speech recognition device 70, the morphological analysis unit 72, the vector generation unit 110, the vector normalization unit 111, the F0 extraction unit 76, the F0 average storage unit 78, and the speech recognition device 70 operate in the same manner as in the second embodiment. The bottleneck neural network 140 receives a vector output from the vector normalization unit 111 and outputs a bottleneck feature amount. The speaker normalizing unit 80 gives the vector having the latest predetermined number as an element to the classifier 142 out of the normalized F0 for every 10 milliseconds of the voice of the immediately preceding predetermined time.

  The classifier 142 receives a feature vector obtained by connecting the vector from the bottleneck neural network 140 and the vector from the speaker normalization unit 80, estimates the sequence of the state of the discourse function immediately before the phrase boundary, and determines the last discourse. The function 132 is output.

[Fourth Embodiment]
<Constitution>
FIG. 8 shows a schematic configuration of a discourse function estimation apparatus 250 according to the fourth embodiment of the present invention. Referring to FIG. 8, discourse function estimation apparatus 250 is different from discourse function estimation apparatus 130 according to the third embodiment in that instead of speech recognition apparatus 70, speech recognition is performed and text data is output. In addition to the above, a speech recognition device 260 having a function of detecting a phrase end of an utterance and outputting a signal specifying a phrase end section is included, and instead of the speaker normalizing unit 80, an output of the F0 extracting unit 76 is provided. The F0 is normalized every 10 milliseconds by subtracting the F0 average value stored in the F0 average storage unit 78, normalized, and stored in response to the signal output from the speech recognition device 260. A speaker normalization unit 262 that outputs a sequence of F0 corresponding to a period of time as a vector, and F0 output from the speaker normalization unit 262, and a phrase end that outputs a fixed-length vector representing the prosody of the phrase end A point including an interval normalization unit 264, Instead of the classifier 142 according to the third embodiment, the vector composed of the bottleneck features output from the bottleneck neural network 140 and the prosody of the phrase end interval output from the phrase end interval normalization unit 264 are shown. This is a point including a classifier 266 that receives a vector obtained by concatenating a fixed-length vector as a feature vector, and estimates and outputs a discourse function 252 corresponding to the feature vector.

  The number of elements of the vector output from the speaker normalization unit 262 varies as the length of the phrase end section varies. The phrase end interval normalization unit 264 normalizes this variable length vector to a fixed length. For example, the phrase end interval normalization unit 264 sets the tone of the speech represented by the input variable length vector to the first category (up tone, down tone, flat tone, down / up tone, etc.) and the second category. Each is classified according to category (short, long, very long), and a fixed-length vector representing a combination of information representing these categories is output. Alternatively, the phrase end interval normalization unit 264 may output a fixed-length vector by down-sampling a variable number of F0s to a fixed number.

  The classifier 266 may have the same configuration as the classifier 142 if the dimension of the vector output by the phrase end interval normalization unit 264 is the same as that in the third embodiment. However, it goes without saying that the learning data should be changed.

<Operation>
The speech recognition device 260 of the discourse function estimation device 250 recognizes the speech 42 and outputs text data, and provides a signal to the speaker normalization unit 262 to detect the end of the phrase and identify the end of the phrase. The morphological analysis unit 72, the vector generation unit 110, the vector normalization unit 111, and the bottleneck neural network 140 operate in the same manner as in the third embodiment, and the feature vector obtained based on the language information is input to the classifier 266. give. The F0 extraction unit 76 calculates F0 of the voice 42 every 10 milliseconds and gives it to the speaker normalization unit 262. The speaker normalizing unit 262 normalizes the value by subtracting the average value of F0 stored in the F0 average storage unit 78 from this value, and stores it as a time series. When a signal specifying the phrase end period is given from the speech recognizer 260, the speaker normalizing unit 262 gives the normalized F0 sequence for the period to the phrase end period normalizing unit 264. The phrase end interval normalization unit 264 classifies the F0 series according to the above-described two types of categories, and provides the classifier 266 with data indicating the classified categories in a vector format. The classifier 266 includes the bottleneck neural network 140.
The feature vector based on the linguistic information from and the feature vector based on the prosodic information from the phrase end interval normalization unit 264 are received as a feature vector, and the discourse function at the end of the phrase is estimated according to the learning parameters The discourse function 252 is output.

[Experimental result]
The following experiment was performed using the configuration of the discourse function estimation apparatus 100 of the second embodiment described above. A preliminary experiment was performed to estimate the discourse function based only on linguistic information without using prosodic information (F0). When LDA was used for the vector dimension reduction processing unit 112, BOW, POS unigrams, bigrams and trigrams were used. Higher accuracy was obtained than when either of these was used. Therefore, the accuracy of using LDA as the vector dimension reduction processing unit 112 using only language information was compared with the accuracy when LDA was used for the vector dimension reduction processing unit 112 by adding prosodic information in addition to language information. The results are shown in FIG.

  Referring to FIG. 9, the horizontal axis represents the number of LDA topics, and the vertical axis represents the prediction accuracy. A graph 300 shows prediction accuracy when only language information is used, and a graph 302 shows prediction accuracy when prosodic information is added in addition to language information. As is clear from this graph, the accuracy of prediction was significantly increased by adding prosodic information. Further, since there is a clear difference in accuracy between the case of 100 topics and the other cases, it can be seen that when the number of topics is reduced to 100, a part of information is lost, resulting in a decrease in accuracy.

  FIG. 10 shows an experimental result using LDA as the vector dimension reduction processing unit 112 and using DNN as the classifier 114 in FIG. 3 instead of SVM. In the experiment, the number of nodes in each hidden layer of the DNN was changed 512 by 512 from 512 to 2048, and the change in the prediction accuracy was examined. As can be seen from this graph, when DNN is used, the accuracy is improved by increasing the number of nodes in the hidden layer. Further, it can be seen that higher accuracy is obtained than when the SVM is used for the classifier 114 except when the number of nodes in the hidden layer is 512.

[Effect of the embodiment]
As described above, according to the embodiment of the present invention, the discourse function at the end of a phrase is estimated in consideration of not only language information but also prosodic information at the end of the phrase. Therefore, it is possible to estimate the discourse function with higher accuracy than in the case of using only language information. Further, by using SVM, DNN, or a combination of a Markov model and DNN as a classifier, it is possible to estimate the discourse function at the end of a phrase with stable and high accuracy reflecting the learning result. Therefore, more natural interaction can be realized by constructing a human-machine interface using this discourse function.

[Realization by computer]
The discourse function estimation apparatus according to each embodiment of the present invention can be realized by computer hardware and a computer program executed on the computer hardware. FIG. 11 shows the external appearance of the computer system 530, and FIG. 12 shows the internal configuration of the computer system 530.

  Referring to FIG. 11, the computer system 530 includes a computer 540 having a memory port 552 and a DVD (Digital Versatile Disc) drive 550, a keyboard 546, a mouse 548, and a monitor 542.

  12, in addition to the memory port 552 and the DVD drive 550, the computer 540 includes a CPU (Central Processing Unit) 556, a bus 566 connected to the CPU 556, the memory port 552, and the DVD drive 550, and a boot program. And the like, a read only memory (ROM) 558 for storing etc., a random access memory (RAM) 560 connected to the bus 566 for storing program instructions, system programs, work data and the like, and a hard disk 554. The computer system 530 is further connected to the bus 566 and provides a connection to a sound board 568 for digitizing and converting the audio signal into a form that can be processed by the computer, and a network 568 that allows communication with other terminals. Network interface card (NIC) 574. A microphone 570 is connected to the sound board 568.

  A computer program for causing the computer system 530 to function as each function unit of the discourse function estimation apparatus according to each of the above-described embodiments is stored in the DVD 562 or the removable memory 564 installed in the DVD drive 550 or the memory port 552, and Transferred to the hard disk 554. Alternatively, the program may be transmitted to the computer 540 through the network 568 and stored in the hard disk 554. The program is loaded into the RAM 560 when executed. The program may be loaded directly into the RAM 560 from the DVD 562, from the removable memory 564, or via the network 568.

  This program includes an instruction sequence including a plurality of instructions for causing the computer 540 to function as each functional unit of the discourse function estimation apparatuses 44, 100, 130, and 250 according to the above embodiments. Some of the basic functions necessary to cause the computer 540 to perform this operation are an operating system or third party program running on the computer 540 or various dynamically linkable programming toolkits or programs installed on the computer 540. Provided by the library. Therefore, this program itself does not necessarily include all the functions necessary for realizing the system, apparatus, and method of this embodiment. The program is a system as described above by dynamically calling an appropriate program in an appropriate function or programming toolkit or program library in a controlled manner to obtain a desired result among instructions, It is only necessary to include an instruction for realizing a function as an apparatus or a method. Of course, all necessary functions may be provided only by the program.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are included. Including.

40 operator 42 speech 44, 100, 130, 250 discourse function estimation device 46 head movement 48 robot 50, 102, 132, 252 discourse function 70, 260 speech recognition device 72 morpheme analysis unit 74 time-series part-of-speech information storage unit 76 F0 Extraction unit 78 F0 average storage unit 80, 262 Speaker normalization unit 82, 114, 142, 266 Classifier 84 Phrase boundary information 110 Vector generation unit 111 Vector normalization unit 112 Vector dimension reduction processing unit 140 Bottleneck neural network 150 Input Vector 152 Input layer 154, 158, 192 Hidden layer 156 Bottleneck layer 160, 194 Output layer 162 Output vector 164 Bottleneck feature 180 DNN
182 feature vector 190 input layer 210, 212, 214, 216 state of discourse function 218 maximum likelihood sequence 264 phrase end interval normalization unit

Claims (13)

First vector generating means for receiving text data of utterance, and generating morphological analysis of the text data to generate a first feature vector for estimating a discourse function in the utterance;
In the speech signal corresponding to the speech, a fundamental frequency component is extracted from the speech signal in a predetermined section immediately before the end of a phrase detected during speech, and a second vector representing a change in the fundamental frequency component is generated. Two vector generation means;
The machine receives a feature vector composed of the first vector and the second vector as an input, and classifies the discourse function of the utterance at the end of the phrase into one of a plurality of predetermined discourse functions. A discourse function estimation device including a learned classification means. The second vector generation means includes
Dividing means for dividing the predetermined section immediately before the end of the phrase detected during utterance into a plurality of divided sections;
The discourse function estimation apparatus according to claim 1, further comprising means for generating the second vector using the fundamental frequency of each divided section divided by the dividing means as an element. The first vector generation means includes:
Morphological analysis means for receiving text data of speech, performing morphological analysis on the text data, and outputting a morpheme string;
A morpheme sequence storage unit for storing the morpheme sequence output by the morpheme analysis unit in time series;
Means for generating and outputting the first vector to the classifier using at least part-of-speech information obtained from each of the latest predetermined number of morphemes stored in the morpheme string storage means as an element. The discourse function estimation apparatus according to claim 1 or 2. The first vector generation means includes:
Morphological analysis means for receiving text data of speech, performing morphological analysis on the text data, and outputting a morpheme string;
BOW vector generation means for generating a BOW vector representing a set of words (BOW) appearing in the morpheme sequence;
BOW vector normalization for normalizing the elements of the BOW vector generation means by the appearance frequency of each word in a predetermined data set and the appearance frequency of each word in the utterance, and outputting a normalized BOW vector Means,
The discourse function according to claim 1, further comprising dimension reduction means for reducing a dimension of the normalized BOW vector output from the BOW vector normalizing means and outputting the reduced vector as the first vector. Estimating device. The dimension reduction means includes means for reducing the dimension of the normalized BOW vector output from the BOW vector normalization means by a latent Dirichlet allocation method (LDA) to generate the first vector. The discourse function estimation apparatus according to claim 4. The dimension reduction means includes
A bottleneck neural network trained in advance so that the input and the output are equal, connected to receive the normalized BOW vector output from the BOW vector normalizing means;
Means for generating the first vector using as an element the value output from each node of the bottleneck layer of the bottleneck neural network in response to the provision of the normalized BOW vector. The discourse function estimation apparatus according to claim 4. The classification means includes a support vector machine that has received the feature vector as an input and has been learned to classify the discourse function of the utterance at the end of the phrase into one of a plurality of predetermined discourse functions. The discourse function estimation apparatus according to claim 6. The classification means includes
A hidden Markov model expressing the transition path of the hidden state corresponding to the discourse function of the utterance and the output probability of each element of the feature vector in each hidden state;
Deep neural network that has been trained by machine learning in advance to output the probability that the state of the hidden Markov model transitions to each of the hidden states after the hidden state that has received the feature vector as input and output the feature vector When,
Maximum likelihood estimation means for estimating a maximum likelihood path as a transition path of a hidden state of an utterance based on the feature vector, the hidden Markov model, and the output of the deep neural network;
The discourse function estimation apparatus according to any one of claims 1 to 6, further comprising means for estimating a discourse function of the utterance based on the route estimated by the maximum likelihood estimation means. The second vector generation means includes
A fundamental frequency extracting means for extracting a fundamental frequency component from the speech signal in a predetermined section immediately before the end of a phrase detected during the speech and storing it as a logarithmic fundamental frequency component in the speech signal corresponding to the utterance;
Basic frequency average storage means for storing an average value of logarithm of the fundamental frequency of the voice of the speaker of the utterance extracted in advance;
The logarithmic fundamental frequency component is normalized by subtracting the average value stored in the fundamental frequency average storage means from the logarithmic fundamental frequency component extracted by the fundamental frequency extracting means, and the normalized logarithmic fundamental frequency The discourse function estimation apparatus according to any one of claims 1 to 8, further comprising: means for generating the second vector using a component as an element. Fundamental frequency calculation means for calculating a logarithm of the fundamental frequency of the voice of the speaker in the utterance every predetermined time;
The discourse function according to claim 9, further comprising means for calculating an average value of logarithm of the fundamental frequency calculated every predetermined time by the fundamental frequency calculating means and storing the average value in the fundamental frequency average storage means. Estimating device. Further comprising a phrase end detection means for detecting the phrase end of the utterance and outputting a phrase end signal;
The first vector generation unit and the second vector generation unit respectively include the first vector and the second vector from the text data and the speech signal immediately before the end of the phrase detected by the end of phrase detection unit. The discourse function estimation apparatus according to claim 1, wherein the vector is generated and output. The phrase ending detection means performs speech recognition on the utterance and outputs the text data;
A phrase end specifying means for specifying a phrase end section to be treated as the phrase end from the phoneme information immediately before the end of the phrase of the text data output by the speech recognition device;
The second vector generation means includes
Means for dividing the phrase end section into partial sections each having a predetermined length and extracting the logarithm of the fundamental frequency of each partial section;
And means for generating the second vector of fixed length based on the relationship between the logarithm of the fundamental frequency of each subsection extracted by the means for extracting. Discourse function estimation device. A computer program that causes a computer to function as the discourse function estimation device according to any one of claims 1 to 12.