JP2014119716A - Interaction control method and computer program for interaction control - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an interaction control method capable of improving sensory quality of a user in response time after a user performs voice input by properly setting length of message voice to be reproduced in the response time in a voice interaction device.SOLUTION: An interaction control method includes steps of: extracting a predetermined keyword from a voice signal representing voice of a user collected by a voice input part; retrieving a content according to the keyword; estimating a response time until the content is presented to the user after input of the voice of the user is terminated; setting a length of message voice by referring to a table representing correspondence relation among a plurality of combinations of sample values of a silent period in which the voice is not output and sample values of the length of the message voice occupying the response time and sensory quality of the user, generating message voice having the length, and reproducing the message voice in the response time.

Description

  The present invention relates to, for example, a dialog control method and a computer program for dialog control in a voice dialog device that recognizes a voice uttered by a user and executes processing according to the recognition result.

  In recent years, a voice interaction system has been developed in which a voice signal input via a terminal is transmitted to a server via a communication network, the server selects desired content according to the voice signal, and the content is returned to the terminal. ing. In such a spoken dialogue system, when the load on the server exceeds a certain level, or when the communication network is congested, the delay in returning information from the server to the terminal may increase. there were. When the delay increases, the response time during which the terminal does not respond increases from the voice input by the user until the terminal presents the content corresponding to the voice to the user. For this reason, the user feels anxiety or the user's discomfort increases, and as a result, the user's convenience for the voice interaction system may be reduced. In view of this, there has been proposed a voice interaction device that predicts the time required for reading the recognition dictionary based on the size information of the recognition dictionary and outputs a response voice having a length corresponding to the predicted time to a speaker (for example, , See Patent Document 1). This spoken dialogue apparatus prevents such a long silence period from being read when the recognition dictionary is read out by outputting such a response voice.

JP 2001-22384 A

  However, the user feels uncomfortable if the message voice played while the user is waiting for a response is too long. For this reason, there is a possibility that the user's anxiety and discomfort may not necessarily be reduced if the message voice is simply lengthened because the silent period is long.

  Therefore, the present specification describes a dialog control that can improve a user's quality of experience during a response time by appropriately setting a length of a message voice to be reproduced during the response time after the user performs voice input in the voice dialog device. It aims to provide a method.

  According to one embodiment, a dialog control method is provided. In this interactive control method, a predetermined keyword is extracted from a voice signal representing the user's voice collected by the voice input unit, a content corresponding to the keyword is searched, and after the user's voice input is completed, the user The response time until the content is presented to the user is estimated, and a combination of a sample value of the length of the silent period during which no voice is output and a sample value of the length of the message voice, and the user's quality of experience, The length of the message voice is set by referring to the table representing the correspondence relationship, and the message voice having the length is generated, and the message voice is reproduced during the response time.

The objects and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.
It should be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention as claimed.

  The dialogue control method disclosed in the present specification can appropriately set the length of the message voice to be reproduced during the response time, and can improve the quality of experience of the user during the response time.

1 is a schematic configuration diagram of a voice interaction system according to a first embodiment. It is a functional block diagram of the control part of the server by 1st Embodiment. It is a figure which shows an example of an information source table. It is a figure which shows an example of a response time table. It is a figure which shows an example of the relationship between the message voice which occupies for response time, and a silence period. (A) is a schematic diagram showing the relationship between the time length and silence period of a message sound, and a user's experience quality. (B) is a schematic diagram showing the relationship between response time and user experience quality when the length of the message voice is L. FIG. (C) is a schematic diagram showing the relationship between the length of the message voice and the user's quality of experience. (A) is a conceptual diagram which shows an example of MOS value calculation by linear interpolation. (B) is a conceptual diagram showing an example of determination of message voice length and MOS value calculation when there are a plurality of message voice length candidates included in the estimated response time. It is a figure which shows an example of a quality evaluation table. It is an operation | movement flowchart of a message audio | voice production | generation process. It is an operation | movement sequence diagram of the dialogue control process by 1st Embodiment. It is a functional block diagram of the control part of the server by 2nd Embodiment. It is a figure which shows an example of a preference information table. It is an operation | movement sequence diagram of the dialogue control process by 2nd Embodiment.

Hereinafter, a voice interaction device and a dialogue control method of the voice interaction device according to various embodiments will be described with reference to the drawings.
For an arbitrary response time, there are a plurality of combinations of the length of the message voice reproduced during the response time and the length of the silent period during which no message voice is output during the response time. And the user's experience quality differs for each combination. However, if the length of the message voice is constant, the inventor has almost no change in the user's experience quality when the duration of the silence period is 2 seconds or less, and the length of the silence period is 2 seconds or more and 4 seconds or less. Then, the knowledge that a user's bodily sensation quality falls linearly with respect to the length of a silence period was acquired. Therefore, the inventor determined the user's quality of experience in the silent period of 2 seconds and 4 seconds for each of the plurality of time lengths of the message voice in advance by experiment, and the time length and silent period of the message voice and the user's quality of experience. A quality evaluation table showing the relationship between Then, by referring to the quality evaluation table, the voice interaction apparatus obtains the message voice time length that gives the highest user experience quality for any response time, and puts the message having the time length within the response time. Reproduce.

  FIG. 1 is a schematic configuration diagram of a voice dialogue system according to a first embodiment of a voice dialogue apparatus. In this embodiment, the voice interaction system 1 selects a terminal 2 for receiving content provided according to the input voice by the user, and a content corresponding to the voice input via the terminal 2. And a server 3 for returning the selected content to the terminal 2.

  The terminal 2 and the server 3 can communicate with each other via a communication network 4 such as a public communication line. Further, the server 3 may be able to communicate with an external information source 5 such as a Web server or an ftp server via the communication network 4. Note that the terminal 2 and the server 3 may be connected to a communication line such as a dedicated line separate from the communication network 4.

  The terminal 2 is, for example, a mobile phone, a portable information terminal, or a fixed terminal, and includes a voice input unit 21, a voice output unit 22, a communication unit 23, a storage unit 24, and a processing unit 25. Further, the terminal 2 may further include a display unit (not shown) such as a liquid crystal display.

  The voice input unit 21 includes, for example, a microphone and an analog / digital converter. The voice uttered by the user is converted into an audio signal which is an analog electric signal by the microphone. The analog audio signal is sampled at a predetermined sampling frequency by an analog / digital converter, converted into a digital audio signal, and then sent to the processing unit 25.

  The audio output unit 22 is an example of an output unit that presents content to the user, and includes, for example, a digital / analog converter and a speaker. Then, the audio output unit 22 converts the audio signal received from the processing unit 25 into an analog signal by a digital / analog converter, and the analog audio signal is converted into a sound by a speaker and output to a user.

  The communication unit 23 includes an interface circuit for connecting the terminal 2 to the communication network 4. And the communication part 23 transmits the audio | voice signal received from the process part 25 to the server 3 via the communication network 4 according to the communication system with which the communication network 4 is based. On the other hand, the communication unit 23 receives the content, the message voice reproduced during the response time, and the reproduction delay period from the server 3 via the communication network 4 and passes them to the processing unit 25.

  The storage unit 24 includes, for example, a semiconductor memory. And the memory | storage part 24 memorize | stores the various programs for the process part 25 to control the terminal 2, the application program which operate | moves on the terminal 2, and various data required for execution of these programs.

  The processing unit 25 includes one or more processors and peripheral circuits. Then, the processing unit 25 outputs the audio signal input via the audio input unit 21 to the communication unit 23. The processing unit 25 is an example of a playback time control unit, and starts counting from the time when the voice input by the user is finished. When the playback delay period notified from the server 3 elapses, the message voice is voiced. Playback is performed via the output unit 22. Note that the processing unit 25 obtains a power spectrum for each frame by, for example, performing time-frequency conversion on the input audio signal in units of predetermined frames in order to determine whether or not the audio input has been completed. Then, the processing unit 25 determines that the voice input is completed when a predetermined number or more of frames having the average power value or less continue after the frame in which the average power value for each frequency band is equal to or higher than the predetermined threshold value. To do.

  Further, when the processing unit 25 receives the content from the server 3 after the reproduction of the message sound is completed, the processing unit 25 reproduces the sound signal included in the content via the sound output unit 22.

If the content includes an image or video stream, the processing unit 25 may display the image or video stream on a display unit (not shown). Further, the processing unit 25 may cause the display unit to display the text information even when the content includes text information.
Furthermore, when the application program such as a game is included in the content, the processing unit 25 executes the application program, causes the display unit to display an image corresponding to the execution result, or outputs an audio signal corresponding to the execution result. The output unit 22 may reproduce the data.

  Referring to FIG. 1 again, the server 3 includes a communication unit 31, a storage unit 32, and a control unit 33.

The communication unit 31 has an interface circuit for connecting the server 3 to the communication network 4. Then, the communication unit 31 passes the audio signal received from the terminal 2 via the communication network 4 to the control unit 33 in accordance with the communication method that the communication network 4 complies with. On the other hand, the communication unit 31 transmits the content, message voice, and the like received from the control unit 33 to the terminal 2 via the communication network 4. Further, the communication unit 31 may transmit an information acquisition request signal to the external information source 5 via the communication network 4 and receive data including predetermined content from the external information source 5.
Hereinafter, for convenience of explanation, the user's voice signal received by the server from the terminal 2 is referred to as an input voice signal.

  The storage unit 32 includes, for example, at least one of a semiconductor memory, a magnetic recording device, and an optical recording device. And the memory | storage part 32 memorize | stores the various programs for the control part 33 to control the server 3, the application program which operate | moves on the server 3, and various data required for execution of these programs. For example, the storage unit 32 stores various information used for speech recognition or speech synthesis, various contents requested by the user, or addresses of external information sources. Further, the storage unit 32 stores various types of information used for determining the length of the message voice reproduced during the response time and the content of the message voice.

  The control unit 33 includes one or more processors, a readable / writable semiconductor memory, and a peripheral circuit. And the control part 33 extracts a keyword from an input audio | voice signal, and searches the content which a user desires based on the keyword. Further, the control unit 33 estimates the response time based on the keyword, sets the time length of the message voice reproduced within the response time according to the estimated response time, and sets the message voice corresponding to the time length. create. Then, the control unit 33 returns the searched content and message voice to the terminal 2.

FIG. 2 is a functional block diagram of the control unit 33 of the server 3 included in the voice interaction system. The control unit 33 includes a voice recognition unit 34, a search unit 35, a delay estimation unit 36, a message generation unit 37, and a voice synthesis unit 38.
Each of these units included in the control unit 33 is, for example, a functional module realized by a computer program that operates on a processor included in the control unit 33. Note that these units included in the control unit 33 may be mounted on the server 3 as one integrated circuit that realizes the functions of the units.

  The voice recognition unit 34 extracts a keyword for content search registered in advance from the input voice signal. In order to extract a keyword, the speech recognition unit 34 may use any of various speech recognition methods such as a method using an acoustic model or a method using a dynamic time stretching method. In the present embodiment, the speech recognition unit 34 extracts a keyword using a word dictionary storing a keyword to be recognized and an acoustic model created in advance.

  The acoustic model is generated, for example, by dividing the pronunciation of a keyword into unit sounds such as phonemes or syllables, and connecting unit acoustic models corresponding to the unit sounds in accordance with the order of the unit sounds. Is remembered. The unit acoustic model and the acoustic model are each represented by, for example, a hidden Markov model (HMM). The speech recognition unit 34 uses an HMM representing a unit acoustic model, and based on one or more feature amounts extracted from a predetermined section of the input speech signal, the predetermined section for a specific unit sound is an estimated sound. Probability or likelihood is calculated.

  Specifically, the voice recognition unit 34 extracts feature amounts used for voice recognition from the input voice signal. For this purpose, the speech recognition unit 34 obtains a spectrum for each frame by performing frequency conversion such as fast Fourier transform for each frame having a predetermined frame length on the input speech signal. The frame length is set to, for example, about 10 milliseconds to 100 milliseconds. Based on the spectrum, the voice recognition unit 34 obtains, for example, a Mel Frequency Cepstral Coefficient (MFCC) or a power difference value between frames as a feature amount for each frame. For example, when the MFCC is calculated as the feature amount, the speech recognition unit 34 converts the spectrum of each frame into a mel scale power value, and then performs frequency conversion such as discrete cosine conversion again on the logarithm of the power value. To calculate the MFCC. Further, when obtaining the power difference value between the frames as the feature amount, the voice recognition unit 34 obtains, for example, the sum of the squares of the spectrum for each frequency band of each frame as the power, and the power between the two consecutive frames. The difference value is obtained by obtaining the difference.

  Note that the speech recognition unit 34 may extract any of various other feature amounts (for example, fundamental frequency) used in speech recognition using an acoustic model as the feature amount. The speech recognition unit 34 may extract a plurality of types of feature amounts for each frame from the input speech signal.

  The speech recognition unit 34 collates the acoustic model with a set of feature values obtained from one or more frames, thereby for each keyword included in the word dictionary, the likelihood for each keyword generated by concatenating unit sounds. Find the degree. Then, the voice recognition unit 34 extracts a predetermined number of keywords in descending order of likelihood. The predetermined number is set to about 1 to 5, for example. Then, each time a keyword is detected, the voice recognition unit 34 notifies the search unit 35, the delay estimation unit 36, and the message generation unit 37 of the keyword.

  The search unit 35 specifies an information source corresponding to the detected keyword by referring to an information source table representing the correspondence relationship between the keyword and the information source. And the search part 35 acquires the content corresponding to a keyword from the information source. The content includes, for example, at least one of text information, image data, and an application program.

  FIG. 3 is a diagram illustrating an example of the information source table. At least one keyword is stored in each column in the left column of the information source table 300. On the other hand, each column in the right column of the information source table 300 stores address information indicating information sources corresponding to the keywords shown in the same row. This address information is, for example, a uniform resource locator (URL) for specifying the external information source 5 or a file name including specific content stored in the storage unit 32. One address information may be associated with a plurality of keywords, and one keyword may be associated with a plurality of address information.

The search unit 35 refers to the information source table and identifies address information corresponding to the input keyword. And the search part 35 accesses the information source shown by address information, and acquires the content corresponding to a keyword. For example, the search unit 35 reads a file having the file name indicated in the address information from the storage unit 32. Alternatively, the search unit 35 receives the web page specified by the URL from the external information source 5 specified by the URL indicated in the address information via the communication network 4. And the search part 35 analyzes the source | sauce of the received web page, and extracts the text information and image information which are displayed on a screen within a web page as a content.
The search unit 35 outputs the content to the communication unit 31. If the text information is included in the content, the search unit 35 passes the text information to the speech synthesis unit 38.

  The delay estimation unit 36 obtains an estimated value of response time corresponding to the extracted keyword. In the present embodiment, the delay estimation unit 36 refers to a response time table that is stored in the storage unit 32 and represents a correspondence relationship between a keyword and an estimated response time, so that the response time corresponding to the extracted keyword is obtained. Is estimated. In addition, the estimated value of the response time for each keyword is, for example, a statistical representative value of response time when the server 3 has received an input voice signal including the keyword in the past (for example, an average value, a median value of response times, or Maximum value).

  FIG. 4 is a diagram illustrating an example of a response time table. At least one keyword is stored in each column in the left column of the response time table 400. On the other hand, each column in the right column of the response time table 400 stores response times corresponding to the keywords shown in the same row. For example, when the keyword is “news”, the response time is 5 seconds.

  The delay estimation unit 36 notifies the message generation unit 37 of the estimated response time.

  The message generation unit 37 determines the time length of the message sound reproduced during the response time so that the user's quality of experience is the best, and the text information of the message that is the source of the message sound according to the length is obtained. Generate.

FIG. 5 is a diagram illustrating an example of the relationship between response time and message voice. In the present embodiment, the message voice 503 is reproduced between the silence period 501 and the silence period 502 within the response time 500. In the present embodiment, the silent period 501 and the silent period 502 are set to the same time length. This is because one silent period is lengthened and the user's quality of experience is suppressed from being lowered.
In the following description, the time length of the silent period is the total time length of the silent period 501 and the silent period 502.

  FIG. 6A is a schematic diagram showing the relationship between the time length and silence period of the message voice and the user's quality of experience. In FIG. 6A, the vertical axis represents the response time, and the horizontal axis represents the time length of the message voice. The time obtained by subtracting the time length of the message voice from the response time is a silent period included in the response time. Although not shown, the vertical direction perpendicular to the paper surface represents the quality value of the user's experience. This experience quality value is represented by a mean opinion score (MOS) value. The MOS value takes a value of 1 to 5, and the higher the MOS value, the higher the quality of experience of the user. A line 601 represents a case where the response time is equal to the time length of the message voice and the silence period is zero. Further, lines 602 and 603 represent cases in which the response time includes a silence period of 2 seconds and a silence period of 4 seconds in addition to the time length of the message voice. The numerical value on the line 602 is the MOS value when the length of the silent period is 2 seconds in each time length of the message voice, and the numerical value on the line 603 is the silent value in each time length of the message voice. This is the MOS value when the length of the period is 4 seconds. However, the MOS value is the sensation quality at the playback start position of the message voice where the sensation quality is maximum with respect to the response time, that is, at the playback start position where the silent time before and after is equal (line 602 is the silence time before and after the message voice). Is 1 second and the line 603 is 2 seconds. Further, in FIG. 6B, a graph 610 represents the relationship between the response time and the user's quality of experience when the time length of the message voice is L. In FIG. 6B, the horizontal axis represents the response time, and the vertical axis represents the MOS value. In FIG. 6B, Q1 represents the quality of experience when the response time includes a silent period of 2 seconds, and Q2 represents the quality of experience when the response time includes 4 seconds of silence. That is, the MOS values on the lines 602 and 603 in FIG. 6A correspond to Q1 and Q2 in FIG.

  As shown in FIG. 6 (b), if the length of the message voice is constant, if the length of the silent period is 2 seconds or less, the user's quality of experience is almost constant at the quality of experience Q1 at 2 seconds. It is. On the other hand, when the length of the silence period is 2 seconds or more and 4 seconds or less, the quality of experience decreases linearly with respect to the length of the silence period from the quality of experience Q1 at 2 seconds to the quality of experience Q2 at 4 seconds. . Although not shown in the figure, if the length of the silence period is longer than 4 seconds, the user's bodily sensation quality deteriorates sharply, so it is necessary to select a message voice whose silence period is 4 seconds or less with respect to the response time There is. As described above, if the user experience quality Q1 and Q2 when the length of the silent period is 2 seconds and 4 seconds for each message voice length is measured in advance, the message generator 37 It is possible to estimate a user's quality of experience for an arbitrary length of message voice at an arbitrary response time.

  For example, when the estimated response time is D, the message generation unit 37 may identify the length of the message voice that maximizes the MOS value along the line 604. In FIG. 6C, a graph 620 represents the relationship between the length of the message voice and the user's quality of experience along the line 604. In FIG. 6C, the horizontal axis represents the length of the message voice, and the vertical axis represents the MOS value. In this example, the MOS value is maximized when the length of the message sound is included in the period 621 or when it is included in the period 622, but the message sound having the maximum MOS value in this way. When there are a plurality of lengths, the message generator 37 selects the shortest message voice.

  In order to set the length of the message voice as described above, the message generator 37 uses a quality evaluation table that represents the relationship between the length of the message voice and the silence period and the quality of the user's experience.

  FIG. 8 is a diagram illustrating an example of a quality evaluation table that represents the relationship between the length and silence period of a message voice and the user's quality of experience. In the quality evaluation table 800, each column “message length” in the leftmost column stores a sample value of the length of the message voice. On the other hand, in each column “Q1” in the center column of the quality evaluation table 800, the maximum value of the silent period in which the user's sensation quality is substantially constant (first time) is added to the sample value of the message voice length in the same column. (Silence period), that is, the MOS value (first quality value) when a 2-second silence period is added is stored. Similarly, in each column “Q2” in the rightmost column of the quality evaluation table 700, the maximum length of the silent period in which the user's bodily sensation quality decreases at a certain rate is added to the sample value of the message voice length in the same column (the first column). 2), that is, the MOS value (second quality value) when a 4-second silence period is added.

When the length of the message voice is determined, the message generator 37 creates text information of the message corresponding to the length. In the present embodiment, the message generation unit 37 creates message text information by combining a plurality of pre-registered standard messages and input keywords. The standard message is a message that follows the keyword such as “is” or “I will tell you” and is stored in the storage unit 32 in advance. In addition, the storage unit 32 stores a fixed message table representing the correspondence between a fixed message and the time length of the fixed message. Then, the message generator 37 sets the remaining length obtained by subtracting the length of the input keyword from the length of the message voice as the time length of the standard message. Then, the generation unit 37 selects a fixed message having a time length closest to the set time length by referring to the fixed message table. The message generation unit 37 creates text information of the message by combining the input keyword and the standard message, for example, by inserting the keyword at a designated position in the standard message. For example, if the input keyword is “news” and the selected standard message is “I will tell you”, the message generator 37 will use “I will tell you the news” as the text information of the message.
The message generator 37 notifies the text synthesizer 38 of the message text information.

FIG. 9 is an operation flowchart of message generation processing executed by the message generation unit 37.
The message generation unit 37 calculates the shortest candidate length Lmin (= D-4) of the message voice, which is a time length obtained by subtracting 4 seconds, which is the maximum allowable time of the silent period, from the estimated response time D. Further, the message generator 37 calculates a reference length Lref (= D−2) of the message voice, which is a time length obtained by subtracting the maximum value of 2 seconds of the silent period during which the quality of experience is substantially constant from the estimated value D of the response time. (Step S101).

また、最短候補長L_minが品質評価テーブルに格納されたメッセージ音声の長さL_iとL_i+1の間に含まれる場合、メッセージ生成部３７は、（１）式に従ってL_iに対するMOS値Q_iとL_i+1に対するMOS値Q_i+1を算出する。そしてメッセージ生成部３７は、（１）式と同様に、L_i+1とL_iとの差に対する、L_minとL_iの距離の比に応じて、MOS値Q_iとMOS値Q_i+1とを線形補間することにより、最短候補長L_minに対するMOS値Q_minを算出する（図７（ａ））。同様に、メッセージ生成部３７は、基準長L_refについてのMOS値Q_refを算出すればよい。 Next, the message generator 37 includes the length of the message voice included between the shortest candidate length L _min and the reference length L _ref out of the message voice lengths stored in the quality evaluation table (for example, FIG. 8). The maximum value Qa of the MOS values for each is calculated (step S102). For example, in the quality evaluation table, if L ₂ , L ₃ , and L ₄ are included between the shortest candidate length L _min and the reference length L _ref , the message generation unit 37 performs L _min , L ₂ , L ₃ , The MOS value is calculated for each of L ₄ and L _ref . At that time, the message generation unit 37 follows the following expression for the message length Lm based on the MOS value Q ₁ when the silence period is 2 seconds and the MOS value Q ₂ when the silence period is 4 seconds. by linear interpolation, and calculates the MOS value Q _m corresponding to the length Lm of the message.

Further, if the shortest candidate length L _min is comprised between the length L _i and L _{i + 1} of the voice message stored in the quality evaluation table, the message generator 37, (1) MOS value for L _i according formula The MOS value Q _{i + 1} for Q _i and L _{i + 1} is calculated. Then, similarly to the equation (1), the message generator 37 determines the MOS value Q _i and the MOS value Q _{i +} according to the ratio of the distance between L _min and L _i with respect to the difference between L _{i + 1} and L _i. by _one and the linear interpolation, and calculates the MOS value Q _min to the shortest candidate length L _min (FIG. 7 (a)). Similarly, the message generator 37 may calculate the MOS value Q _ref for the reference length L _ref .

Next, the message generation unit 37 determines the message voice length L _n included in the response time estimation value D from the reference length L _ref out of the message voice lengths stored in the quality evaluation table. L _{n + k} with by calculating the maximum value Q _b of the MOS value (step S103). In this case, since the silence period is less than 2 seconds, the message generation unit 37 sets the MOS value Q _j1 of the silence period 2 seconds for the length L _j (n ≦ j ≦ n + k) to the length L _j . The MOS value may be used. Also, the shorter the message voice, the higher the quality of experience, so the MOS value Q _n of the shortest L _n among the message voice lengths L _n ,..., L _{n + k} is the maximum value Qb to be obtained (FIG. 7B). ).

The message generation unit 37 sets the length of the message voice corresponding to the larger MOS value of the MOS values Qa and Qb as the length L of the message voice to be created, and calculates the length from the estimated response time D. The time length obtained by subtracting the length L is set as a silent period (step S104). Further, the message generation unit 37 sets 1/2 of the silent period as a reproduction delay period from when the user's voice input to the terminal 2 is finished until the message voice is reproduced.
When there are a plurality of message voices having the maximum MOS value, it is preferable that the message generator 37 selects the shortest message voice. Or the message production | generation part 37 may set the length of a message voice so that the length of the combination of a fixed message and a keyword may match among the lengths of these several message voices.

  The message generator 37 calculates the time length of the standard message by subtracting the keyword time length W from the length L of the message voice (step S105). Note that the message generation unit 37 determines the keyword time length W by referring to, for example, a keyword table representing a correspondence relationship between the keyword and its time length. Alternatively, the message generator 37 may determine the time length W of the keyword by multiplying the number of syllables included in the keyword by the unit time per syllable.

The message generator 37 selects a fixed message having a time length closest to the set time length by referring to the fixed message table (step S106). Then, the message generator 37 generates message text information by combining the keyword and the selected standard message (step S107).
The message generator 37 outputs the text information of the message to the speech synthesizer 38 and also outputs the reproduction delay period to the communication unit 31 and ends the message generation process.

  Note that according to the modification, the message generation unit 37 may use the fixed message itself having the message voice length L as the text information of the message. In this case, the fixed message is a message having meaning only by the fixed message, for example, “Please wait for a while” or “Now searching”. In this case, the processes in steps S105 and S107 are omitted.

Further, according to another modification, the message generation unit 37 sets quality values for all message voice lengths shorter than the estimated response time D among a plurality of message voice lengths recorded in the quality evaluation table. It may be calculated. In this case, the message generator 37 does not have to calculate the shortest candidate length L _min . Instead, the message generation unit 37 preferably sets the quality value for the message voice length in which the length of the silence period is longer than 4 seconds to a very low value, for example, 1.0.

The voice synthesizer 38 creates a synthesized voice signal of text included in the content. The voice synthesizer 38 synthesizes the message voice based on the text information of the message reproduced during the response time.
First, the speech synthesizer 38 converts the text to be synthesized into speech into phonetic information. The phonetic information is information representing the reading of the original text included in the text. For example, the phonetic information is information in which the reading of the original text is expressed by katakana characters, and further the position of the accent and the position of the break are added.

  The speech synthesizer 38 reads a language dictionary stored in the storage unit 32 in order to convert text into phonetic information. In the language dictionary, for example, various words assumed to appear in the text information, readings of the words, parts of speech, and utilization forms are registered. Then, for example, using the language dictionary, the speech synthesizer 38 performs morphological analysis on the original sentence included in the text, and determines the text reading, accent position, and break position in the original sentence. At that time, the speech synthesizer 38 sets, for example, a position where a punctuation mark is set in the original text as a break position.

  The speech synthesis unit 38 can use, for example, a method using dynamic programming as the morphological analysis. Then, the speech synthesizer 38 creates phonetic information according to the reading of each word, the position of the accent, and the position of the break.

  Next, the speech synthesizer 38 generates a target prosody for generating synthesized speech based on the phonetic information. For this purpose, the speech synthesis unit 38 reads a plurality of prosodic dictionaries from the storage unit 32. This prosodic dictionary stores prosodic models representing changes in voice pitch and phoneme length over time. Then, the speech synthesizer 38 applies the prosodic model that most closely matches the position in the sentence or the accent position indicated in the phonetic information from the prosodic dictionary. Then, the speech synthesizer 38 creates a target prosody corresponding to the phonetic information in accordance with the applied prosodic model and preset synthesis parameters. Note that the synthesis parameters include, for example, a parameter representing speech speed and a parameter representing voice pitch. Furthermore, the synthesis parameter may include a parameter representing inflection, volume, and the like. The target prosody includes the phoneme length and the pitch frequency for each phoneme that is a unit for determining a speech waveform. Further, the target prosody may include amplitude information of individual phoneme waveforms. Note that the phoneme can be, for example, one vowel or one consonant.

When the target prosody is determined, the speech synthesizer 38 creates a synthesized speech signal by, for example, a vocoder method or a waveform editing method.
For each phoneme, the speech synthesizer 38 selects a speech waveform closest to the phoneme length and pitch frequency of the target prosody from, for example, a plurality of speech waveforms registered in the speech waveform dictionary by pattern matching. For this purpose, the speech synthesis unit 38 reads a speech waveform dictionary from the storage unit 32. The speech waveform dictionary records a plurality of speech waveforms and an identification number of each speech waveform. The voice waveform is, for example, a waveform signal extracted in units of phonemes from voice signals obtained by recording various voices in which one or more narrators read various texts.
Furthermore, in order to enable the speech synthesizer 38 to connect the speech waveforms selected for each phoneme along the target prosody, the selected speech waveform and the waveform pattern of the corresponding phoneme indicated in the target prosody The deviation amount may be calculated as waveform conversion information.
The speech synthesizer 38 creates waveform generation information including the identification number of the speech waveform selected for each phoneme. The waveform generation information may further include waveform conversion information.

The voice synthesizer 38 creates a synthesized voice signal based on the waveform generation information. For this purpose, the speech synthesizer 38 reads the speech waveform signal corresponding to the speech waveform identification number of each phoneme included in the waveform generation information from the speech waveform dictionary stored in the storage unit 32. Then, the speech synthesizer 38 creates a synthesized speech signal by connecting each speech waveform signal continuously. When the waveform conversion information is included in the waveform generation information, the speech synthesizer 38 corrects each speech waveform signal according to the waveform conversion information obtained for the corresponding phoneme and continuously connects the speech waveform signals. By doing so, a synthesized speech signal is created.
The voice synthesizer 38 outputs a synthesized voice signal and message voice of text included in the content to the communication unit 31. And the communication part 31 transmits a reproduction | regeneration delay period to the terminal 2 via the communication network 4 with those synthetic | combination audio | voice signals.

FIG. 10 is an operation sequence diagram of dialogue control processing in the voice dialogue system 1.
First, the processing unit 25 of the terminal 2 acquires an input voice signal corresponding to the voice uttered by the user via the voice input unit 21 (step S201). Then, the processing unit 25 transmits the input audio signal to the server 3 via the communication unit 23 and the communication network 4. In addition, the processing unit 25 starts measuring the elapsed time from the time when the input audio signal ends.

  When the server 3 receives the input voice signal, the voice recognition unit 34 of the control unit 33 extracts keywords included in the input voice signal (step S202).

  Then, the delay estimation unit 36 of the control unit 33 obtains an estimated value D of the response time corresponding to the keyword (Step S203). Then, the message generation unit 37 of the control unit 33 executes a message generation process to obtain a message reproduced during the response time and a reproduction delay period (step S204).

  The voice synthesizer 38 of the control unit 33 synthesizes the voice of the message (step S205). Then, the server 3 notifies the terminal 2 of the synthesized message voice and the reproduction delay period.

  The processing unit 25 of the terminal 2 causes the audio output unit 22 to reproduce the message audio when the reproduction delay period elapses after the voice input by the user is completed (step S206).

  On the other hand, the search unit 35 of the control unit 33 specifies the address information of the information source corresponding to the keyword (step S207). Then, the search unit 35 acquires content from the information source indicated by the address information (step S208).

The voice synthesis unit 38 of the control unit 33 synthesizes a voice signal corresponding to the text included in the content (Step S209). Then, the server 3 notifies the terminal 2 of the synthesized speech signal of the content and the text included in the content.
When the processing unit 25 of the terminal 2 receives the synthesized voice signal of the content and the text included in the content, the processing unit 25 causes the voice output unit 22 to reproduce the synthesized voice signal (step S210).
Then, the voice dialogue system 1 ends the dialogue control process.

  As described above, in this voice dialogue system and the dialogue control method in the voice dialogue system, the length of the message voice reproduced during the response time is related to the length and silence period and the user's quality of experience. It is determined with reference to the quality evaluation table representing. For this reason, the voice dialogue system and the dialogue control method can set the length of the message voice so that the quality of experience of the user is the best. Furthermore, in this voice interactive system and the interactive control method, the user experience quality is substantially constant when the silent period is less than 2 seconds, while linearly according to the length of the silent period when the silent period is 2 seconds or longer and 4 seconds or shorter. The user's experience quality is determined based on the knowledge that the user's experience quality is lowered. Therefore, this voice dialogue system and dialogue control method can determine an appropriate message voice length for an arbitrary response time with a relatively small number of samples.

  Next, a voice dialogue system according to the second embodiment and a dialogue control method in the voice dialogue system will be described. The voice dialogue system and the dialogue control method according to the second embodiment adjust the length of the message voice according to the user's preference.

  Note that the voice interaction system according to the second embodiment differs from the voice interaction system according to the first embodiment in some of the functions realized by the control unit of the server. Therefore, in the following description, differences from the first embodiment among the functions realized by the control unit of the server will be described. For other components of the spoken dialogue system, refer to the description of the corresponding components and the diagram for the first embodiment.

FIG. 11 is a functional block diagram of the control unit 40 of the server 3 according to the second embodiment. The control unit 40 includes a voice recognition unit 34, a search unit 35, a delay estimation unit 36, a message generation unit 37, a voice synthesis unit 38, and a preference determination unit 39.
Each of these units included in the control unit 40 is, for example, a functional module realized by a computer program that operates on a processor included in the control unit 40. Note that these units included in the control unit 40 may be mounted on the server 3 as one integrated circuit that realizes the functions of the units.

  The control unit 40 according to the second embodiment is different from the control unit 33 according to the first embodiment in that it includes a preference determination unit 39. Therefore, hereinafter, the preference determination unit 39 and its related parts will be described.

  In the second embodiment, the user identification number is transmitted from the terminal 2 to the server 3 prior to voice input. For example, when the communication network 4 is a telephone line, the user identification number can be the telephone number of the terminal 2. In this case, the server 3 can acquire the user identification number by transmitting the telephone number from the terminal 2 to the server 3 at the time of call connection between the terminal 2 and the server 3. Further, the user identification number may be a number assigned separately from the terminal 2. In this case, after the connection between the terminal 2 and the server 3 is established, the terminal 2 transmits, for example, the user identification number input via the operation unit (not shown) of the terminal 2 to the server 3. You may send it.

  The preference determination unit 39 specifies a user preference corresponding to the user identification number received from the terminal 2 by referring to a preference information table that represents a correspondence relationship between the user identification number and the message length preference. .

  FIG. 12 is an example of a preference information table. In each column of the left column of the preference information table 1200, user identification numbers are stored. On the other hand, each column in the right column of the preference information table 1200 stores a preference type corresponding to the user identification number stored in the column adjacent to the left side of the column. In the present embodiment, three types of “succinct type”, “uniform type”, and “redundant type” are set as preference types. For example, each user selects one of “concise type”, “equal type”, and “redundant type” when registering in the voice interaction system 1. Then, the preference information table is created or updated based on the selection result.

  The “concise type” is a type in which, for example, “it is news”, the length of the message voice in the response period may be relatively short, and the silent period may be longer correspondingly. In addition, the “redundant type” is a type that prefers that the length of the message voice in the response period is relatively long and the silence period is short, for example, “We are currently searching for news. Please wait for a while”. . The “uniform type” is a type that prefers that the length of the message voice in the response period is longer than the “concise type” and shorter than the “redundant type”, for example, “I will tell you the news”.

In the second embodiment, when a plurality of users are classified into any of the above three types, and for each type, when the silence period is set to 2 seconds and set to 4 seconds for each of the plurality of message voice lengths A quality evaluation table is created in advance by obtaining the MOS value of. Each quality evaluation table is stored in the storage unit 32 in association with the preference type.
In general, if the time length of the message voice is the same, the “redundant” type has a higher MOS value as the silence period is shorter than the other types, and conversely, the longer the silence period, the higher the MOS value. The value becomes lower. In addition, in the “concise type” type, the MOS value is lower as the silence period is shorter than the other types, and conversely, the MOS value is higher as the silence period is longer.

  FIG. 13 is an operation sequence diagram of the dialog control process according to the second embodiment. The operation sequence shown in FIG. 13 differs from the operation sequence of the dialog control process according to the first embodiment shown in FIG. 10 in the process of step S301. Therefore, step S301 will be described below. .

After step S203, the preference determination unit 39 of the control unit 40 refers to the preference information table, identifies the preference type corresponding to the user identification number, and stores the quality evaluation table corresponding to the identified type in the storage unit 32. (Step S301). Then, the preference determination unit 39 passes the read quality evaluation table to the message generation unit 37.
In step S204, the message generation unit 37 determines the length of the message voice and the reproduction delay period using the quality evaluation table received from the preference determination unit 39. Then, the message generator 37 creates a message text corresponding to the length. In this embodiment as well, the reproduction delay period is preferably set to ½ of the silence period included in the response time.

  According to the second embodiment, since the voice dialogue system and the dialogue control method can adjust the length of the message voice inserted during the response time according to the user's preference, the user experience about the voice dialogue system can be adjusted. The quality can be further improved.

  According to the modification to each of the above embodiments, the delay estimation unit 36 may estimate the delay time based on information other than the keyword. For example, the delay estimation unit 36 acquires information representing the traffic status of the communication network 4 at the time of receiving the input voice from the management device (not shown) that manages the communication network 4 via the communication unit 31. Also good. Then, the delay estimation unit 36 may estimate the delay time by referring to a table representing a correspondence relationship between the traffic situation and the estimated value of the delay time. Alternatively, the delay estimation unit 36 may estimate the delay time according to the number of terminals 2 that are connected to the server 3.

  According to another modification, the control unit of the server 3 may update the response time table. In this case, for example, every time a user's voice input is performed, the terminal 2 measures a response time from the voice input to content reproduction, and transmits the measured value to the server 3 via the communication unit 23. Each time the control unit of the server 3 receives a response time measurement value for each keyword, the control unit obtains an average value of the past response time measurement value and the latest response time measurement value for the keyword. Update the response time value for that keyword in the response time table.

  According to still another modification, the processing unit 25 of the terminal 2 may have functions of a speech recognition unit and a speech synthesis unit. Thereby, the processing load of the server 3 is reduced.

  According to still another modification, each unit included in the terminal and each unit included in the server in the above-described embodiment or the modification thereof may be mounted on one device. One or a plurality of processors included in the apparatus may execute each function of the processing unit of the terminal and each function of the control unit of the server.

  Furthermore, a computer program that causes a computer to realize each function of the control unit of the server according to each of the above embodiments or modifications thereof may be provided in a form recorded on a computer-readable recording medium. The computer-readable recording medium can be, for example, a magnetic recording medium, an optical recording medium, or a semiconductor memory. However, the recording medium does not include a carrier wave.

  All examples and specific terms listed herein are intended for instructional purposes to help the reader understand the concepts contributed by the inventor to the present invention and the promotion of the technology. It should be construed that it is not limited to the construction of any example herein, such specific examples and conditions, with respect to showing the superiority and inferiority of the present invention. Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention.

The following supplementary notes are further disclosed regarding the embodiment described above and its modifications.
(Appendix 1)
A predetermined keyword is extracted from the voice signal representing the voice of the user collected by the voice input unit;
Search for content according to the keyword,
Estimating the response time from the end of the user's voice input to the presentation of the content to the user,
By referring to a table representing a correspondence relationship between a plurality of combinations of a sample value of a length of a silent period in which no voice is output and a sample value of a length of a message voice in the response time and a user's bodily sensation quality, Set the length of the message voice,
Generating the message voice with the length;
Playing the message audio during the response time;
Dialog control method including that.
(Appendix 2)
The table shows, for each sample value of the message voice length, the user's experience quality when adding a silence period of a first time that is the maximum length of the silence period in which the experience quality is constant. The user's sensation when adding a first quality value to be expressed and a second period of silence that is longer than the first time and is the maximum length of the silence period in which the sensation quality decreases at a constant rate Storing a second quality value representing quality,
Setting the length of the message voice corresponds to the length from the first quality value and the second quality value for each sample value of the message voice length below the response time. The dialogue control method according to supplementary note 1, wherein a quality value representing a user's bodily sensation quality is calculated, and a length at which the quality value is maximized is defined as a length of the message voice.
(Appendix 3)
Setting the length of the message voice means that when the difference obtained by subtracting the sample value of the message voice length from the response time is shorter than the first time, the sample time of the length is set. If the first quality value is the quality value for the sample value of the length, while the difference is not less than the first time and not more than the second time, the first quality value The dialogue control method according to appendix 2, wherein the quality value is calculated by linearly interpolating a quality value and the second quality value.
(Appendix 4)
Further including selecting a table corresponding to the user's preference from among the plurality of tables corresponding to the preference for the length of the message voice;
The dialogue control method according to any one of appendices 1 to 3, wherein setting the length of the message voice sets the length of the message voice with reference to the selected table.
(Appendix 5)
Setting the length of the message voice sets the timing to play the message voice at the time when the user's voice has ended by half the difference between the response time and the length of the message voice. The dialog control method according to any one of appendices 1 to 4.
(Appendix 6)
The message voice is generated by selecting a fixed message having a time length obtained by subtracting the length of the keyword from the length of the message voice from among a plurality of fixed messages stored in the storage unit in advance. The dialogue control method according to any one of appendices 1 to 4, wherein the message voice is generated by combining the selected fixed message and the keyword.
(Appendix 7)
A predetermined keyword is extracted from the voice signal representing the voice of the user collected by the voice input unit;
Search for content according to the keyword,
Estimating the response time from the end of the user's voice input to the presentation of the content to the user,
By referring to a table representing a correspondence relationship between a plurality of combinations of a sample value of a length of a silent period in which no voice is output and a sample value of a length of a message voice in the response time and a user's bodily sensation quality, Set the length of the message voice,
Generating the message voice with the length;
Playing the message audio during the response time;
A computer program for interactive control for causing a computer to execute the above.
(Appendix 8)
A voice recognition unit that extracts a predetermined keyword from a voice signal representing the user's voice collected by the voice input unit;
A search unit for searching for content according to the keyword;
A delay estimation unit that estimates a response time from the end of input of the user's voice to presentation of the content to the user;
By referring to a table representing a correspondence relationship between a plurality of combinations of a sample value of a length of a silent period in which no voice is output and a sample value of a length of a message voice in the response time and a user's bodily sensation quality, A message generator for setting the length of the message voice and generating the message voice having the length;
A playback time control unit for causing the voice output unit to play the message voice during the response time;
Spoken dialogue apparatus having

DESCRIPTION OF SYMBOLS 1 Voice dialogue system 2 Terminal 3 Server 4 Communication network 5 External information source 21 Voice input part 22 Voice output part 23 Communication part 24 Storage part 25 Processing part 31 Communication part 32 Storage part 33, 40 Control part 34 Voice recognition part 35 Search part 36 delay estimation unit 37 message generation unit 38 speech synthesis unit 39 preference determination unit

Claims (5)

A predetermined keyword is extracted from the voice signal representing the voice of the user collected by the voice input unit;
Search for content according to the keyword,
Estimating the response time from the end of the user's voice input to the presentation of the content to the user,
By referring to a table representing a correspondence relationship between a plurality of combinations of a sample value of a length of a silent period in which no voice is output and a sample value of a length of a message voice in the response time and a user's bodily sensation quality, Set the length of the message voice,
Generating the message voice with the length;
Playing the message audio during the response time;
Dialog control method including that. The table shows, for each sample value of the message voice length, the user's experience quality when adding a silence period of a first time that is the maximum length of the silence period in which the experience quality is constant. The user's sensation when adding a first quality value to be expressed and a second period of silence that is longer than the first time and is the maximum length of the silence period in which the sensation quality decreases at a constant rate Storing a second quality value representing quality,
Setting the length of the message voice corresponds to the length from the first quality value and the second quality value for each sample value of the message voice length below the response time. The dialogue control method according to claim 1, wherein a quality value representing a user's bodily sensation quality is calculated, and a length at which the quality value is maximized is set as a length of the message voice.   Setting the length of the message voice means that when the difference obtained by subtracting the sample value of the message voice length from the response time is shorter than the first time, the sample time of the length is set. If the first quality value is the quality value for the sample value of the length, while the difference is not less than the first time and not more than the second time, the first quality value The dialogue control method according to claim 2, wherein the quality value is calculated by linearly interpolating a quality value and the second quality value. Further including selecting a table corresponding to the user's preference from among the plurality of tables corresponding to the preference for the length of the message voice;
The dialogue control method according to any one of claims 1 to 3, wherein the setting of the length of the message voice sets the length of the message voice with reference to the selected table. A predetermined keyword is extracted from the voice signal representing the voice of the user collected by the voice input unit;
Search for content according to the keyword,
Estimating the response time from the end of the user's voice input to the presentation of the content to the user,
By referring to a table representing a correspondence relationship between a plurality of combinations of a sample value of a length of a silent period in which no voice is output and a sample value of a length of a message voice in the response time and a user's bodily sensation quality, Set the length of the message voice,
Generating the message voice with the length;
Playing the message audio during the response time;
A computer program for interactive control for causing a computer to execute the above.

Images