JP6590617B2 - Information processing method and apparatus - Google Patents

Description

  The present invention relates to a technique for generating a dialogue corpus relating to a dialogue between an automatic dialogue system and a human.

  The dialogue between the automatic dialogue system and the human being is recorded and a dialogue corpus is generated. By analyzing the dialogue corpus, human characteristics and problems of the automatic dialogue system are clarified, and the quality of the dialogue provided by the automatic dialogue system is improved. To be done. The dialogue quality indicators here include efficiency, time, ease of understanding, enjoyment, and comfort.

  As a conventional technology, in order to generate a dialogue corpus, utterance texts are sequentially output based on a script that shows pre-registered dialogue examples, and the dialogue history is also registered in the dialogue log database, which is registered as dialogue log data. There is a technique for recording speech data of a speaker according to the output of synthesized speech of speech text, and then texting the dialogue part of the speaker in the dialogue log database based on the speech data.

  This prior art shows that the output start time and output end time of synthesized speech are also recorded, but the output start time and output end time are recorded directly by the unit that performs speech synthesis output.

  However, generally, since a buffer exists in a unit that performs speech synthesis output (more specifically, a device driver for speech synthesis output), an event occurrence time of speech synthesis output and a time actually output to a human In the meantime, a difference of several tens of milliseconds to several hundred milliseconds occurs. Since this difference fluctuates depending on the load situation of the automatic dialog system, the actual output time cannot be specified from the log record of the event occurrence time of the speech synthesis output.

  In the evaluation of spoken dialogue, it is important to grasp the time from the sound output from the automatic dialogue system to the human reaction, but the conventional technology cannot correctly grasp the time.

JP 2001-166785 A

  Therefore, the objective of this invention is providing the technique for grasping | ascertaining the output time of the sound from an automatic dialog system correctly as one side surface.

  The information processing method according to the present invention includes (A) a specifying step of specifying an output time of an output sound from the automatic dialog system from recording data obtained by recording the output sound from the automatic dialog system and the user's voice; And (B) storing the output time of the identified output sound in the data storage unit in association with the data of the event related to the output sound from the automatic dialogue system.

  According to one aspect, the output time of the sound from the automatic dialog system can be accurately grasped.

FIG. 1 is a diagram illustrating a configuration example of a system according to an embodiment. FIG. 2 is a diagram illustrating an example of data. FIG. 3 is a diagram illustrating an example of event data before correction. FIG. 4 is a diagram illustrating a processing flow according to the embodiment. FIG. 5 is a diagram illustrating an example of event data after correction.

  A configuration example of a system according to an embodiment of the present invention is shown in FIG.

  The system according to the present embodiment includes an automatic dialog system 100 and a recording system 200 connected to the automatic dialog system 100 via a network or the like. The automatic dialogue system 100 includes a dialogue control unit 101, a dialogue script storage unit 102, a touch panel display unit 103, a touch panel input unit 104, a touch panel 105, a voice recognition unit 106, a microphone 107, and a voice synthesis unit 108. A sound effect reproduction unit 109, a speaker 110, a sound DB 111, a sound effect DB 112, an event recording unit 113, and an output unit 114.

  The dialogue script storage unit 102 stores data representing a dialogue scenario. The dialogue scenario is, for example, “display a button on the touch panel and sound a sound effect“ ping-pong ”when the button is pressed”.

  The dialogue control unit 101 instructs the touch panel display unit 103, the voice recognition unit 106, the voice synthesis unit 108, the sound effect reproduction unit 109, and the like according to the dialogue scenario stored in the dialogue script storage unit 102, and the touch panel input unit. 104, based on input data from the voice recognition unit 106 and the like, control is performed in accordance with the dialogue scenario.

  The touch panel display unit 103 displays images and text on the touch panel 105 based on instructions from the dialogue control unit 101. Touch panel input unit 104 outputs the touch content input on touch panel 105 to dialog control unit 101. The voice recognition unit 106 takes in voice data from the microphone 107 based on an instruction from the dialogue control unit 101, converts the voice data into text data by voice recognition, and outputs the text data to the dialogue control unit 101. In the present embodiment, the microphone 107 captures not only the user's speech but also the output sound from the speaker 110 as sound data, and outputs the sound data to the voice recognition unit 106 and the output unit 114.

  The voice synthesizer 108 causes the speaker 110 to output voice data of a voice message stored in the voice DB 111 based on an instruction from the dialogue control unit 101. The sound effect reproducing unit 109 causes the speaker 110 to output the sound effect stored in the sound effect DB 112 based on an instruction from the dialogue control unit 101.

  The event recording unit 113 includes processing contents related to events processed by the touch panel display unit 103, the touch panel input unit 104, the voice recognition unit 106, the voice synthesis unit 108, the sound effect reproduction unit 109, and the like in response to an instruction from the dialogue control unit 101. Processing time (start time and end time) data (also referred to as event data) is received and recorded from them, and output to the output unit 114. The processing content includes, for example, message output, sound effect output, data display, data input, and voice input (more specifically, voice recognition is performed).

  The output unit 114 reads out sound data stored in the sound DB 111 and sound effect data stored in the sound effect DB 112 based on the event data from the event recording unit 113, and sound data from the microphone 107 (hereinafter, referred to as “sound data”). Together with event data) and event data.

  As described above, in the automatic dialogue system 100 according to the present embodiment, the microphone 107 captures not only the user's speech but also the output sound from the speaker 110 as sound data, and generates recording data. Recording data, event data, and sound data (sound data stored in the sound DB 111 and sound effect data stored in the sound effect DB 112) are mainly output to the recording system 200.

  The recording system 200 includes a data acquisition unit 201, a recording data storage unit 202, an event data storage unit 203, a sound data storage unit 204, a selection unit 205, a collation unit 206, a data correction unit 207, and an extraction unit. 208, a voice recognition unit 209, and a modified event data storage unit 210.

  The data acquisition unit 201 acquires recording data, event data, and sound data from the output unit 114 of the automatic dialog system 100, stores the recording data in the recording data storage unit 202, and stores event data in the event data storage unit 203. The sound data is stored in the sound data storage unit 204.

  The selection unit 205 reads the sound data stored in the sound data storage unit 204 based on the event data stored in the event data storage unit 203 and outputs the sound data to the collation unit 206. The collation unit 206 collates the recording data stored in the recording data storage unit 202 with the sound data, specifies the actual sound output start time and end time from the portion that matches, and the data correction unit It outputs to 207. The data correction unit 207 corrects the event data output from the selection unit 205 based on the actual sound output start time and end time data from the matching unit 206 and stores the data in the corrected event data storage unit 210. .

  The collation unit 206 outputs the actual sound output start time and end time to the extraction unit 208. Then, the extraction unit 208 extracts, from the recorded data, a portion where the voice data of the voice message and the sound data of the sound effect are not present and where the voice is recorded, and recognizes the recorded data of the extracted portion as a voice. Output to the unit 209.

  The voice recognition unit 209 performs voice recognition processing (including speaker recognition) on the recording data output from the extraction unit 208, and outputs the processing result to the data correction unit 207. The data correction unit 207 corrects the event data from the selection unit 205 based on the processing result of the voice recognition unit 209 and stores it in the corrected event data storage unit 210.

  More specific processing contents will be described with reference to FIGS.

  Hereinafter, the case where the event data as shown in FIG. 2A and the recording data as shown in FIG. 2A are acquired will be described as an example. 2A shows a state in which event data as shown in FIG. 3 is arranged in time series.

  Specifically, a “message 1 output” event that outputs a message 1 “Please tell me your address after a beeping sound” starts at time a and ends at time b. Thereafter, the “sound effect x output” event for outputting the sound effect x “pon” starts at time c and ends at time d. Thereafter, voice input by the user is performed, and the “voice input 1” event starts at time e and ends at time f. In the automatic dialogue system 100, since the voice recognition processing is performed by the voice recognition unit 106, the content A of the voice input 1 "Eat 27, Onoecho, Naka-ku, Yokohama-shi" is obtained as a recognition result.

  As can be seen by comparing the timing of the event shown in FIG. 2A with the timing of the speech waveform shown in FIG. 2A, the sound output by the speech synthesizer 108 and the sound effect output by the sound effect playback unit 109 It can be seen that the actual sound output is performed later than those recorded as the start time and the end time.

  On the other hand, for voice input, the voice recognition unit 106 specifies the start time and end time of voice input and then performs voice recognition processing. In this example, the start time and end time completely match the recorded data. .

  In the present embodiment, in the recording system 200, as shown in FIG. 2C, the start time and end time of the “message 1 output” event and the “sound effect x output” event are corrected based on the recording data. Furthermore, the speech recognition result is also corrected.

  The processing flow for that is shown in FIG. First, the data acquisition unit 201 receives event data, recording data, and sound data before correction from the output unit 114 of the automatic dialogue system 100, and an event data storage unit 203, a recording data storage unit 202, and a sound data storage unit, respectively. It stores in 204 (step S1).

  Then, the selection unit 205 reads event data of unprocessed events stored in the event data storage unit 203 for one event (step S3). Then, the selection unit 205 determines whether or not the event type of the read event data is related to sound (step S5). The event data may include event data related to display on the touch panel 105 and input to the touch panel 105. Therefore, when the event type of the read event data is not related to sound, the selection unit 205 outputs the read event data as it is to the data correction unit 207 and stores it in the correction event data storage unit 210 as it is. (Step S21). Then, the process proceeds to step S23.

  On the other hand, if the event type of the read event data relates to sound, the selection unit 205 determines whether or not the event type of the read event data relates to sound output (step S7). The sound output event is a sound output event by speech synthesis or a sound effect output event.

  If the event type of the read event data is related to sound output, the selection unit 205 extracts sound data corresponding to the event of the read event data from the sound data storage unit 204 from the sound data storage unit 204. And output to the collation unit 206 (step S9). Further, the selection unit 205 outputs the read event data to the data correction unit 207.

  The collation unit 206 reads the recording data from the recording data storage unit 202, collates the recording data with the sound data received from the selection unit 205, and specifies a matching portion in the recording data (step S11). That is, the collation unit 206 specifies the start time and the end time of the voice output in the recording data and outputs them to the data correction unit 207. Note that the start time and end time of the matching portion in the recording data are output to the extraction unit 208.

  The data correction unit 207 corrects the event data received from the selection unit 205 with the start time and end time of the sound output (sound effect or sound output) in the recording data, and stores the data in the correction event data storage unit 210 ( Step S13). Then, the process proceeds to step S23.

  In the case of the first event in FIG. 3, message 1 “Please tell me your address after the sound of popping” is output, so the sound data and recording data (FIG. 2 (A)) of message 1 are output. The start time aa and end time bb of the matching part are identified, and the first event data is corrected and stored in the corrected event data storage unit 210. For example, it is corrected like the first event data in FIG.

  Similarly, in the case of the second event in FIG. 3, since the sound effect x is output, the sound data of the sound effect x and the recorded data (FIG. 2 (a)) are collated to match The start time cc and the end time dd are specified, and the second event data is corrected and stored in the corrected event data storage unit 210. For example, it is corrected like the second event data in FIG.

  On the other hand, if the event type of the read event data is not related to sound output but related to sound input, the selection unit 205 outputs the read event data to the data correction unit 207 and the extraction unit 208. Then, the extraction unit 208 extracts recording data other than the sound effect part and the output sound part from the recording data stored in the recording data storage unit 202, and outputs the recording data to the voice recognition unit 209 (step S15). More specifically, it is recording data from the vicinity of the start time to the end time included in the event data related to the processing, and there is no sound effect and output sound specified by the collation unit 206, and the sound is recorded. The recorded data is extracted. Thereby, the recognition accuracy of voice recognition processing can be improved.

  Then, the voice recognition unit 209 performs voice recognition processing on the recording data from the extraction unit 208 and outputs the processing result to the data correction unit 207 (step S17). Note that the speech recognition processing includes known speaker recognition processing and the like.

  Then, the data correction unit 207 corrects the event data received from the selection unit 205 based on the voice recognition result by the voice recognition unit 209 and stores it in the correction event data storage unit 210 (step S19). Then, the process proceeds to step S23.

  In the example of FIG. 2, the speech recognition result “27 of Onoe-cho, Naka-ku, Yokohama-shi, 2” is obtained, so the result is the same as the event data. However, if the result of the speaker recognition is that the user A is obtained, the data “user A” of the speaker recognition result is also stored. Therefore, in the example of FIG. The content is corrected to “AA”.

  The selection unit 205 determines whether or not unprocessed event data exists in the event data storage unit 203 (step S23). If unprocessed event data exists, the process returns to step S3. On the other hand, if there is no unprocessed event data, the process ends.

  By executing the processing as described above, it is possible to generate event data in which a sound output delay caused by a buffer existing in the speech synthesis unit 108 or the sound effect reproduction unit 109 is corrected.

  If such accurate event data can be automatically generated, it becomes easy to clarify the problems of the dialog of the automatic dialog system, and it becomes easy to improve the quality of the dialog of the automatic dialog system.

  Although the embodiment of the present invention has been described above, the present invention is not limited to this. The system configuration shown in FIG. 1 is an example, and may not match the program module configuration or the file configuration. The automatic dialogue system 100 and the recording system 200 may be integrated. As for the processing flow, as long as the processing result does not change, the processing order may be changed or may be executed in parallel.

  Moreover, although the example which included start time and end time was shown about event data, you may make it data, such as start time and length, end time, and length. Further, in the example described above, an example in which data before correction is discarded is shown, but it may be left without being discarded. The time may be a relative time from some point in time.

  The automatic dialog system 100 and the recording system 200 described above are computer devices, and display control connected to a memory, a CPU (Central Processing Unit), a hard disk drive (HDD), and a display device. And a removable disk drive device, an input device, and a communication control unit for connecting to a network are connected by a bus. An operating system (OS) and an application program for performing the processing in this embodiment are stored in the HDD, and are read from the HDD to the memory when executed by the CPU. The CPU controls the display control unit, the communication control unit, and the drive device in accordance with the processing content of the application program, and performs a predetermined operation. Further, data in the middle of processing is mainly stored in the memory, but may be stored in the HDD. In the embodiment of the present invention, an application program for performing the above-described processing is stored and distributed on a computer-readable removable disk, and installed from the drive device to the HDD. In some cases, the HDD is installed via a network such as the Internet and a communication control unit. Such a computer apparatus realizes various functions as described above by organically cooperating hardware such as the CPU and memory described above with programs such as the OS and application programs.

  The above-described embodiment can be summarized as follows.

  The information processing method according to the embodiment includes (A) the output time of the output sound from the automatic dialog system (for example, the output start time) from the recording data obtained by recording the output sound from the automatic dialog system and the user's voice. And an output end time, an important one of them, one of them and the length thereof), and (B) an event related to the output sound from the automatic dialog system, the output time of the specified output sound Storing the data in the data storage unit in association with the data.

  By specifying the output time of the output sound from the automatic dialog system based on the recorded data, it becomes possible to specify the timing when the user actually heard the output sound, so it is possible to analyze the dialog more appropriately according to the actual situation. Will be able to do.

  In addition, the information processing method described above includes (C) a step of extracting data of a user's voice from recorded data using data about an output sound from the automatic dialogue system, and (D) an extracted user A voice recognition process may be performed on the voice data, and a result of the voice recognition process may be stored in the data storage unit in association with event data related to the user's voice. In this way, the accuracy of the speech recognition process can be improved. Note that speaker recognition may also be performed.

  Further, the specific step described above includes (a1) acquiring output sound data from the automatic dialog system based on the output sound type included in the event data related to the output sound from the automatic dialog system; a2) A step of identifying an output sound from the automatic dialog system in the recording data by comparing with data of the output sound from the automatic dialog system may be included. The timing can be specified with high accuracy.

  A program for causing a computer to perform the above method can be created, and the program is a computer-readable storage medium or storage device such as a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, or a hard disk. Stored in The intermediate processing result is temporarily stored in a storage device such as a main memory.

DESCRIPTION OF SYMBOLS 200 Recording system 201 Data acquisition part 202 Recording data storage part 203 Event data storage part 204 Sound data storage part 205 Selection part 206 Collation part 207 Data correction part 208 Extraction part 209 Voice recognition part 210 Correction event data storage part

Claims (5)

A specific step of specifying the output time of the output sound from the automatic dialog system from the recording data obtained by recording the output sound from the automatic dialog system and the user's voice;
The specified output time of the output sound corresponds to the event data related to the output sound from the automatic dialog system among the data of the event output from the automatic dialog system and including the processing type of the event And storing in the data storage unit,
A program that causes a computer to execute. Extracting data of the user's voice in the recording data using data about the output sound from the automatic dialogue system;
Performing speech recognition processing on the extracted user's voice data, and storing the result of the voice recognition processing in the data storage unit in association with event data related to the user's voice;
The program according to claim 1, further causing the computer to execute. The specific step includes
Obtaining output sound data from the automatic dialogue system based on the output sound type included in the event data relating to the output sound from the automatic dialogue system;
Identifying the output sound from the automatic dialog system in the recording data by comparing with the data of the output sound from the automatic dialog system;
The program according to claim 1 or 2, comprising: A specific step of specifying the output time of the output sound from the automatic dialog system from the recording data obtained by recording the output sound from the automatic dialog system and the user's voice;
The specified output time of the output sound corresponds to the event data related to the output sound from the automatic dialog system among the data of the event output from the automatic dialog system and including the processing type of the event And storing in the data storage unit,
An information processing method executed by a computer. Means for specifying the output time of the output sound from the automatic dialog system from the recording data obtained by recording the output sound from the automatic dialog system and the user's voice;
The specified output time of the output sound corresponds to the event data related to the output sound from the automatic dialog system among the data of the event output from the automatic dialog system and including the processing type of the event Means for storing in the data storage unit,
An information processing apparatus.

Images