JP2017068061A - Communication terminal and voice recognition system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a communication terminal and a voice recognition system for reducing communication load on a voice recognition processing server apparatus performing a voice recognition process on collected voice data.SOLUTION: A communication terminal transmits voice data produced by a user to a voice recognition processing server apparatus performing a voice recognition process, and receives a voice recognition processing result for the voice data. The communication terminal comprises: a volume measuring unit 112 that measures a volume of the voice data acquired by a voice input unit; and a voice data output control unit 113 that transmits the voice data to the voice recognition processing server apparatus. When the volumes of sequentially input voice data are lower than a predetermined threshold a recognition processing state of the voice recognition process on the voice data received from the voice recognition processing server apparatus is being an unrecognized state indicating a waiting state, the voice data output control unit 113 exerts a control not to transmit silence voice data to the voice recognition processing server apparatus.SELECTED DRAWING: Figure 2

Description

  Embodiments described herein relate generally to a voice recognition system that performs voice recognition processing on a user's voice collected by a communication terminal using a voice recognition processing server device and provides a voice recognition result to the communication terminal.

  Conventionally, there is a technology for recognizing a voice uttered by a user and converting it into text data. Since the voice recognition process has a high processing load, there is a server / client type voice recognition system in which voice data is transmitted from the client side and the voice recognition process is performed by the server device.

Japanese Patent No. 4197271

  Provided are a communication terminal and a voice recognition system capable of reducing a communication load on a voice recognition processing server device that performs voice recognition processing on voice data collected by a communication terminal.

  The communication terminal according to the embodiment transmits voice data issued by a user to a voice recognition processing server apparatus that performs voice recognition processing, and receives a voice recognition processing result for the voice data from the voice recognition processing server apparatus. The communication terminal includes a volume measuring unit that measures the volume of the voice data acquired by the voice input unit, and a voice data output control unit that transmits the voice data to the voice recognition processing server device. The voice data output control unit is configured such that the volume of the voice data sequentially input is less than a predetermined threshold value indicating silence, and the recognition processing state of voice recognition processing for the voice data received from the voice recognition processing server device is Control is performed so that the silent voice data is not transmitted to the voice recognition processing server apparatus when the voice data is not recognized indicating standby.

It is a figure which shows the structure of the speech recognition system of 1st Embodiment. It is a figure which shows the functional block of the communication terminal of 1st Embodiment. It is a figure for demonstrating the speech recognition process of 1st Embodiment. It is a figure which shows the processing flow of the speech recognition processing server apparatus of 1st Embodiment. It is a figure for demonstrating the audio | voice data output control of the communication terminal of 1st Embodiment. It is a figure which shows the processing flow of the communication terminal of 1st Embodiment. It is a figure for demonstrating the modification of the audio | voice data output control of the communication terminal of 1st Embodiment. It is a figure which shows the processing flow of the communication terminal which concerns on the modification shown in FIG.

  Hereinafter, embodiments will be described with reference to the drawings.

(First embodiment)
1 to 8 are diagrams showing a voice recognition system according to the first embodiment. FIG. 1 is an overall configuration diagram of a voice recognition system. The voice recognition system includes a communication terminal 100 on the user (user) side and a voice recognition processing server device 300 (hereinafter referred to as server device 300) that performs voice recognition processing on voices uttered by the user collected (acquired) by the communication terminal. ) And.

  The communication terminal 100 and the server device 300 are connected by a wireless communication network or a wired communication network. For example, a communication network such as the Internet network (IP network), a communication network for mobile devices such as 3G, 4G, and LTE including PHS are included. Further, it may be a PSTN (Public Switched Telephone Network).

  The communication terminal 100 is an information terminal device having a communication function. For example, there are mobile terminals having a call / communication function such as a mobile phone and a multi-function mobile phone, and mobile communication terminal devices such as a PDA (Personal Digital Assistant) having a communication function. The communication terminal 100 also includes an information processing terminal device having a communication function such as a personal computer.

  As shown in FIG. 1, the communication terminal 100 includes a CPU 110 that performs overall control, a storage unit 120, a communication unit 130 that performs communication control with the server device 300, a microphone (sound collector) 140, a speaker (audio output). Device) 150, a display unit 160 such as a liquid crystal display, and an operation unit 170 such as a touch panel and operation keys.

  FIG. 2 is a functional block diagram of the communication terminal 100. The communication terminal 100 includes an A / D conversion unit 111 connected to a microphone 140, a sound volume measurement unit 112, an audio data output control unit 113, a recognition state confirmation unit 114, and a display control unit 115.

  The A / D converter 111 converts an analog audio signal output from the microphone 140 into digital data, and generates audio data. The sound volume measurement unit 112 receives sound data from the A / D conversion unit 111 and measures the sound volume of the sound uttered by the user from the sound data. The audio data output control unit 113 receives the audio data from the A / D conversion unit 111 and receives the sound volume measurement result, and performs control to output (transmit) the generated audio data to the server device 300. The recognition state confirmation unit 114 confirms (sets) the recognition state (processing state) of the speech recognition processing of the server device 300. The display control unit 115 performs display control for displaying voice recognition result information received from the server device 300, for example, text data on the display unit 160.

  As shown in FIG. 1, the server apparatus 300 performs a voice recognition process, a CPU 310 that performs overall control, a storage unit 320, a communication unit 330 that performs communication control with the communication terminal 100, and outputs a voice recognition result. A voice recognition unit 340 is included. The speech recognition unit 340 is configured by software, and the CPU 310 can perform speech recognition processing or can be configured by hardware as a speech recognition control device (control circuit).

  The voice recognition unit 340 performs voice recognition processing on the voice data transmitted from the communication terminal 100. The voice recognition process is a process of performing acoustic analysis of input voice data, matching it with an acoustic model or a language model, and converting it into text (character) data.

  The acoustic model includes a phoneme waveform sample and text data corresponding to the waveform sample. The language model is data representing the probability of appearance of word-to-word links, in other words, the connection of words using probability. These acoustic models, language models, and other various information necessary for speech recognition processing are stored in the storage unit 320.

  The voice recognition process of the voice recognition unit 340 includes a voice data detection process (VAD: Voice Activity Detection, hereinafter) that determines voice (sound) / non-speech (silence) and detects a voice (sound) section. , Referred to as VAD processing). The voice recognition unit 340 can perform a voice recognition process in which an acoustic model or the like is applied to a voiced section extracted by the VAD process. Note that a known method can be appropriately applied to the voice recognition processing of the present embodiment.

  In the speech recognition system according to the present embodiment, resources for speech recognition processing for speech data are collected on the server device 300 side. For this reason, the communication terminal 100 basically only collects and generates voice data necessary for voice recognition and transmits the voice data to the server apparatus 300. The voice recognition processing including VAD processing is performed on the communication terminal 100 side. I will not. With this configuration, the processing load on the communication terminal 100 can be reduced.

  FIG. 3 is a diagram for explaining the voice recognition processing of the server apparatus 300 for the voice data collected by the communication terminal 100 of the present embodiment. As shown in FIG. 3, when an operation for starting speech recognition (for example, activation of a speech recognition application) is performed, the communication terminal 100 activates the microphone 140 and collects the speech uttered by the user. The process for generating audio data is started.

  The sound collected from the microphone 140 is sequentially input to the A / D conversion unit 111 of the communication terminal 100. The A / D converter 111 performs A / D conversion in real time at predetermined time intervals to generate voice packet data. The voice data output control unit 113 sequentially transmits voice packet data to the server apparatus 300 in time series.

  The communication terminal 100 continues to sequentially transmit the voice data collected through the microphone 140, starting from the timing when the operation for starting voice recognition is performed and the timing when the voice starts the sound collection process by the microphone 140, Until the condition for ending speech recognition is satisfied, control is performed so that the server apparatus 300 receives the data as audio stream data. Here, the condition for ending the voice recognition can be triggered by, for example, an operation by the user for ending the voice recognition or a voice recognition result not being received from the server device 300 for a predetermined time or more.

  When the server apparatus 300 receives the voice data, the server apparatus 300 performs a VAD process, determines a voice / silence, detects a voice section, and performs a voice recognition process on the voice section using an acoustic model or the like. The server apparatus 300 receives the voice data “Today is a good weather” as voice packet data that is continuous in time order in the time order of voices uttered by the user. Voice recognition processing is performed, and the voice is sequentially converted into text data.

  The server apparatus 300 can start voice recognition processing including VAD processing triggered by the reception of the first voice packet data regardless of whether the communication terminal 100 is voiced or silent. On the other hand, the started voice recognition process can be configured to end once when silent voice data is continuously input for a certain period of time. For example, when no voiced voice section is detected for a certain time (T), in other words, when silence is continuously detected for a certain time (T), voice data continuously input from the communication terminal 100 The voice recognition processing for is temporarily ended and a standby state is entered. Then, when voiced voice data is detected after the continuous silent section, the voice recognition process can be started again.

  FIG. 4 is a diagram illustrating a processing flow of the voice recognition processing of the server apparatus 300 according to the present embodiment. As shown in FIG. 4, when voice data is received (YES in S301), the voice recognition unit 340 starts voice recognition processing and transmits (outputs) an SOS (Start of Speech) signal to the communication terminal 100 (S302). ). The SOS signal is recognition state information indicating a recognition state of the speech recognition process, and indicates that the recognition state is “recognition process in progress”.

  The voice recognition unit 340 performs the voice recognition process described above (S303), and sequentially transmits the voice recognition process result for the voice data to the communication terminal 100. The speech recognition unit 340 determines whether or not the recognition process end condition is satisfied during execution of the speech recognition process after the output of the SOS signal (S304), and when it is determined that the recognition process end condition is satisfied (YES in S304). Then, the voice recognition process being executed is ended (shifted to standby), and an EOS (End of Speech) signal indicating the end of one cycle of the voice recognition process for the SOS signal is transmitted (output) to the communication terminal 100 (S305). . The EOS signal is recognition state information indicating the recognition state of the speech recognition process, and indicates that the recognition state is “unrecognized (standby)”. Here, the recognition process end condition in step S304 can be whether or not the duration of the silent section during the speech recognition process has exceeded a predetermined time T.

  Note that “Today is a nice weather” in FIG. 3 includes silence indicated by “...”, But the voice recognition unit 340 indicates that the silence indicated by “. Since the duration t1 is shorter than the above-described predetermined time T for determining the end of the started voice recognition process, the one-cycle voice recognition process is continuously performed without ending the voice recognition process. . In other words, in order to perform “Today's good weather” with one cycle of speech recognition processing, the silence period t1 between phrases is sampled in advance, and a predetermined time T longer than the silence period t1 between phrases is set. can do. Note that the converted text data is transmitted to the communication terminal 100 a plurality of times for each converted character or phrase, for example, during one cycle of speech recognition processing, or at the end of one cycle of speech recognition processing. Then, it can be configured to be transmitted to the communication terminal.

  As described above, in the voice recognition process of the present embodiment, there are two statuses of “recognition process in progress” and “unrecognized”, and the section between the pair of SOS signals and the EOS signal is being executed. The section from the EOS signal to the SOS signal of the speech recognition process in the next cycle indicates that the speech recognition process is on standby (see FIG. 3). If the EOS signal is not received after receiving the SOS signal, the recognition state confirmation unit 114 of the communication terminal 100 updates the status of the voice recognition processing of the server device 300 to “recognition processing in progress” and receives the EOS signal. If the SOS signal has not been received after this, the status of the speech recognition process of the server apparatus 300 is updated to “Unrecognized”. The recognition state confirmation unit 114 outputs the status update information of the voice recognition process to the voice data output control unit 113.

  The voice recognition unit 340 according to the present embodiment performs voice recognition processing on voice data continuously transmitted from the communication terminal 100, but receives voice data and starts voice recognition processing. When the silence for a predetermined time T continues, the voice recognition process being executed for the continuous silent section after the start of the voice recognition process is temporarily stopped and waits until the next voice is input, Voice recognition processing is performed again when is input. By comprising in this way, execution of useless speech recognition processing can be suppressed and the processing load of the server apparatus 300 can be reduced.

  Here, as shown in FIG. 3, the user's voice collected by the microphone 140 includes sound and silence. However, the communication terminal 100 may perform predetermined processing even if silence is included in the sound data. Voice packet data divided at time intervals is continuously transmitted to the server apparatus 300. In the example of FIG. 3, for example, it is assumed that the user issues “Today is a nice weather”. “...” indicates silence. The voice data “Today is a nice weather” is not partitioned by the “...” silence on the communication terminal 100 side, and the silence represented by “...” is also converted into voice data as voice data. Subsequently, it is transmitted to the server apparatus 300. This is because VAD processing or the like for voice data is not performed on the communication terminal 100 side in order to reduce the processing load on the communication terminal 100 by consolidating voice recognition processing resources on the server device 300 side.

  For this reason, as shown in FIG. 3, the communication terminal 100 continues to transmit silent sound data to the server device 300 even when the one-cycle speech recognition processing on the server device 300 side is completed. Communication traffic (communication data amount) with the apparatus 300 increases, which places a burden on the network. Therefore, in the present embodiment, the processing state of the voice recognition process of the server apparatus 300 is confirmed based on the SOS signal and the EOS signal, and when the voice recognition process is on standby, silent voice data is transmitted to the server apparatus 300. Control not to.

  FIG. 5 is a diagram for explaining audio data output control of the communication terminal 100 according to the present embodiment. As shown in FIG. 5, the volume measuring unit 112 measures the volume of the audio data and performs a volume check process for determining whether the sound input through the microphone 140 is silent or sound. For example, when the measured volume is equal to or higher than a predetermined threshold, it is determined as sound, and when the volume is less than the threshold, it can be determined as silence. The sound volume check result is output to the audio data output control unit 113.

  When it is determined that there is no sound in the volume check process, the voice data output control unit 113 determines that the status of the voice recognition process is “Unrecognized” on the server device 300 side based on the status update information input from the recognition state confirmation unit 114. Is determined. The voice data output control unit 113 performs control so that silent voice data is not transmitted when the voice recognition processing state is “Unrecognized”.

  In other words, the audio data output control unit 113 is based on the EOS signal received after receiving the SOS signal from the server device 300 until the audio data becomes sound, in other words, after receiving the EOS signal, Until the sound data of the volume is input, the generation of the sound data and the transmission of the sound data to the server apparatus 300 are prohibited, and the sound data output control is performed so that the sound data is not transmitted to the server apparatus 300.

  FIG. 6 is a diagram illustrating a processing flow of audio data output control of the communication terminal 100 according to the present embodiment. When an operation for starting voice recognition is performed (S101), the communication terminal 100 activates the microphone 140 and performs voice data generation processing and volume check processing (S102). In step S101, a communication process for establishing a communication session with the server apparatus 300 can be performed.

  The communication terminal 100 starts recognition state information update processing from the server device 300 in accordance with an operation for starting voice recognition (S103). The update process is performed each time an SOS signal and an EOS signal are received in parallel with other processes such as the voice data generation process until the condition for ending the voice recognition on the communication terminal 100 side is satisfied. Is called.

  The communication terminal 100 measures the volume of the generated voice data and determines whether the voice input through the microphone 140 is silent or voiced (S104). When it is determined that the measured sound volume is equal to or higher than the predetermined threshold (sound), the communication terminal 100 performs a sound data transmission process for transmitting sound data to the server device 300 (S105).

  On the other hand, when it is determined in step S104 that the volume is less than the threshold value (silence), the communication terminal 100 proceeds to step S106, and the voice recognition process on the server apparatus 300 side is based on the recognition state information. Is determined. If it is determined that “recognition processing is in progress”, the communication terminal 100 proceeds to step S105 and performs audio data transmission processing for transmitting audio data to the server device 300. When it is determined that it is not “recognition process in progress” (“unrecognition is in progress”), the communication terminal 100 skips step S105 and performs control so as not to transmit silent audio data.

  When the communication terminal 100 receives the voice recognition result for the voice data transmitted to the server device 300 (YES in S107), the communication terminal 100 performs display control to display the voice recognition result on the display unit 160 (S108). The communication terminal 100 repeatedly performs step S104 to step S108 until the condition for ending speech recognition is satisfied (NO in S109). When the condition for ending the speech recognition is satisfied, for example, when an operation for ending the activated speech recognition application is performed (YES in S109), the communication terminal 100 performs the process shown in FIG. finish.

  According to the present embodiment, the processing performance of the communication terminal 100 is not allocated to the resources necessary for voice recognition processing including VAD processing, so that the processing load on the communication terminal 100 can be reduced and unnecessary voice is transmitted to the server device 300. Therefore, communication traffic (communication data amount) with the server apparatus 300 can be reduced.

  Next, a modification of this embodiment will be described. FIG. 7 is a diagram for explaining a modified example of the audio data output control of the communication terminal 100, and FIG. 8 is a diagram illustrating a processing flow of the communication terminal 100 according to the modified example.

  As shown in FIG. 7, in this modification, silent sound data is not transmitted to the server apparatus 300 until a sound is input after an operation for starting speech recognition is performed. Control. In the audio data output control shown in FIGS. 5 and 6, the audio data is transmitted to the server device 300 at the timing when the operation for starting the speech recognition is performed or when the sound starts the sound collecting process by the microphone 140. Was.

  Therefore, for example, after receiving an SOS signal from the server apparatus 300 after an operation for starting voice recognition is performed, the voice data is transmitted to the server apparatus 300 even if there is no sound (in FIG. 6). From NO at step S104 to YES at step S106).

  Therefore, in this modified example, after an operation for starting voice recognition, that is, after the voice data acquisition process is started by the microphone 140, first, voice data having a volume equal to or higher than a predetermined threshold (sound voice data). Until the silent sound data collected by the microphone 140 is not transmitted to the server device 300, and in addition to the sound data output control shown in FIGS. Communication traffic (communication data amount) with the server apparatus 300 is reduced.

  First, at the start of the recognition state information update process in step S103 of FIG. 8, the recognition state information is initialized to “unrecognized”. After the operation for starting the speech recognition, the time until the first reception of the SOS signal is set as “Unrecognized”. With this configuration, as shown in FIG. 7, it is possible to prevent silent audio data from being transmitted to the server device 300 regardless of whether or not the SOS signal is received.

  Next, in the example of FIG. 8, unlike the steps S104 and S106 of FIG. 6, when the voice data is transmitted for the first time after the operation for starting the voice recognition, the server apparatus 300 side is based on the recognition state information. It is determined whether the speech recognition process is “recognition process in progress” (S104A). When it is determined that the communication terminal 100 is “Unrecognized”, the communication terminal 100 measures the volume of the generated voice data and determines whether the voice input through the microphone 140 is silent or voiced. (S106A). If it is determined that the measured volume is less than the predetermined threshold (silence), the communication terminal 100 skips step S105 and performs control so that silent audio data is not transmitted to the server device 300.

  Referring to the example of FIG. 7, when voice data is first transmitted after an operation for starting voice recognition, the voice recognition process status is initially set to “Unrecognized”, so voice data output control is performed. The unit 113 does not transmit the audio data to the server device 300. For this reason, the server apparatus 300 does not output the SOS signal.

  When the voice data output control unit 113 receives voiced voice data in a state where voice data has not yet been transmitted after the operation for starting voice recognition, the voice recognition processing status is “Unrecognized”. Even if “medium”, the audio data is transmitted to the server apparatus 300 (NO in S104A to YES in S106A). Receiving the voice data, the server apparatus 300 transmits an SOS signal to the communication terminal 100, and the status of the voice recognition process is updated to “recognition process in progress”.

  On the other hand, if it is determined in step S104A that the voice recognition process on the server apparatus 300 side is “recognition process in progress”, the voice data output control unit 113 transmits the voice data to the server apparatus as it is even when there is no sound. Audio data transmission processing is performed (S105). The other processes are the same as those described with reference to FIG.

  As described above, in the voice recognition system of the present embodiment, the communication terminal 100 can perform compression processing on voice data and transmit the compressed voice data to the voice recognition processing server apparatus 300. At this time, the voice recognition processing server apparatus 300 can perform voice recognition processing by decompressing the compressed voice data.

  Moreover, each function of the communication terminal 100 and the speech recognition processing server apparatus 300 can be configured as a program. For example, a program for each function stored in an auxiliary storage device (not shown) of a computer and stored in the auxiliary storage device by a control unit such as a CPU is read into the main storage device, and the program read into the main storage device is read out. The control unit can execute the function of each unit according to this embodiment.

  Further, the program can be provided to a computer in a state where the program is recorded on a computer-readable recording medium. Computer-readable recording media include optical disks such as CD-ROM, phase change optical disks such as DVD-ROM, magneto-optical disks such as MO (Magnet Optical) and MD (Mini Disk), floppy (registered trademark) disks, Examples include magnetic disks such as removable hard disks, memory cards such as compact flash (registered trademark), smart media, SD memory cards, and memory sticks. A hardware device such as an integrated circuit (IC chip or the like) specially designed and configured for the purpose of the present invention is also included as a recording medium.

  In addition, although embodiment of this invention was described, the said embodiment is shown as an example and is not intending limiting the range of invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

100 communication terminal 110 control unit (CPU)
111 A / D conversion unit 112 Volume measurement unit 113 Audio data output control unit 114 Recognition state confirmation unit 115 Display control unit 120 Storage unit 130 Communication unit 140 Microphone 150 Speaker 160 Display unit 170 Operation unit 300 Voice recognition processing server apparatus 310 Control unit (CPU)
320 storage unit 330 communication unit 340 voice recognition unit

Claims (5)

A communication terminal that transmits voice data issued by a user to a voice recognition processing server device that performs voice recognition processing, and receives a voice recognition processing result for the voice data from the voice recognition processing server device,
A volume measuring unit for measuring the volume of audio data acquired by the audio input unit;
A voice data output control unit that transmits the voice data to the voice recognition processing server device,
The voice data output control unit is configured such that the volume of the voice data sequentially input is less than a predetermined threshold value indicating silence, and the recognition processing state of voice recognition processing for the voice data received from the voice recognition processing server device is A communication terminal, which is controlled so as not to transmit the silent voice data to the voice recognition processing server device when unrecognized indicating standby. The signal indicating the recognition processing state is paired with the SOS signal and the SOS signal indicating the start of the voice recognition processing in response to the reception of the voice data, and the silent voice data is detected for a predetermined time in the started voice recognition processing. An EOS signal indicating that the speech recognition process is to be terminated when it continues,
The voice data output control unit controls not to transmit the voice data to the voice recognition processing server apparatus until the voice data having a volume equal to or higher than the predetermined threshold is input after receiving the EOS signal. The communication terminal according to claim 1.   The audio data output control unit outputs the audio data indicating silence during a period from when the audio data acquisition process is started by the audio input unit to when the audio data having a volume equal to or higher than the predetermined threshold is input. The communication terminal according to claim 1, wherein control is performed so as not to transmit to the voice recognition processing server device. A program executed by a communication terminal that transmits voice data issued by a user to a voice recognition processing server device that performs voice recognition processing, and receives a voice recognition processing result for the voice data from the voice recognition processing server device,
A first function for measuring the volume of audio data acquired by the audio input unit;
A second function of transmitting the voice data to the voice recognition processing server device,
In the second function, the volume of the voice data that is sequentially input is less than a predetermined threshold value indicating silence, and the recognition processing state of the voice recognition process for the voice data received from the voice recognition processing server apparatus is on standby. When the program is unrecognized, the program is controlled so as not to transmit the silent voice data to the voice recognition processing server apparatus. A speech recognition processing server device that performs speech recognition processing; and a communication terminal that transmits speech data issued by a user to the speech recognition processing server device and receives a speech recognition processing result for the speech data from the speech recognition processing server device. A speech recognition system comprising:
The voice recognition processing server device transmits a signal indicating a recognition processing state of voice recognition processing to the received voice data to the communication terminal,
The communication terminal is
A volume measuring unit for measuring the volume of audio data acquired by the audio input unit;
A voice data output control unit that transmits the voice data to the voice recognition processing server device,
The sound data output control unit, when the volume of the sound data sequentially input is less than a predetermined threshold value indicating silence, and the recognition processing state is unrecognized indicating standby for sound recognition processing, A voice recognition system that controls not to transmit voice data to the voice recognition processing server device.