JP2015184487A - Voice processor and voice processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To acquire a text of utterance content without a user uttering from the start again when voice recognition is performed during the user's utterance and voice recognition is failed on the way.SOLUTION: As an embodiment of the current invention, a voice processor includes an acquisition part, a transmission part, a storage part, a reception part, and a control unit. The acquisition part successively acquires voice data representing a content a user has uttered. The transmission part transmits to a voice recognition system a voice recognition request of the voice data acquired by the acquisition part. The storage part stores the voice data acquired by the acquisition part. The reception part receives from the voice recognition system a voice recognition response including a text converting the voice data by voice recognition or information showing failure of voice recognition of the voice data. The control unit performs control so as to identify the voice data whose voice recognition is failed on the basis of the voice recognition response and to transmit to the voice recognition system the voice recognition request of data including the voice data whose voice recognition is failed on the basis of the voice data stored to the storage part.

Description

  Embodiments described herein relate generally to a voice processing apparatus and a voice processing method.

  In the field of home medical care and nursing care, several staff members of medical and nursing care staff are involved in the care of patients and care recipients and the care of daily life. A plurality of staff members observe and diagnose the condition of the patient and the cared person. A single staff member does not observe the patient continuously or continuously, but multiple staff members from various occupations visit and observe patients at different times and intervals. For this reason, in order for each staff member to share information about a patient, the observation result of the patient is registered in an electronic medical record system, a nursing / nursing care recording system, or an SNS.

  As a system for sharing observation results for patients, an information sharing system using a voice message (hereinafter referred to as a voice tweet system) is known. In the voice tweet system, each staff member utters a patient's observation result to a microphone of a portable terminal such as a smartphone, and records the result with a voice tweet registration application installed in the portable terminal, thereby generating a voice message. Each staff member sends the generated voice message to the server and registers it to be shared among the staff members. At this time, the voice message is converted into a text by voice recognition, and the tags such as the text, the patient ID to be uttered, the staff ID of the utterer, the utterance time, the utterance place, the keyword extracted from the voice message, etc. Attached to the voice message. Such a voice message to which text or a tag is attached is called a voice tweet. Each staff member can browse or view the voice tweets stored on the server from a portable terminal or a personal computer.

  In the case of a voice tweet system, the staff member may want to confirm in advance whether the content of the text matches the spoken voice before sending the voice-recognized text to the server. In general, when the uttered content is converted into text and stored in the user terminal, it may be desired to confirm whether the uttered content is correctly recognized. At this time, it is desired to reduce the power consumption and cost of the user terminal while making it possible to confirm the speech-recognized text as quickly as possible.

  Here, voice recognition for converting a voice message into text can be performed inside the terminal or using an external voice recognition system. In addition, there are batch speech recognition for recognizing sound files in which utterances are recorded, and real-time speech recognition for performing speech recognition during utterances. It is conceivable to use real-time speech recognition using an external speech recognition system from the viewpoint of reducing power consumption and cost of the user terminal while confirming speech recognition as described above as quickly as possible.

  However, in the case of real-time speech recognition, speech communication may fail if communication is interrupted in the middle of an utterance or if the speech recognition system recognizes many users' voices and resources are tight. Is expensive. When voice recognition fails, the user has to start speaking again from the beginning, and the load on the user is great.

Patent registration No. 5414865

  Embodiments of the present invention make it possible to acquire the text of the uttered content without re-speaking the user even if the speech recognition fails during the speech recognition when performing the speech recognition during the user's utterance. With the goal.

  As an embodiment of the present invention, a speech processing apparatus includes an acquisition unit, a transmission unit, a storage unit, a reception unit, and a control unit.

  The acquisition unit sequentially acquires audio data representing the content uttered by the user.

  The transmission unit transmits a voice recognition request for the voice data acquired by the acquisition unit to a voice recognition system.

  The said memory | storage part memorize | stores the audio | voice data acquired by the said acquisition part.

  The receiving unit receives, from the voice recognition system, a voice recognition response including text converted from the voice data by voice recognition or information indicating a voice recognition failure of the voice data.

  The control unit identifies voice data that has failed in the voice recognition based on the voice recognition response, and requests voice recognition of data including the voice data that has failed in voice recognition based on the voice data stored in the storage unit Is transmitted to the voice recognition system.

The functional block diagram of the speech processing unit which concerns on embodiment of this invention. The flowchart which shows operation | movement of the audio processing apparatus of FIG. The flowchart following FIG. The figure which shows the example of a recognition result table. The flowchart of control of re-speech recognition. The flowchart of the operation | movement which concerns on 3rd Embodiment. The whole block diagram of the system concerning a 5th embodiment. The whole system block diagram concerning a 6th embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a functional block diagram of a speech processing apparatus according to an embodiment of the present invention. A voice processing apparatus 101 is connected to a voice recognition system 201 via a network 301.

  The voice processing device 101 includes a recording unit (acquisition unit) 11, a file storage unit 12, a transmission unit 13, a reception unit 14, a recognition result storage unit 15, a control unit 16, a display unit 17, and an input unit 18.

  The voice processing apparatus 101 can be mounted on a user terminal such as a smartphone, a mobile terminal, a tablet, or a PC. The functions of the processing units included in the voice processing device 101 can be realized using a CPU, a memory, an auxiliary storage device, a communication device, and an input / output interface that are generally provided in a user terminal. It is assumed that the user terminal on which the voice processing apparatus 101 is mounted is provided with a microphone that collects voices spoken by the user and converts them into electric signals, or can be attached by external connection. A microphone can also be incorporated into the audio processing apparatus 101.

  The operation of each processing unit in FIG. 1 can be realized as an operation achieved by an operating system (OS) running on a CPU and an application running on the OS, for example.

  The input unit 18 is an input interface through which a user inputs various instructions. For example, there are a touch panel, an input button, a mouse, a keyboard, and the like. Information input through the input unit 18 is sent to the control unit 16.

  The display unit 17 is an output interface that displays an image based on an image signal input from the outside. Examples of the display unit 17 include a liquid crystal panel, an organic EL panel, and an electronic ink panel.

  As described above, the user terminal on which the voice processing device is mounted has a microphone or can be externally connected. This microphone collects voices spoken by the user and converts the voices into analog electrical signals. The electrical signal is further converted into audio digital data of a predetermined format and input to the recording unit 11. The conversion into audio digital data is performed, for example, by the CPU of the user terminal, the inside of the microphone, or another processing circuit.

  The recording unit 11 includes an acquisition unit (not shown) that acquires data as audio data for each predetermined size from the beginning of the input audio digital data. The recording unit 11 sequentially stores the audio data acquired by the acquisition unit in a memory (not shown). When storing in the memory, the audio data is arranged to follow the audio data acquired immediately before. Thereby, on the memory, the respective audio data acquired from the start to the end of the recording are combined in time series. The recording unit 11 creates an audio file in a predetermined format by adding a file header or the like to the combined audio data. The file format may be arbitrary. As an example, there are a WAVE (RIFF waveform Audio Format) format and a PCM (Pulse Code Modulation) format. An irreversible compression format such as mp3 format may be used. Note that the audio file may be reversibly compressed in a predetermined format, and the compressed audio file may be stored instead. The recording unit 11 stores the audio file created on the memory in the file storage unit 12. If the memory and the file storage unit 12 are the same medium, this storing operation can be omitted.

  In addition, the recording unit 11 assigns an identifier to the audio data extracted for each fixed size as described above. You may give the acquisition time of audio | voice data as an identifier. The time may be acquired from a system clock (not shown). Instead of the acquisition time, another type of identifier such as a sequentially increasing sequence number may be given. In addition, information (header or the like) corresponding to the protocol of the voice recognition service provided by the voice recognition system 201 may be added to the voice data. The identifier may be included in a predetermined field in the header. The recording unit 11 sends the audio data to which the identifier is assigned to the transmission unit 13. If the user is speaking, voice data is sent to the transmission unit 13 in parallel with the user's speaking.

  The file storage unit 12 stores therein the audio file generated by the recording unit 11. The file storage unit 12 is managed by, for example, a file system installed in the OS. The file storage unit 12 can be composed of an arbitrary storage medium such as a hard disk, SSD, or memory.

  The transmission unit 13 communicates with the voice recognition system 201 based on a predetermined communication protocol. The communication protocol to be used may be arbitrary, but, for example, TCP (or UDP) / IP based protocol processing is performed. As TCP / IP-based protocol processing, it is possible to use http higher than TCP / IP. The transmission unit 13 also processes a protocol for communication with the network 301. For example, a protocol according to a wireless LAN standard, a protocol compatible with a cellular system such as 3G, or an Ethernet protocol may be used.

  The transmission unit 13 generates a speech recognition request for requesting speech recognition of the speech data from the speech data of a certain size input from the recording unit 11 and its identifier. The voice recognition request includes voice data and its identifier. The transmission unit 13 packetizes the generated voice recognition request data and transmits it to the voice recognition system 201 via the network 301. If the user is speaking, a voice recognition request is transmitted to the voice recognition system 201 while speaking.

  The speech recognition system 201 receives a speech recognition request from the speech processing apparatus 101, and extracts speech data and an identifier from the speech recognition request. The voice recognition system 201 converts the voice data into text by voice recognition. This text is an arbitrary character string. If the speech recognition system 201 succeeds in speech recognition, the speech recognition system 201 generates a speech recognition result including the generated text and the extracted identifier. The speech recognition result may include success information indicating that the speech recognition is successful.

  On the other hand, when the speech recognition system 201 fails, the speech recognition system 201 generates a speech recognition result including failure information indicating that the speech recognition has failed and an identifier. When speech recognition fails, there are cases where the speech recognition system 201 is processing a large amount of speech data and there are not enough resources or speech recognition itself cannot be performed due to a system trouble. Or, there may be a case where the audio data itself is abnormal (a value that cannot be processed).

  The speech recognition system 201 generates a speech recognition result and transmits a speech recognition response including the speech recognition result to the speech processing apparatus 101 in both cases where the speech recognition succeeds and fails.

  The receiving unit 14 communicates with the voice recognition system 201 based on a predetermined communication protocol. The receiving unit 14 performs processing of the network 301 and a wireless or wired communication protocol. These communication protocols are the same as those of the transmission unit 13. The receiving unit 14 receives a voice recognition response from the voice recognition system 201 via the network 301. The receiving unit 14 extracts a speech recognition result from the speech recognition response. If the speech recognition is successful, the speech recognition result includes text converted from speech data and an identifier. Furthermore, success information indicating that the speech recognition is successful may be included. On the other hand, when voice recognition of voice data fails, the voice recognition result includes failure information and an identifier indicating that voice recognition has failed.

  The control unit 16 controls operations including the start and end of the recording unit 11. For example, the recording unit 11 is activated by a user instruction from the input unit 18 to start voice recording. Further, the voice recording is ended by stopping the recording unit 11 in accordance with a user instruction from the input unit 18.

  The control unit 16 manages the transmitted audio data by storing the identifier added to each audio data as a list in the recognition result storage unit 15 or another memory when transmitting each audio data. Further, the received voice recognition results are managed by storing each voice recognition result acquired by the receiving unit 14 as a list in the recognition result storage unit 15 or another memory. The control unit 16 compares the identifier added to the transmitted voice data with the identifier in the received voice recognition result, and which voice data of the transmitted voice data is received. Can be grasped.

  Based on each voice recognition result, the control unit 16 determines whether or not the voice recognition for the current user's speech has succeeded. Specifically, if speech recognition of at least H (H is an integer of 1 or more) speech data among speech data transmitted to the speech recognition system 201 fails, it is determined that speech recognition for the current utterance has failed. To do. On the other hand, when the number of unsuccessful audio data is less than H, it is determined as successful. For example, in the case of H = 1, if the speech recognition is successful for all transmitted speech data, it is determined that the speech recognition of the current utterance is successful, and if the speech recognition fails even for one speech data, this time It is determined that the voice recognition of the utterance has failed. In the following description, the case of H = 1 is assumed, but the present embodiment is not limited to this.

  When the control unit 16 determines success for the current utterance, the control unit 16 generates the utterance text by combining the text converted from each voice data in time series, and displays the utterance text on the screen.

  For example, consider a case where a user utters “I visited Mr. Yamada. The complexion is good, but I prescribe antipyretic drugs just in case. It is assumed that a plurality (four in this case) of voice data are sequentially acquired based on this utterance, a voice recognition request including each is transmitted to the voice recognition system 201, and four voice recognition results are returned. The first speech recognition result includes the text “I visited Mr. Yamada.” The second speech recognition result includes the text “Face Color is Good.” The third speech recognition It is assumed that the result includes the text “But prescribe antipyretic drugs just in case” and the fourth speech recognition result includes the text “It is over”. In this case, since the speech recognition of all the speech data was successful, the success decision was made, and the speech text “Yamada-san that combined these texts was visited. The complexion is good. That's it! "Is displayed on the screen.

  On the other hand, when the failure is determined, a voice recognition failure message is displayed on the screen. At that time, a message for notifying the user that it is not necessary to speak again the content uttered this time may be displayed on the application screen. Alternatively, a message for notifying the user that the voice recognition system 201 is to be re-requested (voluntarily) for voice recognition may be displayed on the screen. This is because the voice data from the start to the end of the utterance is stored in the voice file, and the voice recognition system 201 can be requested for voice recognition later using this voice file. As an example of a message displayed on the screen in the case of voice recognition failure, the message “Real-time voice recognition has failed, but voice recognition is attempted again using a voice file that has been recorded from the start to the end of speech” is displayed. Also good. As a result, the user can grasp that it is not necessary to utter again the contents once uttered. The message for notifying the user that it is not necessary to speak again the content uttered this time, such as immediately after the acquisition of the voice data, immediately after the generation of the voice file, or immediately after the transmission of the voice recognition request is completed It may be displayed at an arbitrary time after the acquisition of audio data is completed.

  The recognition result storage unit 15 stores correspondence information for managing the correspondence between the user's speech, the speech recognition result for the speech, and the speech file. The correspondence information can be held in an arbitrary format such as a table format. Here, the correspondence information has a table format, and this table is called a recognition result table. The recognition result storage unit 15 can be composed of an arbitrary storage medium such as a hard disk, SSD, or memory. The same device as the file storage unit 12 or a different device may be used.

  FIG. 4 shows an example of the recognition result table. The recognition result table can be held as a text file or a database, for example. The recognition result table has columns of “utterance date / time”, “recognition result”, and “voice file”. One entry is added by the control unit 16 for each user utterance (one unit of recording).

  The “speech date / time” column holds information for specifying the user's utterance date / time. For example, the date and time when the “registration button” on the application screen is selected (touch, click, etc.) or the date and time when the utterance is started is stored. The date and time when the utterance is started is the date and time at the beginning of the voice data acquired first. The user's utterance is identified by the utterance date and time. When utterance identification information (utterance ID) is issued for each utterance, a column for holding the utterance ID may be provided.

  The “recognition result” column holds speech text obtained by combining text converted from each speech data with respect to the user speech, or information indicating that speech recognition is not completed. As an example of information indicating that the speech recognition is incomplete, there is information indicating that the speech recognition has failed. In FIG. 4, the information <failure> corresponds to this.

  The “voice file” column holds a path to the voice file stored in the file storage unit 12. The path is information for specifying the storage location of the audio file. This information can be acquired from a file system that manages the file storage unit 12.

  As described above, the control unit 16 adds an entry to the recognition result table after determining the success or failure of the speech recognition for the current utterance. Specifically, regardless of success or failure, information for specifying the utterance date and time is stored in the “utterance date and time” column, and the “voice file” column stores information for the voice file related to the user's utterance this time. Stores the path. In the “recognition result” column, the speech text is stored in the case of success, and information (in this case, <failure>) indicating incomplete or failed speech recognition is stored in the case of failure.

  Moreover, the control part 16 checks the speech which failed in speech recognition based on this recognition result table at fixed time intervals. For example, an entry whose “recognition result” column is “<failure>” is specified. The control unit 16 reads out an audio file corresponding to the specified utterance from the file storage unit 12 in accordance with the file path held in the “audio file” column. When the audio file is compressed and stored in a data format not supported by the speech recognition system 201, the read audio file is decoded into a data format supported by the speech recognition system 201.

  The control unit 16 extracts the entire voice data included in the voice file and transmits a voice recognition request for the whole voice data to the voice recognition system 201. Instead of the entire audio data, the entire audio data may be divided for each fixed size and transmitted sequentially from the beginning of the file. Also in this case, each voice data can be associated with a voice recognition result acquired by the receiving unit 14 by giving an identifier to the voice data and transmitting it. If the voice recognition system 201 supports the voice file itself, the voice file itself can be transmitted without extracting voice data from the voice file.

  The receiving unit 14 receives a voice recognition response including a voice recognition result from the voice recognition system 201. If the entire speech data is sent in a batch and the speech recognition is successful, the text (utterance text) in the speech recognition result is added to the “Recognition Result” column of the corresponding entry (overwrite <failure>). The When the voice data is divided and transmitted, the utterance text combined with the text corresponding to each voice data is stored in the “recognition result” column only when the voice recognition of all the voice data is successful. If speech recognition of speech data fails, information indicating failure (<failure> etc.) is stored.

  When the re-speech recognition is successful, the control unit 16 may notify the user by outputting a message to that effect to the display unit 17. Further, the speech text that has been voice-recognized may be displayed on the display unit 17 on the same screen as the message or on another screen. The utterance text may be displayed on the display unit 17 when instructed by the user. Note that a notification sound may be emitted from the speaker or the vibrator may be vibrated when a message is output.

  On the other hand, when speech recognition again fails, a message to that effect may be output to the display unit 17. At this time, a notification sound may be sounded or the vibrator may be vibrated. The pattern and volume of the notification sound and the vibration pattern and strength of the vibration may be changed depending on the success or failure.

  The control unit 16 can perform various controls in addition to the above-described processing. For example, a screen for prompting the user to make various inputs is displayed on the display unit 17 and an operation corresponding to the input instruction is performed.

  Further, the control unit 16 may display a list of utterance dates and the like on the display unit 17 based on the recognition result table, and may display utterance texts corresponding to the utterance dates selected by the user. Further, the text may be corrected upon receiving an instruction to edit the displayed utterance text from the user. Thereby, when there is a part different from the user's utterance content in the utterance text, the user can correct the expression. Moreover, the control part 16 may receive the reproduction | regeneration instruction | indication of an audio | voice file from a user, and may reproduce | regenerate the audio | voice file instruct | indicated from the user using a speaker.

  Further, the control unit 16 may transmit the utterance text of each utterance stored in the recognition result table or the voice file indicated by the file path to a separately provided server. This server collects utterance texts or voice files from a plurality of user terminals (voice processing devices), classifies them into various attributes such as for each user and each utterance date, and manages the utterance texts or voice files. In response to an inquiry from each user terminal, the server may transmit an utterance text or an audio file having the requested attribute to the user terminal.

  2 and 3 show flowcharts of the operation of the speech processing apparatus 101. FIG.

(Step S <b> 101) The control unit 16 displays an application screen on the display unit 17. When the user touches the “registration button” on the application screen via the input unit 18, the control unit 16 detects this and displays a message (“Please speak”) for prompting utterance on the screen. At the same time, the control unit 16 activates the recording unit 11. The user can enter voice.

  At this time, the control unit 16 may display on the application screen that voice input is being performed. Examples of display contents include a bar indicating a user's speech volume and a waveform.

  The user starts speaking toward the microphone mounted on the user terminal. From the beginning of the utterance, digital data obtained by digitizing the audio signal in a predetermined format is input to the recording unit 11. Each time the recording unit 11 reaches a certain size from the beginning of the digital data, the recording unit 11 cuts out the data of a certain size as audio data.

(Step S <b> 102) Each time the sound recording unit 11 acquires sound data of a certain size, the sound recording unit 11 adds the sound data to the memory and combines the sound data in order of the utterance time. When all audio data are combined, information such as a file header is set to generate an audio file.

(Step S103) The recording unit 11 assigns an identifier every time audio data having a certain size is acquired. The transmission unit 13 transmits a voice recognition request including voice data and an identifier to the voice recognition system 201. At this time, the control unit 16 stores the identifier added to the audio data in the memory and manages the list.

  The receiving unit 14 receives a voice recognition response including a voice recognition result from the voice recognition system 201 via the network 301. The control unit 16 stores the speech recognition result in the recognition result storage unit 15 and manages the list. If the speech recognition is successful, the speech recognition result includes text and an identifier obtained by converting the speech data. If the speech recognition result is unsuccessful, the speech recognition result includes failure information and an identifier indicating that the speech recognition has failed. When the speech recognition is successful, information indicating success may be further included in the speech recognition result.

(Step S <b> 104) The control unit 16 periodically checks the recognition result storage unit 15 to check whether a speech recognition result corresponding to the speech recognition request transmitted from the transmission unit 13 is acquired. The control unit 16 confirms whether a speech recognition result having the same identifier added to the speech data exists in the recognition result storage unit 15.

(Step S105) When the speech recognition result exists in the recognition result storage unit 15, the control unit 16 examines the content of the speech recognition result.

(Step S106) When failure information is included in the speech recognition result, it is determined that speech recognition of the current utterance has failed. That is, when at least one of the voice data transmitted to the voice recognition system 201 fails in voice recognition, it is determined that voice recognition for the current user's speech has failed. At this time, in order to store the determination of failure, the control unit 16 stores a predetermined flag in advance in a predetermined area of the recognition result storage unit 15 or an area on the memory, and the flag indicates a value indicating the failure. Is set (step S107). Assume that a value indicating success is set as the initial value of the flag.

(Step S108) After recording the failure in step S107, it is determined whether or not the user's speech has ended. When the text (or information indicating success) is included in the speech recognition result in step S105, or when it is determined that there is no speech recognition result in step S105, it is determined whether the user's speech has ended. When the user ends the utterance, the user presses the “utterance end button” on the application screen, and the control unit 16 determines the end of the utterance by detecting the pressing of this button. When the control unit 16 determines the end of the utterance, the control unit 16 checks the processing state of the recording unit 11. If the processing of the recording unit 11 is completed, the recording unit 11 is stopped. If the recording unit 11 is still operating, it waits until the processing is completed. Thereby, recording ends.

  Note that, in addition to the user explicitly ending the recording, the control unit 16 can automatically end the recording by detecting a blank space in the utterance. For example, there may be a case where a section where the volume is equal to or lower than a threshold value continues for a certain time or longer. If the user's utterance has not been completed yet, the process returns to step S101 to perform voice data acquisition processing.

(Step S109) When it is determined that the user's utterance has ended, the control unit 16 checks whether untransmitted voice data remains. For example, when an unsent voice confirmation request remains in the transmission buffer of the transmission unit 13 (including a case where a delivery confirmation response (ACK) is not returned from the voice recognition system 201), unsent voice data remains. Judge that Alternatively, even when the user's utterance has ended, but the operation of the recording unit 11 continues, it is determined that untransmitted audio data remains. If untransmitted audio data remains, the process returns to step S103 to continue transmission processing.

(Step S110) If unsent audio data does not remain, that is, if all audio data has been transmitted, it is determined whether or not the audio recognition results corresponding to all audio data have been received. If there is a voice recognition result that has not yet been received, the process returns to step S104.

(Step S111) If the voice recognition results corresponding to all the voice data have been received, the predetermined flag described above is checked. At this point, the user's utterance has been completed, all voice data has been transmitted, and voice recognition results for all voice data have been received.

(Step S112) When a value indicating failure is set in the predetermined flag, this means that voice recognition of at least one voice data among all voice data from the start to the end of the utterance by the user has failed. Means. In this case, a voice recognition failure message is displayed on the screen. At that time, as described above, a message notifying the user that the same content as the current utterance does not need to be uttered again may be displayed on the application screen.

(Step S113) On the other hand, when a value indicating failure is not set in the predetermined flag, this means that the voice recognition has succeeded for all the voice data from the start to the end of the utterance by the user. In this case, an utterance text in which the text included in the speech recognition result is combined in time series is generated, and the generated utterance text is displayed on the screen.

(Step S114) The control unit 16 adds an entry related to the current utterance to the recognition result table in the recognition result storage unit 15. That is, the control unit 16 stores the current utterance date and time in the “utterance date” column and the path to the audio file related to the current utterance in the “voice file” column. In the “recognition result” column, when the speech recognition of the current utterance is successful, the utterance text is stored, and when it is unsuccessful, information indicating failure (here, <failure>) is stored.

  Here, depending on the situation of the network 301, it may take time to transmit a voice recognition request, or it may take time to receive all voice recognition responses from the voice recognition system 201. Therefore, if transmission of all voice recognition requests is not completed within a certain time from the date and time of utterance, or if all voice recognition responses are not received, it is determined that the current voice recognition has failed, as shown in FIG. 2 and FIG. In the flow, the same processing as when it is determined as failure may be performed. Here, the date and time of utterance is set as the starting point for a certain time, but any date and time may be set as the starting point.

  FIG. 5 is a flowchart of control of re-speech recognition (batch speech recognition) for an utterance in which speech recognition has failed.

(S201) The control unit 16 checks utterances that have failed in voice recognition based on the recognition result table held in the recognition result storage unit 15 at regular time intervals.

(S202) The control unit 16 reads an audio file corresponding to the specified utterance from the file storage unit 12 according to the file path held in the “audio file” column.

(S203) The control unit 16 extracts voice data included in the read voice file and transmits a voice recognition request for the voice data to the voice recognition system 201. As described above, as a method of transmitting audio data, the entire audio data may be transmitted in a lump, or the audio data may be divided for each predetermined size and transmitted sequentially from the beginning of the file. .

(S204) The receiving unit 14 receives a voice recognition response including a voice recognition result from the voice recognition system 201. When the entire speech data is transmitted at once and the speech recognition is successful, the text in the speech recognition result is written as the utterance text in the “recognition result” column of the entry. When the voice data is divided and transmitted, the speech corresponding to each voice data is combined only when the voice recognition of all voice data is successful, as in the processes shown in FIGS. And write in the “recognition result” column. When speech recognition of even one voice data fails, information indicating failure (<failure> etc.) is stored.

(S205) When the re-speech recognition is successful, the control unit 16 may notify the user by outputting a message to that effect to the display unit 17. The speech text that has been voice-recognized may be displayed on the display unit 17 on the same screen as the message or on a different screen. Alternatively, the utterance text may be displayed when instructed by the user. If the voice recognition fails again, a message to that effect may be output to the display unit 17.

  Note that information indicating that speech recognition is being performed is displayed in the “recognition result” column of the recognition result table after the voice file is extracted in step S202 until transmission of the voice recognition request is started and the voice recognition result is returned. It may be stored. As information indicating that speech recognition is in progress, for example, a character “under speech recognition” may be stored. At the time of the check in step S201, the control unit 16 does not extract an audio file for an entry that includes information indicating that voice recognition is being performed in the “recognition result” column. Thereby, it is possible to prevent the same voice file from being repeatedly requested for voice recognition.

  The processing of this flow is performed while the processing of the flow of FIG. 2 and FIG. 3 is performed, that is, while the processing for speech recognition based on the speech is performed while the user is speaking. Is not necessary. Thereby, the load of the speech processing apparatus 101 and the speech recognition system 201 can be reduced, and the possibility of success of real-time speech recognition can be increased.

  The above-described recognition result table can be variously modified. For example, when a plurality of users use one terminal and switch users by login ID and password to the application, a column for holding the user ID may be provided in the recognition result table.

  As another example, the “recognition result” column may be deleted from the recognition result table to create a table including only the “utterance date” and “voice file” columns. An entry is added to this table only when speech recognition for an utterance fails. If successful, the utterance text is presented to the user on the screen, and after a predetermined procedure, it is erased from the speech processing apparatus. As a predetermined procedure, for example, when the user touches a “confirmation button” or the like, the user confirms that the text has been confirmed, or the utterance text is transmitted to another device (for example, the server described above). .

  In this embodiment, the voice processing apparatus 101 communicates with the voice recognition system 201 via the network 301, but the voice recognition system may be incorporated in the voice processing apparatus 101. In this case, the speech recognition system may be connected to another bus via the same bus as the CPU or a chip set. Alternatively, the function of the voice recognition system may be realized as program execution by the CPU.

  As described above, according to the present embodiment, during the user's speech, the speech recognition system is requested to perform speech recognition in parallel with the speech, and the speech content is stored as a speech file. If the voice recognition fails, the voice recognition is requested again based on the voice file. Therefore, voice recognition can be requested without letting the user speak again, and the load on the user can be reduced. In addition, when voice recognition fails, the user can grasp that a second utterance is unnecessary by notifying the user that a second utterance is unnecessary. Therefore, even if the user cannot confirm the success of the voice recognition on the spot, the user can perform the subsequent work with peace of mind.

(Second Embodiment)
In the present embodiment, a message to be notified to the user is controlled according to the user's utterance status, the arrival status of the speech recognition result for the speech data transmitted to the speech recognition system 201, and the content of the speech recognition result.

  Here, a case is shown in which the display of a message notifying that voice recognition is in progress, a message notifying that voice recognition has been successful, and a message notifying that voice recognition has failed have been controlled.

A message notifying that voice recognition is in progress is displayed when no voice recognition result for at least one voice data has been received and no voice recognition result including information indicating failure has been received. To do. Specifically, it is displayed when the following two conditions are satisfied.
(Condition 1) No voice recognition result is returned for any transmitted voice data (Condition 2) No failure is returned for any transmitted voice data As a specific example of a message, for example, “During voice recognition” May be displayed.

The message notifying that the voice recognition is successful is displayed when the voice recognition of all the voice data transmitted to the voice recognition system 201 is successful. Specifically, it is displayed when all the following conditions 3 to 5 are satisfied.
(Condition 3) Speech has ended (Condition 4) There is no untransmitted voice data (Condition 5) Voice recognition succeeded for all transmitted voice data As a specific example of a message, for example, “successful voice recognition” May be displayed.

The message notifying that the voice recognition has failed is displayed when at least one of the voice data transmitted to the voice recognition system 201 has failed in the voice recognition. Specifically, it is displayed when the above conditions 3 and 4 and the following condition 6 are all satisfied.
(Condition 6) As a specific example of a message in which voice recognition has failed for any of the transmitted voice data, for example, a character “speech recognition failure” may be displayed.

  Two or more of the above three messages are not displayed simultaneously. That is, since the conditions 1 and 5 are not satisfied at the same time, “speech recognition in progress” and “successful speech recognition” are not displayed at the same time. Since condition 2 and condition 6 are not satisfied at the same time, “voice recognition in progress” and “voice recognition failure” are not displayed at the same time. Since conditions 5 and 6 are not satisfied at the same time, “speech recognition success” and “speech recognition failure” are not displayed simultaneously.

  As described above, only one of “during speech recognition”, “successful speech recognition”, and “speech recognition failure” is displayed. As a result, it is possible to let the user speak to the end without displaying “speech recognition failure” while speaking. Note that when “speech recognition failure” is displayed when the condition 6 is satisfied during the utterance, a message may be displayed prompting the utterance to continue to the end although the speech recognition has failed in the middle.

(Third embodiment)
In the first embodiment, when the speech recognition of the utterance is successful, the entire text of the utterance is displayed, and when the speech recognition of a part of the speech data fails, a message indicating the failure of the speech recognition is displayed.

  In the present embodiment, the text is displayed for voice data that has been successfully recognized, and the text that indicates that voice recognition has not been completed is displayed for the voice data that has failed. As an example of text indicating that speech recognition has not been completed, there is text (for example, <failure>) indicating that speech recognition has failed. These texts are displayed side by side in utterance time order. As a result, the user can quickly confirm even a part of the text of the content he / she uttered.

  For example, suppose that the user utters "I visited Mr. Yamada. The complexion is good. But just in case I prescribe antipyretic drugs. That's it." Based on this utterance, it is assumed that four voice data are sequentially acquired, a voice recognition request including each is transmitted to the voice recognition system 201, and four voice recognition results are returned. The first speech recognition result includes the text “I visited Mr. Yamada”, the second speech recognition result includes information indicating failure, and the third speech recognition result includes It is assumed that the text “But prescribe antipyretic drugs just in case” is included, and the fourth speech recognition result includes information indicating failure.

  At this time, the speech data that has been successfully recognized by the speech is displayed as a text, and the speech data that has failed the speech recognition is displayed by combining the text indicating the failure (for example, <failure>). As a result, the text that appears is like “I visited Mr. Yamada. <Failure> But prescribe antipyretic drugs just in case. <Failure>”. Such combined text is stored in the “recognition result” column of the recognition result table.

  Here, in order to clarify the correspondence between each speech recognition result and the entire displayed text, the text corresponding to each speech recognition result may be combined with an appropriate symbol (for example, #). At this time, the displayed text is like "I visited Mr. Yamada. # <Failure> # but prescribe antipyretic drugs just in case. # <Failure> #". By viewing the displayed text, the user can recognize that four pieces of voice data have been acquired and voice recognition of the second and fourth voice data has failed.

  Also in the present embodiment, as in the first embodiment, when the voice recognition of a part of the voice data fails, it is determined that the voice recognition for the utterance has failed. In order to record this determination, one column may be added to the recognition result table in addition to the “recognition result” column, and information indicating failure (for example, <failure>) may be stored. In the case of success, the value of this column may be null, or information indicating success (for example, <success>) may be stored. Whether or not there is a failure can be determined by whether or not “<failure>” exists in this column. Of course, without adding such a column, the presence or absence of failure may be detected by searching for a <failure> character in the text of the “recognition result” column.

  When requesting re-speech recognition, as in the first embodiment, speech data of an utterance is acquired from a speech file corresponding to an utterance for which speech recognition has failed, and the speech recognition system 201 is requested to perform speech recognition. The entire acquired audio data may be requested in a lump, or may be requested divided by a certain size.

  Alternatively, re-speech recognition may be requested only for speech data that has failed speech recognition. Specifically, the audio data acquired from the audio file is divided into fixed sizes, and the number of these is specified based on the above-mentioned symbol (for example, #).

  In the example above, “I visited Mr. Yamada. # <Failure> # But prescribe antipyretic drugs just in case. # <Failure> #”, the second and fourth voice data fail to recognize the voice. It can be judged that. Therefore, only the second and fourth voice data are requested to be re-spoken.

  If the re-speech recognition of the second sound data succeeds and the re-speech recognition of the fourth sound data fails, the displayed text (the text stored in the “recognition result” column) is “Yamada-san. #Visit color is good #But prescribe antipyretic drugs just in case. # <Failure> # ". Also in this case, since there is voice data in which re-speech recognition has failed, it is determined that the voice recognition result for the utterance has failed.

  Thereafter, when re-speech recognition is performed, re-speech recognition is requested only for the fourth sound data. If the voice recognition of this voice data is successful, the text displayed is "Visit Yamada-san. # The complexion is good. # But just in case you prescribe antipyretic drugs. "become that way.

  FIG. 6 shows an operation flow when a request for re-speech recognition is made only for speech data that has failed in speech recognition. Step S206 is added between S202 and S203 in the flowchart of FIG. After obtaining the speech file of the speech that failed in speech recognition in step S202, in step S206, only the speech data that failed in speech recognition is cut out from the speech file. In step S203, the voice recognition system is requested for voice recognition only for the cut out voice data. The other steps are the same as in FIG.

  As in the present embodiment, when performing speech recognition only for a part of speech data, the speech recognition system 201 may improve speech recognition accuracy if it also refers to the text of the previous successful portion. Therefore, text obtained by converting one or more previous voice data of the partial voice data may be read from the recognition result table and transmitted together with the partial voice data.

  Alternatively, the voice recognition accuracy of the part of the voice data may be improved by transmitting one or more pieces of voice data before the part of the voice data. As described above, in the present embodiment, when only voice data that has failed in voice recognition is sent, when the voice data and one or more previous voice data are sent, all voice data as described above is sent. Both are included. That is, as long as the data includes voice data that has failed in voice recognition, the voice data to be transmitted is arbitrary. Of course, it is possible to include the next voice data after the voice data that failed to be recognized.

  In the above description, after the success or failure of the voice recognition of each voice data is confirmed, the text is displayed on the screen of the display unit 17. However, each time the voice recognition result is acquired, the screen is sequentially displayed. May be displayed. As a result, the time from voice recognition to result display can be shortened, and display with improved real-time performance is possible.

(Fourth embodiment)
In the first embodiment, when voice recognition fails, the voice recognition system is requested to perform re-voice recognition based on the recorded voice file. However, even if the voice recognition is requested again a certain number of times, the voice recognition may not succeed. In addition, there may be a case where the speech recognition is not successful for a long time even though the re-speech recognition is periodically performed after the first request for the speech recognition is unsuccessful. In such a case, it is considered that there is a possibility that the voice data itself has a problem, so that voice recognition may not be performed any more. Hereinafter, this embodiment will be described in detail.

  The control unit 16 determines whether or not the number of re-requests for speech recognition has reached a certain value for an utterance that has failed speech recognition. For utterances that reach a certain value, control is performed so that speech recognition is no longer performed. In order to store the number of requests for voice recognition, for example, a column of “number of times of re-recognition” may be separately provided in the recognition result table of FIG. This column stores the number of requests for speech recognition using speech files for utterances whose speech recognition result is <failure>.

  The control unit 16 performs control so that re-speech recognition is not requested for an utterance in which the number of times reaches a certain value. In this case, the control unit 16 may display on the display unit 17 a message notifying the user that voice recognition regarding this utterance is no longer performed. For example, a message such as “speech recognition failed over the specified number of times. Voice recognition is not performed again” may be displayed.

  Similarly, when the voice recognition is not successful within a certain time from the first utterance date and time, the control unit 16 controls the voice recognition request not to be performed any more. For example, in the recognition result table of FIG. 4, utterances whose recognition result is <failure> are detected when the interval between the value of “utterance date” and the current date is greater than or equal to a certain time (eg, 3 hours). . The detected utterance is controlled so as not to request further voice recognition. In that case, a message notifying the user that voice recognition regarding this utterance is no longer performed may be displayed on the display unit 17. For example, a message such as “speech recognition is not successful even if a predetermined time has elapsed since the time of utterance. Repeated speech recognition is not performed” may be displayed. Here, the utterance date / time is set as the starting point of a certain time, but any date / time may be set as the starting point.

  As described above, according to the present embodiment, it is possible to reduce the processing load of the voice processing apparatus 101 by preventing voice recognition from being performed any more for an utterance that is considered to be unlikely to succeed in voice recognition. In addition, since the user can shorten the time for waiting for the voice recognition to be successful, the load on the user can be reduced. In addition, the user can speak again without forgetting the utterance content.

(Fifth embodiment)
In the first to fourth embodiments, the voice processing apparatus requests the voice recognition system to perform voice recognition. However, in this embodiment, a management server arranged between the voice processing apparatus and the voice recognition system is acting as a proxy. Do. This reduces the load on the voice processing device and saves the storage area of the voice processing device.

  FIG. 7 is an overall configuration diagram of a system according to the present embodiment.

  A voice processing device 401, a voice file management server 501 (hereinafter referred to as a management server 501), and a voice recognition system 201 are shown. These are connected to each other via a network. The network is a wireless network, a wired network, or a hybrid network thereof. The network between the voice processing device 401 and the management server 501 and the network between the management server 501 and the voice recognition system 201 may be different networks or the same network.

  Since the voice recognition system 201 is the same as the voice recognition system 201 according to the first to fourth embodiments, the description thereof is omitted.

  The functional block diagram of the audio processing device 401 can be the same as that shown in FIG. However, the operation of each block is partially changed or expanded. Below, it demonstrates focusing on the difference with 1st Embodiment.

  As in the first embodiment, the recording unit 11 combines the audio data acquired for each fixed size to generate an audio file, and stores the audio file in the file storage unit 12. However, each audio data is not sent to the transmission unit 13. That is, the voice recognition request for each voice data is not transmitted to the voice recognition system 201.

  The control unit 16 reads the audio file from the file storage unit 12 and passes it to the transmission unit 13. The transmission unit 13 transmits the audio file to the management server 301. The control unit 16 may delete the audio file from the file storage unit 12 when the transmission of the audio file is successful. Thereby, the data storage area can be saved. The control unit 16 may output a message to the display unit 17 notifying the user that it is not necessary to utter the same content again after transmission of the audio file or creation of the audio file.

  The management server 501 receives an audio file from the audio processing device 401. Similar to the voice processing device 401, the management server 501 includes a server-side file storage device and a recognition result storage device. The management server 501 stores the received audio file in the server-side file storage device. The management server 501 transmits a voice recognition request for the voice data included in the voice file to the voice recognition system 201 based on the voice file in the file storage device. The management server 501 receives a voice recognition response including a voice recognition result from the voice recognition system 201. The speech recognition result includes text obtained by converting speech data by speech recognition when speech recognition is successful, and includes information indicating that speech recognition has failed when speech recognition fails. When the speech recognition is successful, information indicating success may be additionally included in the speech recognition result. Based on the voice recognition result included in the received voice recognition response, the management server 501 generates a recognition result table in the format shown in FIG. 4 in the server-side recognition result storage unit 15 as in the first embodiment. Also good.

  Similar to the first embodiment, the request for speech recognition may be performed collectively for the entire audio data in the audio file, or the entire audio data may be divided into predetermined sizes. . Further, when the voice recognition system 201 supports the voice file itself, the voice file itself may be transmitted. If the audio file is compressed into a format that the voice recognition system 201 does not support, the audio file is decoded into a format that the voice recognition system 201 can handle.

  Similar to the operation of the speech processing apparatus 101 of the first to fourth embodiments, the management server 501 controls a re-speech recognition request for an utterance that has failed speech recognition. The management server 501 checks utterances that have failed in voice recognition based on a table held in the recognition result storage unit 15 at regular time intervals. The management server 501 extracts a voice file corresponding to an utterance in which voice recognition has failed, and transmits a voice recognition request for voice data in the voice file to the voice recognition system 201 again. A voice recognition response including a voice recognition result is received from the voice recognition system 201. The recognition result table is updated according to the content of the speech recognition result included in the speech recognition response.

  When the voice recognition of the voice file is successful, the management server 501 may notify the voice processing device 401 of a message indicating that the voice recognition is successful. The management server 501 manages a correspondence table between users of the voice processing device 401 and mail addresses in advance. The management server 501 identifies the mail address of the user of the corresponding voice processing device 401 based on the correspondence table, and transmits a success message to the identified address. A speech-recognized text (utterance text) may be added to the success message, or the speech text may be transmitted as an attached file. Alternatively, the success message may be sent in the form of being pushed on the screen of the application instead of mail. In this case, the utterance text may be simultaneously displayed on the screen of the application. The utterance text may be transmitted in response to a transmission request from the terminal of the user who confirmed the success message on the application screen.

  The management server 501 may transmit a message indicating failure to the voice processing device 401 when the first or subsequent voice recognition fails. At this time, a message indicating that a request for speech recognition is voluntarily performed may be transmitted again. As in the fourth embodiment, a message indicating that speech recognition is impossible may be transmitted when speech recognition fails a certain number of times or when speech recognition is not successful within a certain time.

  When receiving the success message and the utterance text from the management server 501, the reception unit 14 of the speech processing device 401 displays the success message on the display unit 17, and the utterance text is the same as or different from the success message. The image is displayed on the display unit 17 on the screen. The received utterance text is stored in the recognition result storage unit. At this time, it may be stored together with the utterance date and time in a format according to the recognition result table of FIG. In the case where the audio file is deleted from the file storage unit 12, the “audio file” column may be deleted.

  When the reception unit 14 receives a failure message from the management server 501, the reception unit 14 displays the failure message on the display unit 17. In this case, the control unit 16 may record information indicating failure in the “recognition result” column. In addition, when receiving a message indicating that a request for voice recognition is voluntarily performed or a message indicating that voice recognition is not possible, the receiving unit 14 displays the message on the display unit 17.

  The control unit 16 of the voice processing device 401 may periodically inquire the management server 501 as to whether or not voice recognition of the voice file has succeeded. The utterance to be inquired is an utterance that has not yet received a success message. An utterance that has not received a success message after receiving a failure message also corresponds to this. In this case, the management server 501 may send a message to that effect to the voice processing device 401 if the voice recognition of the voice file is successful. If voice recognition of the voice file has not been successful, a message indicating that re-voice recognition is in progress may be sent to the voice processing device 401.

  The operation of the management server 501 described above is an example, and various operations of the voice processing device 401 in the first to fourth embodiments can be combined. For example, as shown in the third embodiment, when the speech recognition is successful only for a part of the speech data of the utterance, the text obtained by combining the text of the successful part and the text indicating the failure is sent to the speech processing device 401. It is also possible to transmit. Also, when controlling the re-speech recognition request, only the failed speech data may be cut out and the re-speech recognition may be requested to the speech recognition system 201.

(Sixth embodiment)
In the fifth embodiment, a voice recognition request and re-request are made from the management server to the voice recognition system. In this embodiment, the voice recognition request is made by the voice processing device, and the voice recognition re-request is sent to the management server. Shows a form to be performed.

  FIG. 8 is an overall configuration diagram of a system according to the present embodiment. A voice processing device 601, a voice file management server 701 (hereinafter referred to as a management server 701), and a voice recognition system 201 are shown. These are connected to each other via a network. The network is a wireless network, a wired network, or a hybrid network thereof. The network between the voice processing device 601 and the voice processing system 201, the network between the voice processing device 601 and the management server 701, and the network between the management server 701 and the voice recognition system 201 are the same even if they are different networks. It may be a network.

  The voice recognition system 201 is the same as the voice recognition system 201 according to the first to fifth embodiments. Further, the functional block diagram of the voice processing device 601 can be the same as that shown in FIG. However, the operation of each block is partially changed or expanded. Below, it demonstrates focusing on the difference with 1st and 5th embodiment.

  The audio processing device 601 performs the processing of steps S101 to S113 in FIGS. 2 and 3 as in the first embodiment. That is, every time voice data is acquired by the recording unit 11, the voice data is transmitted to the voice recognition system 201 via the transmission unit 13, and a voice recognition result is acquired from the voice recognition system 201. When the voice recognition results for all the voice data transmitted to the voice recognition system 201 are acquired, it is determined whether the voice recognition for the current utterance has succeeded or failed. In the case of success, the utterance text obtained by combining the texts included in each speech recognition result is displayed on the screen, and in the case of failure, a failure message or the like is displayed on the screen. In the first embodiment, after that, an entry is added to the recognition result table (see FIG. 4). However, in this embodiment, instead of this, the utterance date and time, the voice recognition result, and the voice file are added to the management server 701. As a set of information. That is, information corresponding to one entry in FIG. 4 (however, not a file path but an audio file body) is transmitted to the management server 701. If the voice processing apparatus 601 successfully transmits the information to the management server 701, the voice processing apparatus 601 may delete the voice file in the voice processing apparatus 601. Note that the voice processing device 601 may add the utterance date and the entry of the recognition result to the recognition result table.

  The management server 701 stores a voice file included in the above information (speech date and time, voice recognition result, and voice file) in a file storage unit on the server side, and acquires a file path for specifying the stored position. In addition, the management server 701 adds an entry including the utterance date and time and the voice recognition result included in the information and the acquired file path to the recognition result table held in the recognition result storage unit on the server side.

  The management server 701 controls a re-speech recognition request for an utterance that has failed speech recognition based on the “recognition result” column of the recognition result table. That is, the management server 701 checks utterances that have failed in voice recognition based on a table held in the recognition result storage unit at regular time intervals. The management server 701 extracts a voice file corresponding to the utterance in which voice recognition has failed, and transmits a voice recognition request for the voice data in the voice file to the voice recognition system 201 again. A voice recognition response including a voice recognition result is received from the voice recognition system 201. The recognition result table is updated according to the content of the speech recognition result included in the speech recognition response.

  When the voice recognition of the voice file is successful, the management server 701 may notify the voice processing device 601 of a message indicating that the voice recognition is successful. A speech-recognized text (utterance text) may be added to the success message, or the speech text may be transmitted as an attached file. Alternatively, the success message may be sent in the form of being pushed on the screen of the application instead of mail. In this case, the utterance text may be simultaneously displayed on the screen of the application. The utterance text may be transmitted in response to a transmission request from the terminal of the user who confirmed the success message on the application screen.

  When the voice recognition fails, the management server 701 may transmit a message indicating the failure to the voice processing device 601. At this time, a message indicating that a request for speech recognition is voluntarily performed may be transmitted again. As in the fourth embodiment, a message indicating that speech recognition is impossible may be transmitted when speech recognition fails a certain number of times or when speech recognition is not successful within a certain time.

  When receiving the success message and the utterance text from the management server 701, the reception unit 14 of the voice processing device 601 displays the success message on the display unit 17, and the utterance text is the same as or different from the success message. The image is displayed on the display unit 17 on the screen. The received utterance text is stored in the recognition result storage unit 15. At this time, the utterance text may be stored together with the utterance date and time in a format according to the recognition result table of FIG. In the case where the audio file is deleted from the file storage unit 12, the “audio file” column may be deleted.

  When the reception unit 14 receives a failure message from the management server 701, the reception unit 14 displays the failure message on the display unit 17. In this case, the control unit 16 may record information indicating failure in the “recognition result” column. In addition, when receiving a message indicating that a re-request for speech recognition is voluntarily performed or a message indicating that speech recognition is not possible, the receiving unit 14 displays the message on the display unit 17.

  Further, the control unit 16 of the voice processing device 601 may periodically inquire the management server 701 as to whether or not voice recognition of the voice file has succeeded. The utterance to be inquired is an utterance that has not yet received a success message. An utterance that has not yet received a success message after receiving a voice recognition failure message also corresponds to this. In this case, the management server 701 may send a message to that effect to the voice processing device 601 if the voice recognition of the voice file is successful. If the voice recognition of the voice file is not yet successful, a message indicating that voice recognition is in progress may be transmitted to the voice processing device 601.

  As described above, according to the present embodiment, the first speech recognition request is made from the speech processing apparatus, and the re-speech recognition request is made from the management server, so that the time from the speech recognition request to the result display can be shortened. At the same time, the processing load due to the re-recognition request can be reduced or reduced.

  Note that the voice processing devices and management servers of the first to sixth embodiments can also be realized by using, for example, a general-purpose computer device as basic hardware. That is, the processing of the blocks provided in the voice processing device and the management server can be realized by causing the processor mounted on the computer device to execute the program. At this time, the voice processing device and the management server may be realized by installing the above program in the computer device in advance, or may be stored in a storage medium such as a CD-ROM or via the network. And may be realized by installing this program on a computer device as appropriate. The storage means included in the voice processing device and the management server is a storage medium such as a memory, a hard disk, an SSD or a CD-R, a CD-RW, a DVD-RAM, a DVD-R, etc. incorporated in or externally attached to the computer device It can be realized by appropriately using the above.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

11: recording unit 12: file storage unit 13: transmission unit 14: reception unit 15: recognition result storage unit 16: control unit 17: display unit 18: input unit 101, 401: voice processing device 301: network 201: voice recognition system

Claims (10)

An acquisition unit for sequentially acquiring audio data representing the content spoken by the user;
A transmission unit that transmits a voice recognition request of the voice data acquired by the acquisition unit to a voice recognition system;
A storage unit for storing voice data acquired by the acquisition unit;
A receiving unit that receives a voice recognition response including text converted from the voice data by voice recognition or information indicating a voice recognition failure of the voice data from the voice recognition system;
Based on the voice recognition response, the voice data that has failed in the voice recognition is identified, and on the basis of the voice data stored in the storage unit, a voice recognition request for data including the voice data that has failed in the voice recognition And a control unit that controls transmission to the system. The control unit determines success or failure of speech recognition for the user's utterance based on the speech recognition response, and if the failure is determined, the control unit re-requests the speech recognition system to perform speech recognition again. The voice processing device according to claim 1, wherein the voice processing device is controlled to display a message to be notified. The control unit controls to display a message notifying the user that it is not necessary to perform the same utterance again after the acquisition unit completes acquisition of all voice data for the user's utterance. Item 3. The voice processing device according to Item 1 or 2. When the voice recognition response for at least one voice data is not received and no voice recognition response including information indicating the failure is received, the control unit indicates that voice recognition is in progress. The voice processing apparatus according to any one of claims 1 to 3, wherein a message for notifying the user of a message is controlled to be displayed. The control unit indicates the text included in the voice recognition response for the voice data for which the voice recognition has been successful, and indicates that the voice recognition has not been completed for the voice data for which the voice recognition has not yet been successful. The speech processing apparatus according to any one of claims 1 to 4, wherein text is controlled to be arranged and displayed according to an order of acquisition of the speech data by the acquisition unit. The speech processing apparatus according to any one of claims 1 to 5, wherein the transmission unit transmits the speech recognition request to the speech recognition system via a network. The acquisition unit combines the audio data acquired by the acquisition unit in the acquisition order to generate an audio file,
The storage unit stores the audio file,
The control unit generates correspondence information in which the date and time when the user utters, the voice recognition result for the user's utterance, and the position where the voice file is stored in the storage unit,
The speech recognition result indicates that if all speech recognition of the speech data is successful, the speech text in which the text corresponding to each speech data is arranged in the order of acquisition of the speech data, speech recognition of at least one speech recognition data The speech processing apparatus according to any one of claims 1 to 6, which is information indicating that speech recognition has not yet been completed when is not successful. If the voice recognition is not successful even if the voice recognition request for the data including the voice data that has failed in the voice recognition is performed a predetermined number of times, or the voice recognition has failed within a predetermined time from a predetermined time. 8. If re-speech recognition of data including speech data is not successful, control is performed so as to display a message notifying the user that speech recognition for the user's utterance is no longer performed. The speech processing apparatus according to the item. An acquisition unit that sequentially acquires audio data representing the content spoken by the user, generates an audio file by combining the acquired audio data in order of acquisition, and
A transmitter that transmits the voice file to a management server that communicates with the voice recognition system via a network;
A receiving unit that receives text obtained by converting voice data included in the voice file by voice recognition from the management server;
After the audio file is generated or after the transmission of the audio file to the management server is completed, a message is displayed to notify the user that it is not necessary to perform the same utterance as the utterance again. A voice processing device comprising a control unit for controlling. An acquisition step of sequentially acquiring audio data representing the content spoken by the user;
A transmission step of transmitting a voice recognition request of the voice data acquired by the acquisition step to a voice recognition system;
A storage step of storing the voice data acquired by the acquisition step in a storage device;
Receiving from the voice recognition system a voice recognition response including text converted from the voice data by voice recognition or information indicating a voice recognition failure of the voice data;
Based on the voice recognition response, the voice data that has failed in the voice recognition is identified, and based on the voice data stored in the storage device, a voice recognition request for data including the voice data that has failed in the voice recognition is received. A voice processing method comprising: a control step for controlling transmission to a system.

Images