JP6610195B2 - Terminal device and communication method - Google Patents

Description

  The present invention relates to a communication technique, and more particularly to a terminal device and a communication method for executing communication of an audio signal.

  The voice recognition process becomes unrecognizable when noise in the surrounding environment is added or the voice is low. Furthermore, even if the voice recognition process is repeatedly executed, recognition may not be possible. In particular, if the operator does not know the reason why the recognition is impossible, the recognition failure is likely to be repeated. In order to prevent this, the operator is notified of the reason why the recognition is impossible (for example, Patent Document 1).

JP 2000-112497 A

  The voice recognition process is generally performed on a voice signal that is voice data. On the other hand, for the voice heard by the receiver, the equalizer is turned on / off, the volume level when being output from the speaker, and the speech speed conversion is turned on / off. Therefore, the voice heard by the receiver may be different from the voice signal. Moreover, even when the voice reproduced from the same voice signal is heard, the way of hearing may differ depending on the receiver. Therefore, it is desirable that the voice recognition process is performed according to the situation on the receiver side.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique for informing the result of speech recognition made according to the situation on the receiver side.

  In order to solve the above-described problem, a terminal device according to an aspect of the present invention is a terminal device, and includes a receiving unit that receives an audio signal from another terminal device, and a reproduction that reproduces the audio signal received by the receiving unit. A processing unit that executes voice recognition processing that reflects how a user using the terminal device hears the voice signal that is played back by the playback unit, and the result of the voice recognition processing by the processing unit as another terminal device. And a transmission unit for transmitting to.

  Another aspect of the present invention is a communication method. This method is a communication method in a terminal device, the step of receiving an audio signal from another terminal device, the step of reproducing the received audio signal, and the terminal device for the reproduced audio signal. A step of executing voice recognition processing reflecting how the user uses it, and a step of transmitting the result of the voice recognition processing to another terminal device.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to notify the result of speech recognition made according to the situation on the receiver side.

It is a figure which shows the structure of the communication system which concerns on Example 1 of this invention. It is a figure which shows the structure of the terminal device of FIG. FIGS. 3A to 3B are diagrams showing screens displayed on the display unit of FIG. It is a sequence diagram which shows the communication procedure by the communication system of FIG. FIGS. 5A to 5B are diagrams illustrating screens displayed on the display unit according to the first embodiment of the present invention. It is a sequence diagram which shows the communication procedure by the communication system which concerns on Example 2 of this invention. It is a figure which shows the structure of the terminal device which concerns on Example 3 of this invention. FIGS. 8A and 8B are diagrams showing screens displayed on the display unit of FIG. It is a flowchart which shows the comparison procedure by the comparison part of FIG. FIGS. 10A to 10C are diagrams illustrating screens displayed on the display unit according to the fourth embodiment of the present invention. It is a sequence diagram which shows the communication procedure by the communication system which concerns on Example 4 of this invention. It is a flowchart which shows the specific procedure by the terminal device which concerns on Example 4 of this invention. It is a flowchart which shows another specific procedure by the terminal device which concerns on Example 4 of this invention. It is a flowchart which shows another specific procedure by the terminal device which concerns on Example 4 of this invention.

Example 1
Before describing the present invention specifically, an outline will be given first. Example 1 of this invention is related with the terminal device which performs the audio | voice communication by PTT (Push to Talk). The terminal device includes a button, and transmission is started when the user depresses the button, and transmission is ended when the button is released. On the other hand, if the button is not pressed, the user only listens to the message. In such a PTT, the speaker is completed only by the action of speaking and transmitting, and how the message is transmitted depends on the response of the receiver. Even if the sender thinks that the spoken content is correctly communicated to the receiver, the voice may not be transmitted to the receiver as intended because the communication status is poor, noisy, or too fast is there.

  Therefore, the terminal device on the reception side converts the received voice signal into text by executing voice recognition processing, and transmits the text data (hereinafter referred to as “text data”) to the terminal device on the transmission side. The terminal device on the transmission side displays the text data, and the sender confirms whether the intended voice is output by confirming the text data. However, since there are individual differences in how the listener hears, the same voice may be heard differently. For this reason, the content of the text data may be different from the content actually recognized by the receiver.

  In order to cope with this, the terminal device according to the present embodiment, particularly the receiving-side terminal device, executes a speech recognition process using a speech recognition model of a user who uses the terminal device, that is, a receiver. Therefore, the content of the text data generated in the terminal device reflects how the listener hears.

  FIG. 1 shows a configuration of a communication system 100 according to Embodiment 1 of the present invention. The communication system 100 includes a first terminal device 10a, a second terminal device 10b, collectively referred to as a terminal device 10, a first base station device 12a, a second base station device 12b, generally referred to as a base station device 12, and a network 14. . Here, the communication system 100 corresponds to, for example, business radio.

  The terminal device 10 is a device that can execute communication by commercial wireless. Since publicly known technology may be used for commercial radio, description thereof is omitted here. Here, the first terminal device 10a corresponds to the transmission side of voice communication by business radio, and the second terminal device 10b corresponds to the reception side of voice communication by business radio. For this reason, the first terminal device 10a is used for a speaker, and the second terminal device 10b is used for a receiver. The relationship between the first terminal device 10a and the second terminal device 10b may be reversed, and the number of terminal devices 10 is not limited to “2”.

  The base station device 12 corresponds to a business radio system, and is connected to the terminal device 10 by business radio on one end side and connected to the base station device 12 on the other end side. The first base station apparatus 12a and the second base station apparatus 12b are installed at different positions. In the business radio, a group can be formed by a plurality of terminal devices 10. The base station apparatus 12 may assign an uplink channel and a downlink channel to the group. Under such circumstances, one terminal apparatus 10 in the group transmits a signal on the uplink channel, and another terminal apparatus 10 in the group receives a signal on the downlink channel.

  The network 14 connects the first base station apparatus 12a and the second base station apparatus 12b. With this connection, the first terminal device 10a and the second terminal device 10b can communicate with each other via the first base station device 12a, the network 14, and the second base station device 12b. Here, it is assumed that the voice communication is based on PTT.

  FIG. 2 shows the configuration of the terminal device 10. The terminal device 10 includes a button 20, a microphone 22, an operation unit 24, a display unit 26, a processing unit 28, a communication unit 30, a playback unit 32, and a speaker 34. The processing unit 28 includes a transmission unit 36 and a setting unit 38, and the communication unit 30 includes a transmission unit 40 and a reception unit 42. The terminal device 10 corresponds to both the terminal device 10 on the transmission side and the terminal device 10 on the reception side. Here, for the sake of clarity, the description will be made in the order of (1) transmission side, (2) reception side, and (3) transmission side in the order of processing.

(1) Transmission side Here, the process in the terminal device 10 on the transmission side will be described. The button 20 corresponds to a PTT button, and is pressed down by the user when transmitting sound by PTT. Further, the button 20 is kept pressed down while the voice is transmitted. The depression of the button 20 corresponds to receiving an instruction for transmitting an audio signal. The button 20 continues to output an instruction to the transmitter 36 while being pressed down. The microphone 22 collects sound around the terminal device 10. The microphone 22 converts the collected sound into an electrical signal (hereinafter referred to as “voice signal”) and outputs the voice signal to the transmitter 36.

  The transmitter 36 receives the audio signal from the microphone 22 when receiving an instruction from the button 20. The transmitter 36 converts the audio signal from an analog signal to a digital signal, and outputs the audio signal converted to a digital signal (hereinafter also referred to as “audio signal”) to the transmitter 40. On the other hand, the transmitter 36 does not execute such processing when it has not received an instruction from the button 20. The transmission unit 40 receives the audio signal from the transmission unit 36 and transmits the audio signal to the terminal device 10 on the reception side. In order to transmit an audio signal, the transmission unit 40 performs encoding, modulation, frequency conversion, amplification, and the like.

(2) Receiving Side Here, the processing in the terminal device 10 on the receiving side will be described following (1). The receiving unit 42 receives an audio signal from the terminal device 10 on the transmission side. The receiving unit 42 performs amplification, frequency conversion, demodulation, decoding, and the like. The receiving unit 42 outputs the processing result (hereinafter also referred to as “audio signal”) to the reproducing unit 32. The reproduction unit 32 receives the audio signal from the reception unit 42 and reproduces the audio signal. Since a known technique may be used for reproduction of the audio signal, description thereof is omitted here. The reproduction unit 32 outputs the reproduced audio signal to the speaker 34 and the processing unit 28. The speaker 34 converts an audio signal, which is an electrical signal, into sound and outputs the sound.

  The processing unit 28 inputs the audio signal from the reproduction unit 32. On the other hand, in the setting unit 38, a voice recognition model of a user who uses the terminal device 10, that is, a specified receiver is set. In the speech recognition model, for example, a waveform of a speech signal corresponding to the phoneme “a” is stored. Further, such a waveform of the audio signal is stored for each phoneme. In particular, the stored phonemes and waveforms are associated with each other so as to be recognized by the identified listener who has heard the speech, and thus these relationships can be said to be a speech recognition model of the listener.

  The processing unit 28 performs voice recognition processing on the voice signal while using the voice recognition model of the listener set in the setting unit 38. Specifically, the processing unit 28 selects a waveform closest to the waveform of the input speech signal from the speech recognition model, and specifies a sound corresponding to the selected waveform. The voice signal is converted into text by the voice recognition process. As described above, the processing unit 28 performs voice recognition processing based on the voice recognition model of the user who uses the terminal device 10, that is, voice recognition processing reflecting how the user hears the voice signal.

  The listener sets the voice recognition model of the receiver in the setting unit 38 while operating the operation unit 24. For example, the setting unit 38 stores a waveform of a test audio signal in advance, and outputs it from the speaker 34 by reproducing it in the reproducing unit 32. The listener inputs the sound recognized by using the operation unit 24 while listening to the sound from the speaker 34. The setting unit 38 sets the voice recognition model of the listener based on the correspondence between the waveform of the test voice signal and the input sound.

  The transmission unit 40 inputs text data from the processing unit 28 as a result of the speech recognition processing in the processing unit 28. The transmission unit 40 transmits the text data to the terminal device 10 on the transmission side. In addition, the transmission of the text data in the transmission unit 40 may be performed alone, or may be performed together with the transmission of the audio signal.

(3) Transmission side Here, the processing in the terminal device 10 on the transmission side following (2) will be described. The receiving unit 42 receives text data from the terminal device 10 on the receiving side. The receiving unit 42 outputs the text data to the processing unit 28. The processing unit 28 inputs the text data from the receiving unit 42 and displays the text data on the display unit 26. The transmitter recognizes how the receiver is listening by checking the text data displayed on the display unit 26. 3A to 3B show screens displayed on the display unit 26. FIG. FIG. 3A shows a case where the sender speaks “Anzen” and the receiver hears “Anzen”. In this case, the content spoken by the sender coincides with the content heard by the receiver. On the other hand, FIG. 3B shows a case where the sender speaks “Anzen” and the receiver hears “Kanzen”. In this case, the content spoken by the sender is different from the content heard by the receiver.

  This configuration can be realized in terms of hardware by a CPU, memory, or other LSI of any computer, and in terms of software, it can be realized by a program loaded in the memory, but here it is realized by their cooperation. Draw functional blocks. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The operation of the communication system 100 configured as above will be described. FIG. 4 is a sequence diagram illustrating a communication procedure performed by the communication system 100. When the first terminal apparatus 10a inputs voice (S10), the first terminal apparatus 10a generates a voice signal (S12). The first terminal device 10a transmits an audio signal (S14). The second terminal device 10b reproduces the audio signal (S16), and outputs the reproduced audio signal from the speaker 34 (S18). The second terminal device 10b executes speech recognition processing using the user's speech recognition model (S20), and generates text data (S22). The second terminal device 10b transmits text data (S24). The first terminal apparatus 10a displays text data (S26).

  According to the present embodiment, since processing based on the voice recognition model of the user who uses the terminal device is executed on the voice signal, the received voice signal can be converted into text while reflecting how the user hears it. In addition, since the received voice signal is converted into text while reflecting how the user hears, accurate information can be notified to the speaker. In addition, since voice recognition processing is performed on the voice signal while using the user's voice recognition model, it is possible to specify the listener's mistake in hearing due to the pronunciation of the sender. In addition, since voice recognition processing is performed on the voice signal while using the user's voice recognition model, it is possible to specify a listener's mistake in listening in the wireless section.

(Example 2)
Next, Example 2 will be described. Similarly to the first embodiment, the second embodiment of the present invention also relates to a terminal device that performs voice communication by PTT, and also relates to a terminal device that converts a voice signal into text using a voice recognition model of a listener. The voice recognition model according to the first embodiment is configured for a waveform of a voice signal recognized by a listener. On the other hand, the voice recognition model according to the second embodiment is configured for a voice speed that can be recognized by the receiver and a volume level that can be recognized by the receiver. The communication system and the terminal device according to the second embodiment are the same type as those shown in FIGS. Here, it demonstrates centering on the difference from before.

  In the above (2), the processing unit 28 inputs the audio signal from the reproduction unit 32. The processing unit 28 converts the voice signal into text by executing voice recognition processing on the voice signal. On the other hand, in the setting unit 38, a voice recognition model of a user who uses the terminal device 10, that is, a receiver is set. In the voice recognition model, for example, at least one of a voice speed value recognizable by the receiver and a volume level value recognizable by the receiver is stored.

  The processing unit 28 derives the value of the voice speed of the voice signal during a certain period by counting the number of characters in the text data. The processing unit 28 compares the derived voice speed value with the voice speed value stored in the setting unit 38 to determine whether the voice signal is equal to or lower than the voice speed that can be recognized by the receiver. Execute the process. If the derived voice speed value is larger than the voice speed value that can be recognized by the receiver, the processing unit 28 will select a portion of the text data that is larger than the voice speed that the receiver can recognize. Convert to face down. If the derived voice speed value is equal to or lower than the voice speed value that can be recognized by the receiver, the processing unit 28 does not perform conversion on the text data.

  Further, the processing unit 28 may derive the value of the volume level of the audio signal during a certain period. The processing unit 28 compares the derived volume level value with the volume level value stored in the setting unit 38 to determine whether or not the audio signal has a volume level that can be recognized by the receiver. Execute the process. If the derived volume level value is smaller than the volume level value that can be recognized by the receiver, the processing unit 28 converts each character of the text data to a hidden character. If the derived volume level value is equal to or greater than the volume level value that can be recognized by the receiver, the processing unit 28 does not perform conversion on the text data. As described above, at least one of the volume level and the voice speed is reflected in the voice recognition process in the processing unit 28. Here, the voice recognition model of the receiver is set in the setting unit 38 by operating the operation unit 24. The content to be set is at least one of a voice speed value that can be recognized by the receiver and a volume level value that can be recognized by the receiver.

  The transmission unit 40 inputs the text data from the processing unit 28. The transmission unit 40 transmits the text data to the terminal device 10 on the transmission side. As described above, when the voice speed is larger than the recognizable voice speed value or smaller than the listener's recognizable volume level value, at least some characters of the text data are converted to the hidden character. . This means that the transmission unit 40 transmits a determination result as to whether or not the voice speed is recognizable by the listener, or transmits a determination result as to whether or not the sound volume level is recognizable by the receiver. Equivalent to.

  In the above (3), the receiving unit 42 receives text data from the terminal device 10 on the receiving side, and the processing unit 28 displays the text data on the display unit 26. 5A to 5B show screens displayed on the display unit 26. FIG. FIG. 5A shows a case where the value of the voice speed of the voice signal reproduced in the terminal device 10 on the receiving side is larger than the value of the voice speed that can be recognized by the receiver. In this case, some of the characters are indicated by face-down characters. On the other hand, FIG. 5B shows a case where the value of the volume level of the audio signal reproduced in the terminal device 10 on the receiving side is smaller than the value of the volume level that can be recognized by the receiver. In this case, all the characters are indicated by a turn character.

  The operation of the communication system 100 configured as above will be described. FIG. 6 is a sequence diagram illustrating a communication procedure performed by the communication system 100 according to the second embodiment of the present invention. When the first terminal apparatus 10a inputs voice (S50), the first terminal apparatus 10a generates a voice signal (S52). The first terminal device 10a transmits an audio signal (S54). The second terminal device 10b reproduces the audio signal (S56), and outputs the reproduced audio signal from the speaker 34 (S58). The second terminal device 10b executes the speech recognition process (S60), and also executes the evaluation based on the voice speed and the volume level (S62), thereby generating text data (S64). The second terminal device 10b transmits text data (S66). The first terminal device 10a displays text data (S68).

  According to the present embodiment, since it is determined whether or not the voice signal is below the voice speed that can be recognized by the user, it is possible to judge whether the voice signal is difficult to hear due to the voice speed. In addition, it is possible to reflect in the text that it is difficult to hear due to the voice speed. In addition, since it is determined whether or not the sound level is equal to or higher than the sound volume level that can be recognized by the user, it is possible to determine whether it is difficult to hear due to the sound volume level. Moreover, it is possible to reflect in the text that it is difficult to hear due to the volume level.

Example 3
Next, Example 3 will be described. The third embodiment of the present invention also relates to a terminal device that performs voice communication using PTT, and also relates to a terminal device that converts a voice signal into text using a voice recognition model of a listener. In the third embodiment, not only the voice signal is converted into text, but also information that can be used to guess the situation in which the listener is listening to the voice is notified to the terminal device on the transmission side. The communication system according to the third embodiment is the same type as that shown in FIG. Here, it demonstrates centering on the difference from before.

  FIG. 7 shows the configuration of the terminal device 10 according to the third embodiment of the present invention. Compared with FIG. 2, the processing unit 28 in the terminal device 10 further includes a comparison unit 46. In the above (2), the processing unit 28 inputs the audio signal from the reproduction unit 32. Similar to the first embodiment, the processing unit 28 performs voice recognition processing based on the voice recognition model of the user who uses the terminal device 10, that is, voice recognition processing reflecting how the user hears the voice signal. . As a result, the audio signal is converted into text (hereinafter, the audio signal converted into text is referred to as “first text”).

  At that time, the processing unit 28 may determine whether there is a phoneme that cannot be recognized in the speech recognition process. For example, when the correlation value between the waveform of one phoneme of the input speech signal and the waveform closest to the waveform of the one phoneme is smaller than a predetermined value, the one phoneme is determined as an unrecognizable phoneme. The The processing unit 28 converts unrecognizable phonemes into face-down characters in the first text. In addition, it may be converted into another predetermined character instead of the hidden character, and a message “There is an unrecognizable phoneme” may be added to the first text.

  On the other hand, the processing unit 28 also stores a speech recognition model that is not specified by a user who uses the terminal device 10, that is, a standard speech recognition model. In the standard speech recognition model, for example, a standard waveform of a speech signal corresponding to the sound “A” is stored. The processing unit 28 executes voice recognition processing based on a standard voice recognition model, that is, voice recognition processing that does not reflect how the user hears the voice signal. Again, the audio signal is converted into text (hereinafter, the audio signal converted into text is referred to as “second text”). Note that the processing unit 28 may convert unrecognizable phonemes into face-down characters or the like in the second text as in the case of the first text.

  The comparison unit 46 inputs the first text and the second text. The comparison unit 46 compares the first text and the second text. Here, as a comparison, the first text and the second text are arranged. The comparison unit 46 outputs text data in which the first text and the second text are arranged to the transmission unit 40. The transmission unit 40 inputs the text data from the processing unit 28. The transmission unit 40 transmits text data as a comparison result to the terminal device 10 on the transmission side.

  In the above (3), the receiving unit 42 receives text data from the terminal device 10 on the receiving side, and the processing unit 28 displays the text data on the display unit 26. FIGS. 8A to 8B show screens displayed on the display unit 26. The first text is shown on the upper side of the screen as “reception voice recognition”, and the second text is shown on the lower side of the screen as “standard voice recognition”. FIG. 8A shows a case where there is a phoneme that cannot be recognized in the first text even though there is no phoneme that cannot be recognized in the second text. This is all recognized when the speech signal corresponding to the speech uttered by the speaker is recognized by the standard speech recognition model, but not recognized when the speech recognition process is performed by the receiver's speech recognition model. This corresponds to the existence of possible phonemes. That is, it can be said that the voice is not recognized depending on how the listener hears.

  FIG. 8B shows a case where there is an unrecognizable phoneme in both the first text and the second text. This corresponds to the presence of unrecognizable phonemes when the speech signal corresponding to the speech uttered by the speaker is subjected to speech recognition using either the standard speech recognition model or the speech recognition model of the receiver. . In this case, for example, the quality of the radio section between the first terminal apparatus 10a and the first base station apparatus 12a or the radio section between the second terminal apparatus 10b and the second base station apparatus 12b may be deteriorated. Presumed.

  The operation of the communication system 100 configured as above will be described. FIG. 9 is a flowchart showing a comparison procedure by the comparison unit 46. The comparison unit 46 acquires the result of the speech recognition process based on the user's speech recognition model (S80). On the other hand, the comparison unit 46 acquires the result of the speech recognition process based on the standard speech recognition model (S82). The comparison unit 46 arranges the results of both voice recognition processes as a comparison (S84).

  According to the present embodiment, since the result of the speech recognition process based on the user's speech recognition model is compared with the result of the speech recognition process based on the standard speech recognition model, an unrecognizable phoneme is generated at any stage. You can specify what to do. In addition, when there is an unrecognizable phoneme in the result of the speech recognition process based on the user's speech recognition model, and there is no unrecognizable phoneme in the result of the speech recognition process based on the standard speech recognition model, It can be recognized that only the user cannot hear. In addition, when there is an unrecognizable phoneme in the result of the voice recognition process based on the user's voice recognition model and the result of the voice recognition process based on the standard voice recognition model, there is a cause in the utterance or communication stage. I can recognize that.

(Example 4)
Next, Example 4 will be described. The fourth embodiment of the present invention also relates to a terminal device that performs voice communication using PTT, and also relates to a terminal device that converts received voice signals into text. In the terminal device on the receiving side, the equalizer may be turned on / off, the volume level when output from the speaker, and the speech speed conversion may be turned on / off. At that time, the sound processed according to such setting is output from the speaker. That is, there is a possibility that the sound actually output from the speaker and the content of the text data are different due to such processing.

  In order to cope with this, the terminal device according to the present embodiment, particularly the receiving-side terminal device, executes a speech recognition process according to the setting made in the terminal device. Therefore, the content of the text data generated in the terminal device reflects processing according to the setting. The communication system and the terminal device according to the fourth embodiment are the same type as those in FIGS. Here, it demonstrates centering on the difference from before.

  In the above (2), various settings are made in the setting unit 38 of the terminal device 10 regarding the output of the sound. With this setting, a setting value to be used when the audio signal received by the receiving unit 42 is reproduced is registered. One of the settings made in the setting unit 38 is on / off of the equalizer. The equalizer is a process for changing the frequency characteristics of the audio signal. When the equalizer is turned on, it is possible to emphasize or reduce a specific frequency band (overtone component, harmonic component or noise component) of the audio signal. Another one of the settings made in the setting unit 38 is on / off of voice speed conversion. The audio speed conversion is a process for increasing or decreasing the audio reproduction speed. Further, another one of the settings made in the setting unit 38 is adjustment of the volume level. The volume level is a volume when outputting sound from the speaker 34. These settings are made by operating the operation unit 24. Further, it is not necessary for the setting unit 38 to have all of these settings, and it is sufficient that at least one setting is made.

  The reproduction unit 32 receives the audio signal from the reception unit 42 and reproduces the audio signal. At that time, the setting value made in the setting unit 38 is reflected. For example, when the equalizer is turned on in the setting unit 38, the reproducing unit 32 reproduces the audio signal while performing the equalizer process. On the other hand, when the equalizer is turned off in the setting unit 38, the reproducing unit 32 reproduces the audio signal without performing the equalizer process.

  When the audio speed conversion is turned on in the setting unit 38, the reproducing unit 32 reproduces the audio signal while converting the audio speed. Note that the audio speed is set to 2 times, 3 times, and 1/2 times. On the other hand, when the audio speed conversion is turned off in the setting unit 38, the reproduction unit 32 reproduces the audio signal without converting the audio speed. Furthermore, at the volume level set by the setting unit 38, the reproduction unit 32 adjusts the volume level when reproducing the audio signal. In addition, since a well-known technique should just be used for an equalizer process, audio | voice speed conversion, volume level adjustment, and audio | voice signal reproduction | regeneration, description is abbreviate | omitted here. As described above, it is not necessary for the setting unit 38 to perform all of these settings, and it is sufficient that at least one setting is made. Therefore, the playback unit 32 may use at least one setting. The reproduction unit 32 outputs the reproduced audio signal to the speaker 34 and the processing unit 28. The speaker 34 converts an audio signal, which is an electrical signal, into sound and outputs the sound.

  The processing unit 28 inputs the audio signal from the reproduction unit 32. The processing unit 28 performs a speech recognition process on the speech signal based on the standard speech recognition model described above. The voice signal is converted into text by the voice recognition process. Further, when there is a phoneme that cannot be recognized in the speech recognition process, the processing unit 28 may specify the reason (hereinafter, “error reason”). Here, as the reason for the error, (A) whether there is a phoneme that cannot be recognized in the voice recognition process, (B) whether the voice speed of the reproduced voice signal is greater than a threshold value, or (C) the reproduced voice Whether the volume level in the signal is less than a threshold is specified. The threshold values in (B) and (C) may be different values. Below, these processes are demonstrated in order.

(A) Is there a phoneme that cannot be recognized in the voice recognition process? The processing unit 28 determines whether there is a phoneme that cannot be recognized in the voice recognition process. For example, when the correlation value between the waveform of one phoneme of the input speech signal and the waveform closest to the waveform of the one phoneme is smaller than a predetermined value, the one phoneme is determined as an unrecognizable phoneme. The The processing unit 28 converts unrecognizable phonemes into hidden characters in the text data. In addition, it may be converted into another predetermined character instead of the hidden character, and a message “There is an unrecognizable phoneme” may be added to the text data. In other words, the processing unit 28 determines whether there is a phoneme that cannot be recognized in the voice recognition process performed on the voice signal reproduced by the reproduction unit 32 while reflecting the setting of the terminal device 10. Identify the reason for the error in the audio signal.

(B) Whether the voice speed of the reproduced voice signal is greater than the threshold value The processing unit 28 counts the number of characters of the text data as in the second embodiment, thereby determining the voice speed of the voice signal in a certain period. Deriving a value. The processing unit 28 determines whether the voice speed in the voice signal is larger than the threshold value by comparing the derived voice speed value with a threshold value stored in advance. If the value of the voice speed is greater than the threshold value, the processing unit 28 converts the portion of the text data that is larger than the threshold value into a hidden character. Further, the processing unit 28 may add a message for indicating that the voice speed is too high to the text data. Note that the processing unit 28 does not perform conversion on the text data if the value of the voice speed is equal to or less than the threshold value.

(C) Whether the volume level of the reproduced audio signal is smaller than the threshold value The processing unit 28 derives the value of the volume level of the audio signal in a certain period, as in the second embodiment. The processing unit 28 determines whether the volume level in the audio signal is smaller than the threshold value by comparing the derived volume level value with the threshold value. If the value of the sound volume level is smaller than the threshold value, the processing unit 28 converts each character of the text data into a hidden character. Further, the processing unit 28 may add a message for indicating that the volume level is too small to the text data. Note that the processing unit 28 does not perform conversion on the text data if the value of the volume level is equal to or greater than the threshold value.

  The transmission unit 40 inputs text data from the processing unit 28. The text data may include the reason for the error specified by the processing unit 28. The transmission unit 40 transmits the text data to the terminal device 10 on the transmission side. If the reason for the error is included, the obscure letter indicates the presence of an unrecognizable phoneme. Also, it is indicated that the voice speed is greater than the threshold value, or that the volume level is less than the threshold value. When the processing unit 28 does not execute the process for specifying the reason for the error, the text data does not include the reason for the error.

  In the above (3), the receiving unit 42 receives the text data from the terminal device 10 on the receiving side. The receiving unit 42 outputs the text data to the processing unit 28. The processing unit 28 inputs the text data from the receiving unit 42 and displays the text data on the display unit 26. When the reason for the error is not included in the text data, the display unit 26 displays the text data as in the first embodiment. The transmitter recognizes how the receiver is listening by checking the text data displayed on the display unit 26.

  On the other hand, the case where the reason for the error is included in the text data will be described below. FIGS. 10A to 10C show screens displayed on the display unit 26 according to the fourth embodiment of the present invention. FIG. 10A shows a case where the speaker speaks “Anzen”, but “A” is an unrecognizable phoneme. In this case, the listener may have heard “Kanzen”, for example. FIG. 10B shows a case where the value of the voice speed is larger than the threshold value. In this case, a part of the phonemes is indicated by the face-down character and a message is displayed. On the other hand, FIG. 10C shows a case where the value of the volume level is smaller than the threshold value. In this case, all phonemes are indicated by face-down characters and a message is displayed.

  The operation of the communication system 100 configured as above will be described. FIG. 11 is a sequence diagram illustrating a communication procedure performed by the communication system 100 according to the fourth embodiment of the present invention. When the first terminal apparatus 10a inputs voice (S110), the first terminal apparatus 10a generates a voice signal (S112). The first terminal device 10a transmits an audio signal (S114). The second terminal device 10b reproduces the audio signal (S116), and outputs the reproduced audio signal from the speaker 34 (S118). The second terminal device 10b executes voice recognition processing (S120) and specifies the reason for the error (S122). The second terminal device 10b generates text data and the reason for the error (S124). The second terminal device 10b transmits the text data and the reason for the error (S126). The first terminal device 10a displays the text data and the reason for the error (S128).

  FIG. 12 is a flowchart illustrating a specific procedure by the terminal device 10 according to the fourth embodiment of the present invention. When the equalizer setting is made in the setting unit 38 (Y in S150), the reproducing unit 32 performs an equalizer process on the audio signal (S152). If the equalizer is not set in the setting unit 38 (N in S150), step 152 is skipped. The reproducing unit 32 performs a voice recognition process (S154). If there is an unrecognizable phoneme (Y in S156), the processing unit 28 specifies the reason for the error (S158). If there is no unrecognizable phoneme (N in S156), step 158 is skipped.

  FIG. 13: is a flowchart which shows another specific procedure by the terminal device 10 which concerns on Example 4 of this invention. When the audio speed conversion setting is set in the setting unit 38 (Y in S200), the reproducing unit 32 adjusts the audio signal (S202). If the audio speed conversion setting is not set in the setting unit 38 (N in S200), step 202 is skipped. The reproducing unit 32 executes a voice recognition process (S204). If the voice speed is greater than the threshold value (Y in S206), the processing unit 28 specifies the reason for the error (S208). If the voice speed is not greater than the threshold value (N in S206), step 208 is skipped.

  FIG. 14 is a flowchart illustrating still another specific procedure by the terminal device 10 according to the fourth embodiment of the present invention. The reproducing unit 32 acquires the volume setting in the setting unit 38 (S250). The reproducing unit 32 adjusts the audio signal (S252). If the volume level is smaller than the threshold value (Y in S254), the processing unit 28 specifies the reason for the error (S256). If the volume level is not smaller than the threshold value (N in S254), step 256 is skipped.

  According to the present embodiment, since the voice recognition process is executed while reflecting the settings of the terminal device on the voice signal, the received voice signal can be converted into text while reflecting the settings of the terminal device. While reflecting the settings of the terminal device, the reason for the error in the reproduced audio signal is specified and notified, so even if the settings related to audio output are made in the terminal device, the reason why the sound is difficult to hear can be notified it can. Further, since the reason why it is difficult to hear the sound is notified, the sound signal can be transmitted while eliminating the reason. In addition, since the settings of the terminal device are reflected, the voice recognition process can be executed so as to be close to how the actual voice is heard.

  Further, since it is determined and notified whether there is a phoneme that cannot be recognized in the voice recognition processing, it is possible to notify that the cause is due to the way the speaker speaks and the communication environment. Further, since it is determined and notified whether or not the voice speed in the voice signal is larger than the threshold value, it can be notified that the cause is due to the way the speaker speaks. Further, since it is determined and notified whether the volume level in the audio signal is smaller than the threshold value, it is possible to notify that the cause is due to the way the speaker speaks.

(Example 5)
Next, Example 5 will be described. Example 5 corresponds to a combination of Example 4 and Example 3. The communication system and the terminal device according to the fifth embodiment are the same type as those shown in FIGS. Here, it demonstrates centering on the difference from before.

  In the above (2), the reproduction unit 32 receives the audio signal from the reception unit 42 and reproduces the audio signal. At that time, as in the fourth embodiment, the setting value made in the setting unit 38 is reflected. The processing unit 28 inputs the audio signal from the reproduction unit 32. The processing unit 28 performs speech recognition processing based on a standard speech recognition model for the speech signal. As a result, the audio signal is converted into text (hereinafter, the audio signal converted into text is referred to as “first text”).

  On the other hand, the reproduction unit 32 receives the audio signal from the reception unit 42 and reproduces the audio signal while the setting value set in the setting unit 38 is not used. The processing unit 28 inputs the audio signal from the reproduction unit 32. The processing unit 28 performs speech recognition processing based on a standard speech recognition model for the speech signal. As a result, the audio signal is converted into text (hereinafter, the audio signal converted into text is referred to as “second text”).

  The comparison unit 46 inputs the first text and the second text. The comparison unit 46 compares the first text and the second text. Here, as a comparison, the first text and the second text are arranged. The comparison unit 46 outputs text data in which the first text and the second text are arranged to the transmission unit 40. The transmission unit 40 inputs the text data from the processing unit 28. The transmission unit 40 transmits text data as a comparison result to the terminal device 10 on the transmission side. The subsequent processing is the same as before, and the description thereof is omitted here.

  According to the present embodiment, the result of the speech recognition process when the set value of the terminal device is used is compared with the result of the speech recognition process when the set value of the terminal device is not used. It is possible to identify whether phonemes that cannot be recognized are generated at each stage. In addition, there is an unrecognizable phoneme in the result of the speech recognition process when using the set value of the terminal device, and the result of the speech recognition process when the set value of the terminal device is not used cannot be recognized. If there is no phoneme, it can be recognized that it cannot be heard by the setting of the terminal device. In addition, when there is a phoneme that cannot be recognized in the result of the speech recognition process when the set value of the terminal device is used and the result of the speech recognition process when the set value of the terminal device is not used, Recognize that there is a cause in the speech or communication stage.

(Example 6)
Next, Example 6 will be described. Example 6 relates to the use of information on sound collected by a microphone when executing speech recognition processing in Example 4. The communication system and the terminal device according to the sixth embodiment are the same type as those in FIGS. Here, it demonstrates centering on the difference from before.

  The microphone 22 collects sounds around the terminal device 10, for example, noise. The microphone 22 converts the collected noise into an electric signal (hereinafter referred to as “noise signal”), and outputs the noise signal to the processing unit 28. As in the fourth embodiment, the processing unit 28 performs voice recognition processing on the voice signal. In particular, when the process (A) described above is executed in order to identify the reason for the error, the processing unit 28 reflects information on the sound collected by the microphone 22 in the voice recognition process. For example, the value to be compared with the correlation value is adjusted according to the magnitude of the noise signal. More specifically, the larger the noise signal, the smaller the value to be compared with the correlation value. The processing unit 28 determines unrecognizable phonemes as before. The subsequent processing is the same as before, and the description thereof is omitted here.

  According to the present embodiment, since the information on the sound around the terminal device collected by the microphone is also reflected, the voice recognition process can be executed so as to be close to how the actual voice is heard. In addition, since voice recognition processing is performed so as to be close to how the actual voice is heard, the accuracy of text conversion can be improved.

(Example 7)
Next, Example 7 will be described. Example 7 corresponds to a combination of Example 6 and Example 5. The communication system and the terminal device according to the seventh embodiment are the same type as those in FIGS. Here, it demonstrates centering on the difference from before.

  In the above (2), the reproduction unit 32 receives the audio signal from the reception unit 42 and reproduces the audio signal. At that time, as in the sixth embodiment, the setting value made in the setting unit 38 is reflected. The processing unit 28 inputs the audio signal from the reproduction unit 32. The processing unit 28 performs speech recognition processing based on a standard speech recognition model for the speech signal. Here, when the process (A) described above is executed in order to identify the reason for the error, the processing unit 28 uses the information of the sound collected by the microphone 22 in the voice recognition process as in the sixth embodiment. To reflect. As a result, the audio signal is converted into text (hereinafter, the audio signal converted into text is referred to as “first text”).

  On the other hand, the reproduction unit 32 receives the audio signal from the reception unit 42 and reproduces the audio signal while the setting value set in the setting unit 38 is not used. The processing unit 28 inputs the audio signal from the reproduction unit 32. The processing unit 28 performs speech recognition processing based on a standard speech recognition model for the speech signal. However, the processing unit 28 does not reflect the information of the sound collected by the microphone 22 in the voice recognition process. That is, the speech recognition process is executed while the sound information is not used. As a result, the audio signal is converted into text (hereinafter, the audio signal converted into text is referred to as “second text”).

  The comparison unit 46 inputs the first text and the second text. The comparison unit 46 compares the first text and the second text. Here, as a comparison, the first text and the second text are arranged. The comparison unit 46 outputs text data in which the first text and the second text are arranged to the transmission unit 40. The transmission unit 40 inputs the text data from the processing unit 28. The transmission unit 40 transmits text data as a comparison result to the terminal device 10 on the transmission side. The subsequent processing is the same as before, and the description thereof is omitted here.

  According to the present embodiment, the result of the voice recognition process when the set value of the terminal device is used and the collected sound information is used, the set value of the terminal device is not used, and the sound collection is performed. Compared with the result of the speech recognition processing when the information of the sound that has been used is not used, it is possible to specify at which stage the phoneme that cannot be recognized is generated. In addition, there is an unrecognizable phoneme in the result of the voice recognition process when using the setting value of the terminal device and using the collected sound information, without using the setting value of the terminal device, If there is no unrecognizable phoneme in the result of the speech recognition process when the collected sound information is not used, it can be recognized that the sound cannot be heard due to the setting of the terminal device and ambient noise. In addition, the result of the speech recognition process when the set value of the terminal device is used and the collected sound information is used, and the collected sound information without using the set value of the terminal device When there is an unrecognizable phoneme in the result of the speech recognition process when not in use, it can be recognized that there is a cause in the utterance or communication stage.

(Example 8)
Next, Example 8 will be described. The eighth embodiment corresponds to the case where the setting made in the terminal device on the receiving side is not reflected when the voice recognition process is executed in the sixth embodiment. The settings made on the terminal device on the receiving side are settings for turning the equalizer on / off, the volume level when being output from the speaker, and on / off of the speech speed conversion. On the other hand, in the eighth embodiment, as in the sixth embodiment, the sound information collected by the microphone is used when the voice recognition process is executed. The communication system and the terminal device according to the eighth embodiment are the same type as those in FIGS. Here, it demonstrates centering on the difference from before.

  In the above (2), the reproduction unit 32 receives the audio signal from the reception unit 42 and reproduces the audio signal. At this time, unlike the sixth embodiment, the setting value made in the setting unit 38 is not reflected. The microphone 22 collects sound around the terminal device 10, for example, noise, converts the collected noise into an electric signal (hereinafter referred to as “noise signal”), and outputs the noise signal to the processing unit 28. As in the sixth embodiment, the processing unit 28 performs a voice recognition process on the voice signal. In particular, when the process (A) described above is executed in order to identify the reason for the error, the processing unit 28 reflects information on the sound collected by the microphone 22 in the voice recognition process. The subsequent processing is the same as before, and the description thereof is omitted here.

  According to the present embodiment, since the information on the sound around the terminal device collected by the microphone is reflected, the voice recognition process can be executed so as to be close to how the actual voice is heard. In addition, since voice recognition processing is performed so as to be close to how the actual voice is heard, the accuracy of text conversion can be improved.

Example 9
Next, Example 9 will be described. Example 9 corresponds to a combination of Example 8 and Example 7. The communication system and the terminal device according to the ninth embodiment are the same type as those in FIGS. Here, it demonstrates centering on the difference from before.

  In the above (2), the processing unit 28 inputs the audio signal from the reproduction unit 32. The processing unit 28 performs speech recognition processing based on a standard speech recognition model for the speech signal. Here, when the process (A) described above is executed in order to identify the reason for the error, the processing unit 28 uses the information of the sound collected by the microphone 22 in the voice recognition process as in the eighth embodiment. To reflect. As a result, the audio signal is converted into text (hereinafter, the audio signal converted into text is referred to as “first text”).

  On the other hand, the processing unit 28 performs voice recognition processing based on a standard voice recognition model on the voice signal. However, the processing unit 28 does not reflect the information of the sound collected by the microphone 22 in the voice recognition process. That is, the speech recognition process is executed while the sound information is not used. As a result, the audio signal is converted into text (hereinafter, the audio signal converted into text is referred to as “second text”).

  The comparison unit 46 inputs the first text and the second text. The comparison unit 46 compares the first text and the second text. Here, as a comparison, the first text and the second text are arranged. The comparison unit 46 outputs text data in which the first text and the second text are arranged to the transmission unit 40. The transmission unit 40 inputs the text data from the processing unit 28. The transmission unit 40 transmits text data as a comparison result to the terminal device 10 on the transmission side. The subsequent processing is the same as before, and the description thereof is omitted here.

  According to the present embodiment, the result of the voice recognition process when the collected sound information is used is compared with the result of the voice recognition process when the collected sound information is not used. It is possible to specify at which stage unrecognizable phonemes are generated. In addition, there is an unrecognizable phoneme in the result of speech recognition processing when using collected sound information, and it is recognized as the result of speech recognition processing when not using collected sound information. When there is no impossible phoneme, it can be recognized that it cannot be heard due to ambient noise. In addition, there are unrecognizable phonemes in the results of the speech recognition process when the collected sound information is used and the results of the speech recognition process when the collected sound information is not used. In this case, it can be recognized that there is a cause in the speech or communication stage.

  In the above, this invention was demonstrated based on the Example. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and such modifications are also within the scope of the present invention. .

  According to the first to ninth embodiments, the communication system 100 uses business radio. However, the present invention is not limited to this. For example, a wireless communication system other than commercial radio may be used. According to the present embodiment, the degree of freedom of configuration can be improved.

  10 terminal devices, 12 base station devices, 14 networks, 20 buttons, 22 microphones, 24 operation units, 26 display units, 28 processing units, 30 communication units, 32 playback units, 34 speakers, 36 transmission units, 38 setting units, 40 transmitting unit, 42 receiving unit, 46 comparing unit, 100 communication system.

Claims (4)

A terminal device,
A receiving unit for receiving an audio signal from another terminal device;
A reproduction unit for reproducing the audio signal received by the reception unit;
A processing unit that performs voice recognition processing that reflects how a user who uses the terminal device hears the audio signal reproduced by the reproduction unit;
A transmission unit that transmits the result of the voice recognition processing in the processing unit to the other terminal device;
A terminal device comprising: The processing unit performs (1) a speech recognition process without reflecting how the user who uses the terminal apparatus is reflected, and (2) a result of the speech recognition process without reflecting the user's hearing, Compare the results of the voice recognition process that reflects how you hear
The terminal device according to claim 1, wherein the transmission unit transmits a comparison result in the processing unit to the other terminal device as a result of a speech recognition process.   The terminal device according to claim 1, wherein the voice recognition process in the processing unit reflects at least one of a volume level and a voice speed. A communication method in a terminal device,
Receiving an audio signal from another terminal device;
Playing back the received audio signal;
Executing a voice recognition process that reflects how the user using the terminal device hears the reproduced voice signal;
Transmitting the result of speech recognition processing to the other terminal device;
A communication method comprising:

Images