JP2020013169A - Terminal device, communication method, and communication program - Google Patents

Abstract

To provide a technique for acquiring contents of voice actually output on a call receiving party side.SOLUTION: A voice signal is transmitted to a terminal device on a receiving side. A result of voice recognition processing executed, in the terminal device on the receiving side, for a voice signal obtained by reproducing the received voice signal is received from the terminal device on the receiving side. The received result of the voice recognition processing is displayed on a display.SELECTED DRAWING: Figure 2

Description

The present invention relates to a communication technology, and more particularly, to a terminal device, a communication method, and a communication program that execute communication of an audio signal.

The voice recognition processing becomes unrecognizable when noise of the surrounding environment is added or the voice is low. Further, even if the voice recognition processing is repeatedly performed, recognition may not be performed. In particular, if the operator does not know the reason for the recognition failure, the recognition failure is likely to be repeated. To prevent this, the operator is notified of the reason for the inability to recognize (for example, Patent Document 1).
.

JP 2000-112497 A

The voice recognition process is generally performed on a voice signal that is voice data. On the other hand, the sound heard by the receiver is set to ON / OFF of an equalizer, a volume level when output from a speaker, and ON / OFF of speech speed conversion. Therefore, the voice heard by the receiver may be different from the voice signal. Further, even when the sound reproduced from the same sound signal is heard, the manner of hearing may differ depending on the listener. Therefore, it is desirable that the voice recognition process be performed according to the situation of the receiver.

The present invention has been made in view of such a situation, and an object of the present invention is to provide a technique for acquiring a speech recognition result made according to a situation of a receiver.

In order to solve the above-described problem, a terminal device according to an aspect of the present invention is a terminal device, a transmitting unit that transmits an audio signal to a receiving terminal device, and a receiving terminal device, A receiving unit that receives, from a terminal device serving as a receiving side, a result of a voice recognition process performed on a voice signal obtained by reproducing a received voice signal, and a processing unit that displays a result of the received voice recognition process on a display unit , Is provided.

In the terminal device according to another aspect of the present invention, the result of the voice recognition processing reflects how the user who uses the terminal device on the receiving side hears the reproduced audio signal in the terminal device on the receiving side. .

In a terminal device according to another aspect of the present invention, the receiving unit, in the terminal device on the receiving side,
(1) Speech recognition processing is performed without reflecting the way of hearing of the user using the terminal device on the receiving side, and (2) the result of the voice recognition processing without reflecting the way of hearing of the user and the user's hearing And a comparison result obtained by comparing the result of the voice recognition processing with the result of the comparison.

In a terminal device according to another aspect of the present invention, the receiving unit converts the result obtained by reflecting at least one of the volume level and the audio speed in the voice recognition processing of the terminal device on the receiving side to the receiving unit. Received from the terminal device.

Another embodiment of the present invention relates to a communication method. This method is a communication method in a terminal device,
Transmitting an audio signal to the terminal device on the receiving side, and executing a voice recognition process on the audio signal obtained by reproducing the received audio signal in the terminal device on the receiving side, and transmitting the result to the receiving side. And a step of acquiring a result of the voice recognition process and displaying the result on a display unit.

Another embodiment of the present invention relates to a communication program to be executed by a computer. This communication program is a process of transmitting an audio signal to a terminal device on the receiving side and a result of executing a voice recognition process on an audio signal obtained by reproducing the received audio signal on the terminal device on the receiving side. Is received from the terminal device on the receiving side, and a process of acquiring the result of the voice recognition process and displaying the result on the display unit is performed.

Note that any combination of the above-described components and any conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, and the like are also effective as embodiments of the present invention.

ADVANTAGE OF THE INVENTION According to this invention, the speech recognition result performed according to the receiver's situation can be acquired.

1 is a diagram illustrating a configuration of a communication system according to a first embodiment of the present invention. FIG. 2 is a diagram illustrating a configuration of a terminal device of FIG. 1. FIGS. 3A and 3B are diagrams showing screens displayed on the display unit in FIG. FIG. 2 is a sequence diagram showing a communication procedure by the communication system of FIG. 1. FIGS. 5A and 5B are diagrams illustrating screens displayed on the display unit according to the first embodiment of the present invention. FIG. 8 is a sequence diagram illustrating a communication procedure by the communication system according to the second embodiment of the present invention. FIG. 9 is a diagram illustrating a configuration of a terminal device according to a third embodiment of the present invention. FIGS. 8A and 8B are diagrams showing screens displayed on the display unit in FIG. 8 is a flowchart illustrating a comparison procedure by the comparison unit in FIG. 7. FIGS. 10A to 10C are diagrams illustrating screens displayed on the display unit according to the fourth embodiment of the present invention. FIG. 13 is a sequence diagram illustrating a communication procedure by the communication system according to the fourth embodiment of the present invention. 14 is a flowchart illustrating a specifying procedure by the terminal device according to the fourth embodiment of the present invention. 13 is a flowchart illustrating another specific procedure performed by the terminal device according to the fourth embodiment of the present invention. 15 is a flowchart illustrating still another specific procedure by the terminal device according to the fourth embodiment of the present invention.

(Example 1)
Before describing the present invention in detail, an overview will first be given. In the first embodiment of the present invention, the PTT (P
The present invention relates to a terminal device that executes voice communication based on “to to talk”. The terminal device includes a button, and when the user presses down the button, transmission starts,
When the button is released, the transmission ends. On the other hand, if the button is not pressed, the user will only hear the message. In such a PTT, the sender simply completes the action of speaking and transmitting, and the only way to transmit it is to rely on the response of the listener. Even if the sender assumes that what he / she spoke is being transmitted correctly to the receiver, the voice may not be transmitted to the receiver as intended due to poor communication conditions, a lot of noise, or too fast. is there.

Therefore, the receiving-side terminal device performs a voice recognition process, converts the received voice signal into text, and transmits text-formatted data (hereinafter, referred to as “text data”) to the transmitting-side terminal device. The terminal device on the transmitting side displays the text data, and the sender confirms the text data to confirm whether the intended sound is output. However, since there is an individual difference in how the listener hears, even the same voice may sound different. Therefore, the contents of the text data may be different from the contents actually recognized by the receiver.

In order to cope with this, the terminal device according to the present embodiment, particularly, the terminal device on the receiving side executes the voice recognition process using the voice recognition model of the user who uses the terminal device, that is, the listener. Therefore, the content of the text data generated in the terminal device reflects how the listener is heard.

FIG. 1 shows a configuration of a communication system 100 according to the first embodiment of the present invention. Communication system 10
0 includes a first terminal device 10a, a second terminal device 10b, which is collectively referred to as a terminal device 10, a first base station device 12a, a second base station device 12b, which is collectively referred to as a base station device 12, and a network 14. Here, the communication system 100 corresponds to, for example, business wireless.

The terminal device 10 is a device capable of executing communication by business wireless communication. Since a known technique may be used for the commercial radio, the description is omitted here. Here, the first terminal device 10a corresponds to the transmitting side of the commercial wireless communication, and the second terminal device 10b corresponds to the receiving side of the commercial wireless communication. Therefore, the first terminal device 10a is used for the transmitter, and the second terminal device 10b is used for the receiver. Note that the relationship between the first terminal device 10a and the second terminal device 10b may be reversed, and the number of terminal devices 10 is not limited to “2”.

The base station device 12 corresponds to a business radio system, and is connected to the terminal device 10 at one end by business radio and connected to the base station device 12 at the other end.
The first base station device 12a and the second base station device 12b are installed at different positions. In addition,
In business wireless, a group can be formed by a plurality of terminal devices 10. The base station apparatus 12 may assign an uplink channel and a downlink channel to a group.
In such a situation, one terminal device 10 in the group transmits a signal on the uplink channel, and another terminal device 10 in the group receives the signal on the downlink channel.

The network 14 connects the first base station device 12a and the second base station device 12b. By such a connection, the first terminal apparatus 10a and the second terminal apparatus 10b are connected to the first base station apparatus 1
2a, the network 14, and the second base station device 12b become communicable. here,
It is assumed that the voice communication is PTT.

FIG. 2 shows a configuration of the terminal device 10. The terminal device 10 includes a button 20, a microphone 22, an operation unit 24, a display unit 26, a processing unit 28, a communication unit 30, a reproduction unit 32, and a speaker 34. The processing unit 28 includes a transmitting unit 36 and a setting unit 38, and the communication unit 30 includes a transmitting unit 40, a receiving unit 4
2 inclusive. The terminal device 10 corresponds to both the terminal device 10 on the transmitting side and the terminal device 10 on the receiving side. Here, in order to clarify the explanation, (1) the transmitting side, (2) the receiving side, and (3) the transmitting side will be described in the order of processing.

(1) Transmission side Here, processing in the terminal device 10 on the transmission side will be described. The button 20 corresponds to a PTT button, and is pressed down by a user when transmitting voice by PTT. Also,
Button 20 is kept depressed while transmitting audio. Pressing down of the button 20 corresponds to receiving an instruction to transmit an audio signal. The button 20 continues to output an instruction to the transmitter 36 while being depressed. Microphone 2
2 collects sound around the terminal device 10. The microphone 22 converts the collected voice into an electric signal (
(Hereinafter, referred to as an “audio signal”), and outputs the audio signal to the transmitting section 36.

The transmitting section 36 inputs a voice signal from the microphone 22 when receiving an instruction from the button 20. The transmitting unit 36 converts the audio signal from an analog signal to a digital signal, and outputs the audio signal converted into a digital signal (hereinafter also referred to as an “audio signal”) to the transmitting unit 40. On the other hand, when the instruction from button 20 has not been received, transmitting section 36 does not execute such processing. The transmitting unit 40 receives the audio signal from the transmitting unit 36 and transmits the audio signal to the terminal device 10 on the receiving side. For transmitting the audio signal, the transmitting unit 40 performs encoding,
Performs modulation, frequency conversion, amplification, etc.

(2) Receiving side Here, the processing in the terminal device 10 on the receiving side following (1) will be described. The receiving unit 42 receives an audio signal from the terminal device 10 on the transmitting side. The receiving unit 42 performs amplification, frequency conversion,
Perform demodulation, decoding, etc. The receiving unit 42 outputs a result of the processing (hereinafter also referred to as an “audio signal”) to the reproducing unit 32. The reproducing unit 32 receives the audio signal from the receiving unit 42 and reproduces the audio signal. A known technique may be used to reproduce the audio signal, and a description thereof will not be repeated. The reproduction unit 32 outputs the reproduced audio signal to the speaker 34 and the processing unit 28. The speaker 34 converts an audio signal, which is an electric signal, into audio and outputs the audio.

The processing unit 28 receives the audio signal from the reproduction unit 32. On the other hand, the voice recognition model of the user who uses the terminal device 10, that is, the specified listener is set in the setting unit 38. In the voice recognition model, for example, a waveform of a voice signal corresponding to the phoneme “A” is stored. The waveform of such an audio signal is stored for each phoneme. In particular, since the stored phonemes and waveforms are associated with each other so that the specified listener who has heard the voice recognizes the voice, the relationship can be said to be a voice recognition model of the listener.

The processing unit 28 performs a voice recognition process on the voice signal while using the voice recognition model of the receiver set in the setting unit 38. More specifically, the processing unit 28 selects a waveform closest to the waveform of the input voice signal from the voice recognition model, and specifies a sound corresponding to the selected waveform. The voice signal is converted into text by the voice recognition processing. in this way,
The processing unit 28 performs a voice recognition process based on a voice recognition model of the user using the terminal device 10, that is, a voice recognition process that reflects how the user hears the voice signal.

The receiver sets the voice recognition model of the receiver in the setting unit 38 while operating the operation unit 24. For example, the setting unit 38 stores a waveform of the test audio signal in advance, and reproduces the waveform from the speaker 34 to reproduce the waveform from the speaker 34. The listener inputs the sound recognized by using the operation unit 24 while listening to the sound from the speaker 34. The setting unit 38 sets the voice recognition model of the listener based on the correspondence between the waveform of the test voice signal and the input sound.

The transmitting unit 40 inputs text data by the voice recognition process from the processing unit 28 as a result of the voice recognition process in the processing unit 28. The transmitting unit 40 transmits the text data to the terminal device 10 on the transmitting side. The transmission of the text data in the transmission unit 40 may be performed by itself, or may be performed together with the transmission of the audio signal.

(3) Transmission side Here, the processing in the terminal device 10 on the transmission side following (2) will be described. The receiving unit 42 receives text data from the terminal device 10 on the receiving side. The receiving unit 42 outputs the text data to the processing unit 28. The processing unit 28 inputs the text data from the receiving unit 42,
The text data is displayed on the display unit 26. The sender recognizes how the receiver is listening by checking the text data displayed on the display unit 26. FIG. 3 (
a)-(b) show screens displayed on the display unit 26. FIG. 3A shows a case where the sender speaks “Anzen” and the receiver hears “Anzen”. In this case, the content spoken by the sender and the content heard by the receiver match. On the other hand, FIG. 3B shows a case where the sender speaks “Anzen” and the listener hears “Kanzen”. In this case, the content spoken by the sender and the content heard by the receiver differ.

This configuration is, in terms of hardware, the CPU, memory, and other L of any computer.
It can be realized by SI, and is realized by software or the like loaded into a memory in software. Here, functional blocks realized by their cooperation are illustrated. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The operation of the communication system 100 having the above configuration will be described. FIG. 4 shows a communication system 100.
FIG. 5 is a sequence diagram showing a communication procedure according to the first embodiment. The first terminal device 10a inputs a voice (S
10), an audio signal is generated (S12). The first terminal device 10a transmits an audio signal (S14). The second terminal device 10b reproduces the audio signal (S16), and outputs the reproduced audio signal from the speaker 34 (S18). The second terminal device 10b performs a voice recognition process using the voice recognition model of the user (S20), and generates text data (S22). The second terminal device 10b transmits the text data (S24). The first terminal device 10a displays the text data (S26).

According to the present embodiment, a process based on the voice recognition model of the user using the terminal device is performed on the voice signal, so that the received voice signal can be converted to text while reflecting how the user hears. In addition, since the received voice signal is converted into text while reflecting how the user hears, accurate information can be notified to the sender. In addition, since the voice recognition process is performed on the voice signal while using the voice recognition model of the user, it is possible to identify a mistake in the listener's hearing due to the pronunciation of the sender. In addition, since the voice recognition process is performed on the voice signal while using the voice recognition model of the user, it is possible to identify a wrong listening of the listener in the wireless section.

(Example 2)
Next, a second embodiment will be described. As in the first embodiment, the second embodiment of the present invention also relates to a terminal device that executes voice communication using PTT and converts a voice signal into a text using a voice recognition model of a receiver. The speech recognition model in the first embodiment is configured for a waveform of a speech signal recognized by a listener. On the other hand, the voice recognition model in the second embodiment is configured for a voice speed recognizable by the listener and a volume level recognizable by the listener.
The communication system and the terminal device according to the second embodiment are of the same type as those in FIGS. Here, the description will focus on the differences from the past.

In the above (2), the processing unit 28 inputs an audio signal from the reproduction unit 32. Also,
The processing unit 28 converts the audio signal to text by executing a voice recognition process on the audio signal. On the other hand, the voice recognition model of the user who uses the terminal device 10, that is, the listener, is set in the setting unit 38. In the voice recognition model, for example, at least one of a voice speed value recognizable by the listener and a volume level value recognizable by the listener is stored.

The processing unit 28 derives the value of the audio speed of the audio signal in a certain period by counting the number of characters of the text data. The processing unit 28 compares the derived voice speed value with the voice speed value stored in the setting unit 38 to determine whether the voice signal is lower than the voice speed that can be recognized by the receiver. Execute the process. If the value of the derived voice speed is higher than the value of the voice speed recognizable by the listener, the processing unit 28 extracts the characters of the portion of the text data that are higher than the voice speed recognizable by the listener. Convert to face down. Note that if the derived voice speed value is equal to or less than the voice speed value that can be recognized by the listener,
Do not convert textual data.

Further, the processing unit 28 may derive the value of the volume level of the audio signal in a certain period.
The processing unit 28 compares the derived volume level value with the value of the volume level stored in the setting unit 38 to determine whether or not the audio signal is higher than the volume level that can be recognized by the listener. Execute the process. If the derived sound volume level value is smaller than the sound volume level value that can be recognized by the listener, the processing unit 28 converts each character of the text data into a hidden character. If the derived volume level value is equal to or greater than the volume level that can be recognized by the listener, the processing unit 28 does not execute the conversion on the text data. As described above, at least one of the volume level and the voice speed is reflected in the voice recognition processing in the processing unit 28. Here, the voice recognition model of the receiver is set in the setting unit 38 by operating the operation unit 24. The content to be set is at least one of a voice speed value recognizable by the listener and a volume level value recognizable by the listener.

The transmission unit 40 inputs the text data from the processing unit 28. The transmitting unit 40 transmits the text data to the terminal device 10 on the transmitting side. As described above, if the voice speed is greater than the value that the listener can recognize, or if the volume level is smaller than the volume level that the listener can recognize,
At least some characters of the text data have been converted to hidden characters. This means that the transmitting unit 40 transmits a result of determination as to whether or not the voice speed is lower than the voice rate recognizable by the listener, or transmits a result of determination as to whether or not the volume level is higher than the volume recognizable by the listener. Equivalent to.

In (3) described above, the receiving unit 42 receives text data from the terminal device 10 on the receiving side, and the processing unit 28 displays the text data on the display unit 26. 5A and 5B show screens displayed on the display unit 26. FIG. FIG. 5A shows a case where the value of the voice speed of the voice signal reproduced in the terminal device 10 on the receiving side is higher than the value of the voice speed that can be recognized by the listener. In this case, some characters are indicated by face-down characters. On the other hand, FIG. 5B shows a case where the value of the volume level of the audio signal reproduced in the terminal device 10 on the receiving side is smaller than the value of the volume level that can be recognized by the listener. In this case, all characters are indicated by face-down characters.

The operation of the communication system 100 having the above configuration will be described. FIG. 6 is a sequence diagram illustrating a communication procedure by the communication system 100 according to the second embodiment of the present invention. First terminal device 10a
Receives a voice (S50) and generates a voice signal (S52). First terminal device 10a
Transmits an audio signal (S54). The second terminal device 10b reproduces the audio signal (S56).
), And outputs the reproduced audio signal from the speaker 34 (S58). The second terminal device 10b
By executing the voice recognition process (S60) and performing the evaluation based on the voice speed and the volume level (S62), text data is generated (S64). Second terminal device 10b
Transmits the text data (S66). The first terminal device 10a displays the text data (S68).

According to the present embodiment, the process of determining whether or not the voice signal is lower than the voice speed recognizable by the user is performed. Therefore, it is possible to determine whether the voice signal is difficult to hear due to the voice speed. In addition, it is possible to reflect the fact that it is difficult to hear due to the voice speed in the text. In addition, since the audio signal is subjected to a determination process of determining whether or not the volume level is higher than a volume level recognizable by the user, it is possible to determine whether the audio signal is difficult to hear due to the volume level. In addition, the difficulty in hearing due to the volume level can be reflected in the text.

(Example 3)
Next, a third embodiment will be described. The third embodiment of the present invention also relates to a terminal device that executes voice communication based on PTT and converts a voice signal into a text using a voice recognition model of a receiver, as in the past. In the third embodiment, not only the audio signal is converted into text, but also information that allows the listener to estimate the situation in which the listener is listening to the audio is notified to the terminal device on the transmission side. The communication system according to the third embodiment is of the same type as that of FIG. Here, the description will focus on the differences from the past.

FIG. 7 illustrates a configuration of the terminal device 10 according to the third embodiment of the present invention. The processing unit 28 in the terminal device 10 further includes a comparing unit 46 as compared with FIG. In the above (2), the processing unit 28 inputs an audio signal from the reproduction unit 32. The processing unit 28 performs a voice recognition process based on a voice recognition model of the user using the terminal device 10, that is, a voice recognition process that reflects how the user hears the voice signal, as in the first embodiment. . As a result, the audio signal is converted to text (hereinafter, the converted audio signal is referred to as “first text”).

At this time, the processing unit 28 may determine whether there is a phoneme that cannot be recognized in the voice recognition processing. For example, when the correlation value between the waveform of one phoneme of the input audio signal and the waveform closest to the waveform of the one phoneme is smaller than a predetermined value, the one phoneme is determined to be an unrecognizable phoneme. You. The processing unit 28 converts an unrecognizable phoneme into a hidden character in the first text. Note that the message may be converted to another predetermined character instead of the hidden character, and a message that “there is an unrecognizable phoneme” may be added to the first text.

On the other hand, the processing unit 28 also stores a speech recognition model that is not specified by the user who uses the terminal device 10, that is, a standard speech recognition model. In the standard voice recognition model, for example, a standard waveform of a voice signal corresponding to the sound “A” is stored. The processing unit 28 also performs a voice recognition process based on a standard voice recognition model, that is, a voice recognition process that does not reflect how the user hears the voice signal. Also in this case, the audio signal is converted to text (hereinafter, the converted audio signal is referred to as “second text”). The processing unit 28 may convert an unrecognizable phoneme into a hidden character or the like in the second text as in the case of the first text.

The comparison unit 46 inputs the first text and the second text. The comparing unit 46 compares the first text with the second text. Here, as a comparison, the first text and the second text are arranged. The comparing unit 46 outputs to the transmitting unit 40 text data in which the first text and the second text are arranged. The transmission unit 40 inputs the text data from the processing unit 28. The transmitting unit 40 transmits the text data as the comparison result to the terminal device 10 on the transmitting side.

In (3) described above, the receiving unit 42 receives text data from the terminal device 10 on the receiving side, and the processing unit 28 displays the text data on the display unit 26. FIGS. 8A and 8B show screens displayed on the display unit 26. At the top of the screen, a first text is shown as "receiver speech recognition", and at the bottom of the screen, a second text is shown as "standard speech recognition". FIG. 8A shows a case where there is an unrecognizable phoneme in the first text even though there is no unrecognizable phoneme in the second text. This means that if a speech signal corresponding to the voice uttered by the speaker is subjected to speech recognition processing using a standard speech recognition model, all recognition is performed, but if speech recognition processing is performed using the speech recognition model of the receiver, recognition is not possible. This corresponds to the existence of possible phonemes. That is, it can be said that the voice is not recognized depending on how the listener hears.

FIG. 8B shows a case where there is an unrecognizable phoneme in both the first text and the second text. This corresponds to the presence of an unrecognizable phoneme when the voice signal corresponding to the voice uttered by the speaker is subjected to voice recognition processing by either the standard voice recognition model or the voice recognition model of the listener. . In this case, for example, the quality of the radio section between the first terminal apparatus 10a and the first base station apparatus 12a or the radio section between the second terminal apparatus 10b and the second base station apparatus 12b may be degraded. Presumed.

The operation of the communication system 100 having the above configuration will be described. FIG. 9 is a flowchart illustrating a comparison procedure performed by the comparison unit 46. The comparing unit 46 acquires the result of the voice recognition processing based on the voice recognition model of the user (S80). On the other hand, the comparing unit 46 acquires the result of the voice recognition processing based on the standard voice recognition model (S82). The comparing unit 46 arranges the results of both voice recognition processes as a comparison (S84).

According to the present embodiment, the result of the speech recognition processing based on the user's speech recognition model is compared with the result of the speech recognition processing based on the standard speech recognition model. Can be specified. In addition, if there is an unrecognizable phoneme in the result of the speech recognition process based on the user's speech recognition model and there is no unrecognizable phoneme in the result of the speech recognition process based on the standard speech recognition model, a specific It is possible to recognize that only the user cannot hear. Also, if there is an unrecognizable phoneme in the result of the voice recognition process based on the user's voice recognition model and the result of the voice recognition process based on the standard voice recognition model, the cause is in the stage of speech or communication. I can recognize that.

(Example 4)
Next, a fourth embodiment will be described. Embodiment 4 of the present invention also relates to a terminal device that executes voice communication using PTT and converts a received voice signal into text, as in the past. In the terminal device on the receiving side, there are cases where the setting of ON / OFF of the equalizer, the volume level when output from the speaker, and the ON / OFF of the speech speed conversion are performed. At that time, a sound processed according to such a setting is output from the speaker. That is, such processing may cause a difference between the sound actually output from the speaker and the content of the text data.

In order to cope with this, the terminal device according to the present embodiment, especially the terminal device on the receiving side, executes the voice recognition processing according to the setting made for the terminal device. Therefore, the content of the text data generated in the terminal device reflects a process according to the setting. The communication system and the terminal device according to the fourth embodiment are of the same type as those in FIGS. Here, the description will focus on the differences from the past.

In the above (2), various settings are made in the setting unit 38 of the terminal device 10 with respect to audio output. With this setting, a setting value to be used when the receiving unit 42 reproduces the received audio signal is registered. One of the settings made in the setting unit 38 is ON / OFF of the equalizer. The equalizer is a process for changing the frequency characteristics of the audio signal. When the equalizer is turned on, it is possible to emphasize or reduce a specific frequency band (a harmonic component, a harmonic component, or a noise component) of the audio signal. Another one of the settings made in the setting unit 38 is ON / OFF of the audio speed conversion. The audio speed conversion is a process for increasing or decreasing the reproduction speed of audio. further,
Yet another one of the settings made in the setting unit 38 is adjustment of the volume level.
The volume level is a volume at which sound is output from the speaker 34. These settings are made by operating the operation unit 24. Further, the setting unit 38 does not need to make all of these settings, and it is sufficient that at least one setting is made.

The reproducing unit 32 receives the audio signal from the receiving unit 42 and reproduces the audio signal. At that time, the set value set by the setting unit 38 is reflected. For example, when the equalizer is turned on in the setting unit 38, the reproducing unit 32 reproduces the audio signal while performing the equalizer processing. On the other hand, when the equalizer is turned off in the setting unit 38, the reproducing unit 32 reproduces the audio signal without performing the equalizer processing.

When the audio speed conversion is turned on in the setting unit 38, the reproducing unit 32 reproduces the audio signal while converting the audio speed. Note that the audio speed is set to be twice, three times, or one half times. On the other hand, when the audio speed conversion is turned off in the setting unit 38,
The reproduction unit 32 reproduces an audio signal without converting the audio speed. Further, at the volume level set by the setting unit 38, the reproduction unit 32 adjusts the volume level when reproducing the audio signal. It should be noted that a known technique may be used for the equalizer processing, the audio speed conversion, the adjustment of the volume level, and the reproduction of the audio signal. As described above, it is not necessary for the setting unit 38 to make all of these settings, and it is sufficient that at least one setting is made. Therefore, the reproducing unit 32 may use at least one setting. The reproduction unit 32 outputs the reproduced audio signal to the speaker 34 and the processing unit 28. The speaker 34 converts an audio signal, which is an electric signal, into audio and outputs the audio.

The processing unit 28 receives the audio signal from the reproduction unit 32. The processing unit 28 performs a voice recognition process on the voice signal based on the standard voice recognition model described above. The voice signal is converted into text by the voice recognition processing. Further, when there is a phoneme that cannot be recognized in the voice recognition processing, the processing unit 28 may specify the reason (hereinafter, referred to as “error reason”). Here, the reason for the error is (A) whether there is a phoneme that cannot be recognized in the voice recognition processing, (B) whether the voice speed of the reproduced voice signal is higher than the threshold, or (C)
) It is specified whether the volume level of the reproduced audio signal is lower than the threshold value. (B)
The threshold values in (C) and (C) may be different values. Hereinafter, these processes will be described in order.

(A) Whether Unrecognizable Phoneme Exists in Speech Recognition Processing The processing unit 28 determines whether there is an unrecognizable phoneme in the speech recognition processing. For example, when the correlation value between the waveform of one phoneme of the input audio signal and the waveform closest to the waveform of the one phoneme is smaller than a predetermined value, the one phoneme is determined to be an unrecognizable phoneme. You. The processing unit 28 converts unrecognizable phonemes into hidden characters in the text data. It should be noted that the message may be converted to another predetermined character instead of the hidden character, and a message indicating “there is an unrecognizable phoneme” may be added to the text data. That is, the processing unit 2
8 determines whether there is an unrecognizable phoneme in the voice recognition process performed on the audio signal reproduced by the reproduction unit 32 while reflecting the setting of the terminal device 10, and Identify the reason for the error.

(B) Whether the audio speed of the reproduced audio signal is higher than the threshold value As in the second embodiment, the processing unit 28 counts the number of characters of the text data to determine the audio speed of the audio signal in a certain period. Derive a value. The processing unit 28 performs a process of determining whether the audio speed of the audio signal is higher than the threshold value by comparing the derived audio speed value with a threshold value stored in advance. If the value of the voice speed is greater than the threshold value, the processing unit 28 converts the characters of the text data, which are greater than the threshold value, into hidden characters. Further, the processing unit 28 may add a message indicating that the audio speed is too high to the text data. If the value of the voice speed is equal to or less than the threshold value, the processing unit 28 does not execute the conversion on the text data.

(C) Whether the Volume Level of the Reproduced Audio Signal is Lower than the Threshold The processing unit 28 derives the value of the volume level of the audio signal in a certain period, as in the second embodiment. The processing unit 28 compares the derived volume level value with the threshold value,
A determination process is performed to determine whether the volume level of the audio signal is smaller than the threshold. The processing unit 28
If the value of the sound volume level is smaller than the threshold value, each character of the text data is converted into a hidden character. Further, the processing unit 28 may add a message indicating that the volume level is too low to the text data. If the value of the volume level is equal to or larger than the threshold, the processing unit 28 does not execute the conversion on the text data.

The transmitting unit 40 inputs text data from the processing unit 28. The text data may include the reason for the error specified by the processing unit 28. The transmitting unit 40 transmits the text data to the terminal device 10 on the transmitting side. If the reason for the error is included, the hidden character indicates the presence of an unrecognizable phoneme. Also, it is indicated that the voice speed is higher than the threshold value or that the volume level is lower than the threshold value. If the processing unit 28 does not execute the process for specifying the reason for the error, the text data does not include the reason for the error.

In the above (3), the receiving unit 42 receives text data from the terminal device 10 on the receiving side. The receiving unit 42 outputs the text data to the processing unit 28. The processing unit 28 receives the text data from the receiving unit 42 and displays the text data on the display unit 26. When the reason for the error is not included in the text data, the display unit 26 displays the text data as in the first embodiment. The sender recognizes how the receiver is listening by checking the text data displayed on the display unit 26.

On the other hand, a case where the text data includes an error reason will be described below. FIG.
0 (a) to (c) show screens displayed on the display unit 26 according to the fourth embodiment of the present invention. FIG.
0 (a) indicates a case where the sender speaks “Anzen” but “A” is an unrecognizable phoneme. In this case, the listener may have heard, for example, “Kanzen”. FIG. 10B shows a case where the value of the voice speed is larger than the threshold value. in this case,
Some phonemes are indicated by face-down characters and a message is displayed. On the other hand, FIG.
(C) shows a case where the value of the volume level is smaller than the threshold value. In this case, all the phonemes are indicated by the hidden characters and a message is displayed.

The operation of the communication system 100 having the above configuration will be described. FIG. 11 shows Embodiment 4 of the present invention.
Is a sequence diagram showing a communication procedure by the communication system 100 according to the first embodiment. First terminal device 10
As for a, when a voice is input (S110), a voice signal is generated (S112). The first terminal device 10a transmits an audio signal (S114). The second terminal device 10b reproduces the audio signal (S116), and outputs the reproduced audio signal from the speaker 34 (S118). The second terminal device 10b executes a voice recognition process (S120), and specifies the reason for the error (S122).
. Further, the second terminal device 10b generates the text data and the reason for the error (S124).
. The second terminal device 10b transmits the text data and the reason for the error (S126). First
The terminal device 10a displays the text data and the reason for the error (S128).

FIG. 12 is a flowchart illustrating a specifying procedure by the terminal device 10 according to the fourth embodiment of the present invention. When the equalizer setting is made in the setting unit 38 (Y in S150), the reproduction unit 32
Performs an equalizer process on the audio signal (S152). If the equalizer setting has not been made in the setting unit 38 (N in S150), step 152 is skipped. The playback unit 32 performs a voice recognition process (S154). If there is an unrecognizable phoneme (Y in S156), the processing unit 28 specifies the reason for the error (S158). If there is no unrecognizable phoneme (N in S156), step 158 is skipped.

FIG. 13 is a flowchart illustrating another specific procedure by the terminal device 10 according to the fourth embodiment of the present invention. When the audio speed conversion setting is made in the setting unit 38 (Y in S200), the reproducing unit 32 adjusts the audio signal (S202). If the audio speed conversion setting has not been made in the setting unit 38 (N in S200), step 202 is skipped. The reproduction unit 32 performs a voice recognition process (S204). If the voice speed is higher than the threshold (Y in S206)
The processing unit 28 specifies the reason for the error (S208). If the voice speed is not greater than the threshold (N in S206), step 208 is skipped.

FIG. 14 is a flowchart illustrating still another specific procedure by the terminal device 10 according to the fourth embodiment of the present invention. The reproduction unit 32 acquires the volume setting in the setting unit 38 (S250).
. The playback unit 32 adjusts the audio signal (S252). If the volume level is lower than the threshold value (Y in S254), the processing unit 28 specifies the reason for the error (S256). If the volume level is not lower than the threshold value (N of S254), step 256 is skipped.

According to the present embodiment, the voice recognition processing is executed while reflecting the settings of the terminal device on the voice signal, so that the received voice signal can be converted to text while reflecting the settings of the terminal device. Since the cause of the error in the reproduced audio signal is specified and notified while reflecting the setting of the terminal device, it is possible to inform the reason why the sound is difficult to hear even when the setting regarding the audio output is made in the terminal device. it can. Further, since the user is notified of the reason why the sound is hard to hear, the sound signal can be transmitted while eliminating the reason. In addition, since the setting of the terminal device is reflected, it is possible to execute the voice recognition process so as to be closer to how the actual voice is heard.

In addition, since it is determined whether there is an unrecognizable phoneme in the voice recognition process and notified,
It is possible to notify that the cause is due to the way of speaking of the sender and the communication environment. In addition, since the determination is made as to whether or not the voice speed of the voice signal is greater than the threshold value, it is possible to notify that the cause is the manner of speaking by the sender. Further, since the determination is made as to whether or not the volume level of the audio signal is smaller than the threshold value, it is possible to notify that the cause is the manner of speaking by the sender.

(Example 5)
Next, a fifth embodiment will be described. Example 5 corresponds to a combination of Example 4 and Example 3.
The communication system and the terminal device according to the fifth embodiment are of the same type as those in FIGS. Here, the description will focus on the differences from the past.

In the above (2), the reproducing unit 32 receives the audio signal from the receiving unit 42 and reproduces the audio signal. At that time, as in the case of the fourth embodiment, the setting value set by the setting unit 38 is reflected. The processing unit 28 receives the audio signal from the reproduction unit 32. The processing unit 28 performs a voice recognition process on the voice signal based on a standard voice recognition model. As a result, the audio signal is converted to text (hereinafter, the converted audio signal is referred to as “first text”).

On the other hand, the reproducing unit 32 receives the audio signal from the receiving unit 42 and reproduces the audio signal without using the set value set in the setting unit 38. The processing unit 28 receives the audio signal from the reproduction unit 32. The processing unit 28 performs a voice recognition process on the voice signal based on a standard voice recognition model. As a result, the audio signal is converted to text (hereinafter, the converted audio signal is referred to as “second text”).

The comparison unit 46 inputs the first text and the second text. The comparing unit 46 compares the first text with the second text. Here, as a comparison, the first text and the second text are arranged. The comparing unit 46 outputs to the transmitting unit 40 text data in which the first text and the second text are arranged. The transmission unit 40 inputs the text data from the processing unit 28. The transmitting unit 40 transmits the text data as the comparison result to the terminal device 10 on the transmitting side. Subsequent processing is the same as that described above, and a description thereof will not be repeated.

According to the present embodiment, the result of the voice recognition process when the setting value of the terminal device is used is compared with the result of the voice recognition process when the setting value of the terminal device is not used. It is possible to specify whether an unrecognizable phoneme occurs at the stage. In addition, there is an unrecognizable phoneme in the result of the voice recognition processing when the setting value of the terminal device is used, and the phoneme cannot be recognized in the result of the voice recognition processing when the setting value of the terminal device is not used. If there is no such phoneme, it can be recognized that it cannot be heard due to the setting of the terminal device. Also, if there is an unrecognizable phoneme in the result of the voice recognition process when using the setting value of the terminal device and the result of the voice recognition process when not using the setting value of the terminal device, It is possible to recognize that there is a cause in the stage of speech or communication.

(Example 6)
Next, a sixth embodiment will be described. The sixth embodiment relates to using information of sound collected by a microphone when performing a voice recognition process in the fourth embodiment. The communication system and the terminal device according to the sixth embodiment are of the same type as those in FIGS. Here, the description will focus on the differences from the past.

The microphone 22 collects sounds around the terminal device 10, for example, noise. The microphone 22 converts the collected noise into an electric signal (hereinafter, referred to as a “noise signal”) and outputs the noise signal to the processing unit 28. The processing unit 28 performs a voice recognition process on the voice signal as in the fourth embodiment. In particular, when the above-described processing (A) is performed to identify the reason for the error, the processing unit 28
Reflects the information of the sound collected by the microphone 22 in the voice recognition processing. For example, the value to be compared with the correlation value is adjusted according to the magnitude of the noise signal. Specifically,
As the noise signal increases, the value to be compared with the correlation value decreases. The processing unit 28 determines an unrecognizable phoneme as before. Subsequent processing is the same as that described above, and a description thereof will not be repeated.

According to the present embodiment, the information of the sound around the terminal device collected by the microphone is also reflected, so that the voice recognition processing can be executed so as to be closer to how the actual voice is heard. In addition, since the speech recognition processing is performed so as to be close to how the actual speech is heard, the accuracy of the text conversion can be improved.

(Example 7)
Next, a seventh embodiment will be described. Example 7 corresponds to a combination of Example 6 and Example 5.
The communication system and the terminal device according to the seventh embodiment are of the same type as those in FIGS. Here, the description will focus on the differences from the past.

In the above (2), the reproducing unit 32 receives the audio signal from the receiving unit 42 and reproduces the audio signal. At this time, as in the case of the sixth embodiment, the setting value set by the setting unit 38 is reflected. The processing unit 28 receives the audio signal from the reproduction unit 32. The processing unit 28 performs a voice recognition process on the voice signal based on a standard voice recognition model. Here, when performing the above-described processing (A) to identify the reason for the error, the processing unit 28 performs processing similar to that of the sixth embodiment to process the information of the sound collected by the microphone 22 in the voice recognition processing. To reflect. As a result, the audio signal is converted to text (hereinafter, the converted audio signal is referred to as “first text”).

On the other hand, the reproducing unit 32 receives the audio signal from the receiving unit 42 and reproduces the audio signal without using the set value set in the setting unit 38. The processing unit 28 receives the audio signal from the reproduction unit 32. The processing unit 28 performs a voice recognition process on the voice signal based on a standard voice recognition model. However, in the voice recognition process, the processing unit 28
The information of the sound collected in step 2 is not reflected. In other words, the voice recognition processing is executed while the sound information is not used. As a result, the audio signal is converted to text (hereinafter, the converted audio signal is referred to as “second text”).

The comparison unit 46 inputs the first text and the second text. The comparing unit 46 compares the first text with the second text. Here, as a comparison, the first text and the second text are arranged. The comparing unit 46 outputs to the transmitting unit 40 text data in which the first text and the second text are arranged. The transmission unit 40 inputs the text data from the processing unit 28. The transmitting unit 40 transmits the text data as the comparison result to the terminal device 10 on the transmitting side. Subsequent processing is the same as that described above, and a description thereof will not be repeated.

According to the present embodiment, the result of the voice recognition processing when the setting value of the terminal device is used and the information of the collected sound is used, and the setting value of the terminal device is not used and Since the result of the speech recognition processing in the case where the information of the generated sound is not used is compared, it is possible to specify at which stage an unrecognizable phoneme occurs. In addition, using the setting value of the terminal device, there is an unrecognizable phoneme in the result of the voice recognition process when using the information of the collected sound, without using the setting value of the terminal device, In addition, when there is no unrecognizable phoneme in the result of the voice recognition processing when the information of the collected sound is not used, it is possible to recognize that the terminal device cannot be heard due to the setting of the terminal device and the surrounding noise. In addition, the result of the voice recognition process when the setting value of the terminal device is used and the information of the collected sound is used, and the information of the collected sound without using the setting value of the terminal device. If an unrecognizable phoneme is present in the result of the voice recognition process when not being used, it can be recognized that there is a cause in the stage of speech or communication.

(Example 8)
Next, an eighth embodiment will be described. The eighth embodiment corresponds to a case in which the settings made in the terminal device on the receiving side are not reflected when executing the voice recognition processing in the sixth embodiment. The settings made on the terminal device on the receiving side are on / off of the equalizer, volume level when output from the speaker, and on / off of speech speed conversion. On the other hand, in the eighth embodiment, similarly to the sixth embodiment, information of the sound collected by the microphone is used when executing the voice recognition processing. The communication system and the terminal device according to the eighth embodiment are of the same type as those in FIGS. Here, the description will focus on the differences from the past.

In the above (2), the reproducing unit 32 receives the audio signal from the receiving unit 42 and reproduces the audio signal. At this time, unlike the sixth embodiment, the setting value set by the setting unit 38 is not reflected. The microphone 22 collects sound around the terminal device 10, for example, noise, converts the collected noise into an electric signal (hereinafter, referred to as a “noise signal”), and outputs the noise signal to the processing unit 28. The processing unit 28 performs a voice recognition process on the voice signal as in the sixth embodiment. In particular,
When performing the above-described process (A) to identify the reason for the error, the processing unit 28 reflects information on the sound collected by the microphone 22 in the voice recognition process. Subsequent processing is the same as that described above, and a description thereof will not be repeated.

According to the present embodiment, since the information of the sound around the terminal device collected by the microphone is reflected, it is possible to execute the voice recognition processing so as to be closer to the way of actually hearing the voice. In addition, since the speech recognition processing is performed so as to be close to how the actual speech is heard, the accuracy of the text conversion can be improved.

(Example 9)
Next, a ninth embodiment will be described. The ninth embodiment corresponds to a combination of the eighth embodiment and the seventh embodiment.
The communication system and the terminal device according to the ninth embodiment are of the same type as those in FIGS. Here, the description will focus on the differences from the past.

In the above (2), the processing unit 28 inputs an audio signal from the reproduction unit 32. The processing unit 28 performs a voice recognition process on the voice signal based on a standard voice recognition model. Here, when the above-described processing (A) is performed to identify the reason for the error, the processing unit 2
8 reflects the information of the sound collected by the microphone 22 in the voice recognition process, as in the eighth embodiment. As a result, the audio signal is converted into a text (hereinafter, the text-converted audio signal is referred to as “
First text).

On the other hand, the processing unit 28 performs a voice recognition process on the voice signal based on a standard voice recognition model. However, the processing unit 28 does not reflect information on the sound collected by the microphone 22 in the voice recognition processing. In other words, the voice recognition processing is executed while the sound information is not used. As a result, the audio signal is converted to text (hereinafter, the converted audio signal is referred to as “second text”).

The comparison unit 46 inputs the first text and the second text. The comparing unit 46 compares the first text with the second text. Here, as a comparison, the first text and the second text are arranged. The comparing unit 46 outputs to the transmitting unit 40 text data in which the first text and the second text are arranged. The transmission unit 40 inputs the text data from the processing unit 28. The transmitting unit 40 transmits the text data as the comparison result to the terminal device 10 on the transmitting side. Subsequent processing is the same as that described above, and a description thereof will not be repeated.

According to this embodiment, the result of the voice recognition process when the information of the collected sound is used is compared with the result of the voice recognition process when the information of the collected sound is not used. At which stage an unrecognizable phoneme occurs. In addition, if there is an unrecognizable phoneme in the result of the voice recognition processing when information on the collected sound is used, and the result of the voice recognition processing when the information on the collected sound is not used is recognized. If there is no impossible phoneme, it is possible to recognize that it cannot be heard due to the surrounding noise. In addition, there are unrecognizable phonemes in the result of the voice recognition process when the information of the collected sound is used and the result of the voice recognition process when the information of the collected sound is not used. In this case, it can be recognized that there is a cause in the stage of speech or communication.

The present invention has been described based on the embodiments. This embodiment is an exemplification, and it is understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and that such modifications are also within the scope of the present invention. .

According to the first to ninth embodiments, the communication system 100 uses business wireless. However, the present invention is not limited to this. For example, a wireless communication system other than the commercial wireless communication may be used.
According to this embodiment, the degree of freedom of the configuration can be improved.

10 terminal device, 12 base station device, 14 network, 20 button, 2
2 microphone, 24 operation unit, 26 display unit, 28 processing unit, 30 communication unit, 3
2 playback unit, 34 speaker, 36 transmission unit, 38 setting unit, 40 transmission unit,
42 receiving unit, 46 comparing unit, 100 communication system.

Claims (6)

A transmitting unit that transmits an audio signal to a terminal device on a receiving side;
In the terminal device on the receiving side, a receiving unit that receives a result of a voice recognition process performed on an audio signal obtained by reproducing the received audio signal from the terminal device on the receiving side,
A processing unit that displays a result of the received voice recognition processing on a display unit,
A terminal device comprising: The result of the voice recognition process is:
2. The terminal device according to claim 1, wherein an audio signal reproduced by the terminal device on the receiving side reflects how the user who uses the terminal device on the receiving side hears. 3. The receiving unit,
In the terminal device on the receiving side,
(1) Speech recognition processing is executed without reflecting the way of hearing of the user who uses the terminal device on the receiving side. The terminal device according to claim 1, wherein the terminal device receives a comparison result obtained by comparing a result of a voice recognition process reflecting a way of hearing. The receiving unit,
In the voice recognition processing of the terminal device on the receiving side, a result obtained by reflecting at least one of a volume level and a voice speed is received from the terminal device on the receiving side. The terminal device according to any one of claims 1 to 3. Transmitting an audio signal to a terminal device on the receiving side;
In the terminal device on the receiving side, for the audio signal reproduced from the received audio signal,
Receiving a result of executing the voice recognition process from the terminal device on the receiving side,
Acquiring the result of the voice recognition process, and displaying the result on a display unit;
A communication method comprising: A process of transmitting an audio signal to a terminal device on the receiving side;
In the terminal device on the receiving side, for the audio signal reproduced from the received audio signal,
A process of receiving the result of executing the voice recognition process from the terminal device on the receiving side,
A process of acquiring a result of the voice recognition process and displaying the result on a display unit;
Communication program that causes a computer to execute.

Images