JP2020197629A - Speech-text conversion system and speech-text conversion device - Google Patents

Abstract

To recognize a speech according to the kind of a connected microphone and to perform text conversion.SOLUTION: A speech-text conversion system is so configured that a terminal device to which a sound receiver for picking up a speech is connected and a server are in communication with each other. The terminal device transmits a speech signal of the speech picked up by the sound receiver to the server. The server determines, on the basis of the speech signal received from the terminal device, which of a bone-conduction speech based upon vibration of the vocal cords of a user and an air conduction speech based upon vibration of the eardrums of the user via air the speech is, and then converts, when the speech is the bone conduction speech, the bone conduction speech into an air conduction speech and the air conduction speech into text information and also outputs the converted text information.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a voice-to-text conversion system and a voice-to-text conversion device.

Patent Document 1 proposes a voice correction device capable of reducing noise and generating a voice signal that is easy to hear. This voice correction device includes an air conduction microphone that collects air conduction sound using the vibration of air, a bone conduction microphone that collects bone conduction sound using the vibration of the user's bone, and a user with air conduction sound. A calculation unit that calculates the ratio of the sound to the noise of the sound, and a storage unit that stores a correction coefficient for matching the frequency spectrum of the bone conduction sound with the frequency spectrum in the air conduction sound when the ratio is equal to or higher than the first threshold value. A correction unit that corrects the bone conduction sound using a correction coefficient, and a generation unit that generates an output signal from the corrected bone conduction sound when the ratio becomes smaller than the second threshold value.

Japanese Unexamined Patent Publication No. 2014-239346

The present disclosure has been devised in view of the above-mentioned conventional circumstances, and an object of the present disclosure is to provide a voice-text conversion system and a voice-text conversion device capable of voice-recognizing voice and text-converting voice according to the type of connected microphone. And.

The present disclosure is a voice-text conversion system capable of communicating between a terminal device to which a sound receiver for collecting voice is connected and a server, and the terminal device is said to have sound picked up by the sound receiver. A voice signal of voice is transmitted to the server, and the server uses the voice signal received from the terminal device, and the voice is a bone conduction voice based on the vibration of the user's vocal band or the user's via air. Any of the air conduction voices based on the vibration of the tympanic membrane is discriminated, and when the voice is the bone conduction voice, the bone conduction voice is converted into the air conduction voice, and the air conduction voice is converted into text information and converted. Provided is a voice-to-text conversion system that outputs the above-mentioned text information.

Further, the present disclosure is a voice text conversion device capable of communicating with a sound receiver that collects voice, and the voice picked up by the sound receiver is a bone based on the vibration of the user's voice band. A voice discrimination unit that discriminates either a guide voice or an air conduction voice based on the vibration of the user's tympanic membrane via air, a voice conversion unit that converts the bone conduction voice into the air conduction voice, and the air conduction voice. Provided is a voice-text conversion device including a voice recognition unit that converts the text information into text information and an output unit that outputs the converted text information.

According to the present disclosure, voice recognition and text conversion can be performed according to the type of connected microphone.

The figure which shows the use case example of the voice-text conversion system which concerns on Embodiment 1. A block diagram showing an example of the internal configuration of the voice-text conversion system according to the first embodiment. Diagram showing an example of using a bone conduction microphone The figure which shows the use example of the air conduction microphone A sequence diagram showing an example of an operation procedure of the voice-text conversion system according to the first embodiment. A flowchart showing an example of a voice discrimination procedure of the voice text conversion system according to the first embodiment. The figure which shows the voice recognition example 1 of the voice text conversion system which concerns on Embodiment 1. The figure which shows the voice recognition example 2 of the voice text conversion system which concerns on Embodiment 1. The figure which shows an example of the voice-text converter

(Background to the contents of the first embodiment)
Patent Document 1 describes a voice correction that corrects the bone conduction sound picked up by the bone conduction microphone based on the ratio (SNR (Signal to Noise Ratio)) of the picked air conduction sound to the noise of the user's voice. A device has been proposed. This voice correction device uses a correction coefficient (for example, a value obtained by dividing the signal strength obtained by the air conduction microphone by the signal strength obtained from the bone conduction microphone) when the ratio becomes equal to or higher than the first threshold value. Match the frequency spectrum of the bone conduction sound with the frequency spectrum in the air conduction sound. The voice correction device repeats the correction until the ratio becomes smaller than the second threshold value, and generates an output signal from the corrected bone conduction sound. However, in the above-mentioned voice correction device, it is necessary to use both the bone conduction microphone and the air conduction sound microphone at the same time to collect the sound, and it is difficult to correct the sound picked up by one of the microphones.

Therefore, in the following various embodiments, an example of a voice-text conversion system and a voice-text conversion device capable of recognizing voice and converting text according to the type of connected microphone will be described.

Hereinafter, embodiments in which the configurations and operations of the speech-text conversion system and the speech-text conversion device according to the present disclosure are specifically disclosed will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations for substantially the same configuration may be omitted. This is to avoid unnecessary redundancy of the following description and to facilitate the understanding of those skilled in the art. It should be noted that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

(Embodiment 1)
FIG. 1 is a diagram showing an example of a use case of the voice text conversion system 100 according to the first embodiment. The voice-text conversion system 100 includes a sound receiver 1, a terminal device 2, and a server 3. Either the bone conduction microphone MC1 or the air conduction microphone MC2 is connected to the sound receiver 1.

The sound receiver 1 is a bone conduction headset connected to the terminal device 2 so as to be capable of wired communication, and includes a microphone connection terminal 11 and a speaker (not shown). The microphone connection terminal 11 is configured so that the bone conduction microphone MC1 and the air conduction microphone MC2 can be switched and connected. The sound receiver 1 is based on an analog audio signal picked up by the connected microphone (for example, a bone conduction sound picked up by the bone conduction microphone MC1 or an air conduction sound picked up by the air conduction microphone MC2). The converted analog audio signal) is transmitted to the terminal device 2. The speaker included in the sound receiver 1 is a bone conduction speaker (not shown).

The bone conduction microphone MC1 is mounted near the user's vocal cords and includes a piezoelectric element that collects the vibration of the vocal cords (bone conduction voice). The bone conduction microphone MC1 generates an electric potential from the mechanical stress associated with the vibration of the picked vocal cords, and converts the electric potential into an audio signal (that is, an analog audio signal). The bone conduction microphone MC1 may be mounted not only in the vicinity of the vocal cords but also on the zygomatic arch, for example, and may pick up the vibration of the nasal cavity sound to which the vibration of the vocal cords propagates.

The bone conduction microphone MC1 has a built-in amplifier (not shown) that amplifies the collected bone conduction voice (vibration), and amplifies the vibration of the bone conduction voice. As a result, the gain of the analog audio signal converted by the bone conduction microphone MC1 is increased by the amplifier, so that the voltage drop during digital signal conversion becomes larger than that of the air conduction microphone MC2. Therefore, the terminal device 2 and the server 3 detect the voltage value of the received analog voice signal or digital voice signal, so that the voice signal is a conversion of the bone conduction sound or the air conduction voice is converted. It is possible to determine whether it is a thing.

Further, the bone conduction microphone MC1 is installed near the vocal cords of the user and has better noise resistance than the air conduction microphone MC2. Therefore, the bone conduction microphone MC1 can pick up the user's voice even at a construction site or under an elevated structure where noise of 80 to 90 dB is generated, for example.

The air conduction microphone MC2 converts the air conduction voice of the user propagating in the air into a voice signal (that is, an analog voice signal). Further, the air conduction microphone MC2 may be either an omnidirectional microphone, a unidirectional microphone or a phase directional microphone, or may be combined and distinguished as each of a plurality of types of microphones. ..

The bone conduction speaker (not shown) converts an audio signal into mechanical vibration, propagates the vibration through the user's skin and skull, and transmits the vibration to the auditory nerve. That is, a normal speaker listens to the sound transmitted by the vibration of air (air conduction sound), whereas the bone conduction speaker listens to the sound transmitted by the vibration of bone (bone conduction sound). In addition, the sound transmitted by bone conduction by the bone conduction speaker is hardly affected by external noise. That is, since the sound receiver 1 provided with the bone conduction speaker does not easily pick up external noise, the noise resistance can be improved. Further, the sound receiver 1 provided with the bone conduction microphone MC1 has a completely open mouth. As a result, the sound receiver 1 can normally communicate even if the user uses, for example, a dustproof / gas mask.

The terminal device 2 is, for example, a smartphone, a tablet terminal, or a PC (Personal Computer), and is connected to the sound receiver 1 in a wired communication manner. Further, the terminal device 2 is wirelessly connected to the server 3 via the network NW1. The terminal device 2 converts the analog voice signal received from the sound receiver 1 into a digital voice signal and transmits it to the server 3. Further, the terminal device 2 receives the text information converted into text based on the analog audio signal, or the audio signal converted based on the text information.

The network NW1 is a wireless network. The wireless network includes, for example, wireless LAN (Local Area Network), wireless WAN (Wide Area Network), 4G (4th generation mobile communication system), LTE (Long Term Evolution), LTE-Advanced, 5G (5th generation mobile communication system). ), Wi-fi®, or WiGig (Wireless Gigabit).

The server 3 is wirelessly connected to the terminal device 2 via the network NW1. The server 3 converts the received digital audio signal into text information and transmits it to the terminal device 2. Further, the server 3 converts the converted text information into a digital voice signal again and transmits the digital voice signal to the terminal device 2.

FIG. 2 is a block diagram showing an example of the internal configuration of the speech-text conversion system 100 according to the first embodiment. Since the sound receiver 1 has been described with reference to FIG. 1, detailed description thereof will be omitted.

First, an example of the internal configuration of the terminal device 2 will be described. The terminal device 2 includes a communication unit 20, a processor 21, a memory 22, and an A / D (Analog-to-Digital) conversion unit 23.

The communication unit 20 is communicably connected to the server 3 via the network NW1. The communication unit 20 transmits the digital audio signal converted by the A / D conversion unit 23 to the server 3, and receives the text information or the digital audio signal generated based on the text information from the server 3.

The processor 21 is configured by using, for example, a CPU (Central Processing unit) or an FPGA (Field Programmable Gate Array), and performs various processes and controls in cooperation with the memory 22. Specifically, the processor 21 refers to the program and data held in the memory 22 and executes the program to realize the functions of each part. The function of each part is, for example, a function of converting an analog voice signal received from the sound receiver 1 into a digital voice signal.

The memory 22 includes, for example, a RAM (Random Access Memory) as a work memory used when executing each process of the processor 21 and a ROM (Read Only Memory) for storing a program and data defining the operation of the processor 21. Have. Data or information generated or acquired by the processor 21 is temporarily stored in the RAM. A program that defines the operation of the processor 21 is written in the ROM. Further, the memory 22 stores the digital audio signal transmitted to the server 3 and the text information received from the server 3.

The A / D conversion unit 23 converts the analog audio signal received from the sound receiver 1 into a digital audio signal. The A / D conversion unit 23 transmits the converted digital audio signal to the server 3 via the network NW1. Further, the A / D conversion unit 23 measures the voltage value dropped due to the voltage drop when the analog audio signal is received from the sound receiver 1. The measured voltage value information is transmitted to the server 3.

Next, an example of the internal configuration of the server 3 will be described. The server 3 includes a communication unit 30, a processor 31, a memory 32, a voice discrimination unit 33, a voice conversion unit 34, a voice recognition unit 35, an output unit 36, a storage unit 37, and a text voice conversion unit 38. And are configured to include. The text-to-speech conversion unit 38 is not an essential configuration and may be omitted or provided in the terminal device 2.

The communication unit 30 is communicably connected to the terminal device 2 via the network NW1. The communication unit 30 receives the digital audio signal from the terminal device 2, and transmits the text information or the digital audio signal generated based on the text information to the terminal device 2.

The processor 31 is configured by using, for example, a CPU (Central Processing unit) or an FPGA (Field Programmable Gate Array), and performs various processes and controls in cooperation with the memory 32. Specifically, the processor 31 refers to the program and data held in the memory 32, and executes the program to realize the functions of each part. The functions of each part include, for example, a function of determining whether a digital voice signal is a bone conduction voice or an air conduction voice, and a function of converting a digital voice signal into text information based on pre-generated learning data. Is.

The memory 32 includes, for example, a RAM (Random Access Memory) as a work memory used when executing each process of the processor 31 and a ROM (Read Only Memory) for storing a program and data defining the operation of the processor 31. Have. Data or information generated or acquired by the processor 31 is temporarily stored in the RAM. A program that defines the operation of the processor 31 is written in the ROM. The memory 32 also stores learning data, an acoustic model, a pronunciation dictionary, a language model, a recognition decoder, and the like.

The voice discrimination unit 33 receives the digital voice signal and the information of the dropped voltage value from the terminal device 2, and based on the voltage value information, the voice that is the basis of the digital voice signal is the bone conduction voice or the air conduction voice. Determine if there is.

When the voice discrimination unit 33 determines that the digital voice signal is a voice signal converted based on the bone conduction voice, the voice discrimination unit 33 assigns an identifier. The identifier is, for example, an artificial sound having a frequency band different from that of human voice, or a specific identification signal (for example, "110011"). When the voice discrimination unit 33 determines that the voltage value information is equal to or less than a predetermined threshold value and is further assigned an identifier, the voice discrimination unit 33 determines that the voice on which the digital voice signal is based is a bone conduction voice. Output to the conversion unit 34.

Further, the voice discrimination unit 33 has a signal level (dB) in a low frequency band (for example, 0 to 1000 Hz) with respect to a signal level (dB) in a high frequency band (for example, 1001 to 8000 Hz) among the spectral characteristics of the digital voice signal. The ratio of the above may be calculated, and based on this ratio, it may be determined whether the voice that is the basis of the digital voice signal is the bone conduction voice or the air conduction voice. Since the bone conduction voice collects the vibration of the user's voice through the body, the signal level becomes small in the high frequency band due to the attenuation in the body. Therefore, the voice discrimination unit 33 determines that the smaller the ratio value, the smaller the relative attenuation of the signal level in the low frequency band with respect to the signal level in the high frequency band, and the larger the relative attenuation is. In that case, it is determined to be a bone conduction voice.

In the method of discriminating the voice that is the basis of the digital voice signal based on the above-mentioned ratio, the spectral characteristics obtained by the individual difference and the environmental difference depending on the user change. Therefore, when the discrimination data generated based on the plurality of spectral characteristics collected in advance is stored in the memory 32, the voice discrimination unit 33 uses the spectral characteristics and the discrimination data to generate a digital voice signal. It may be determined whether the underlying voice is a bone conduction voice or an air conduction voice.

When the digital voice signal input from the voice discrimination unit 33 is the bone conduction voice, the voice conversion unit 34 maps the feature amount of the bone conduction voice to the feature amount of the air conduction voice by using the learning model generated in advance. By doing so, the digital voice signal of the bone conduction voice is converted into the digital voice signal of the air conduction voice. The learning model is stored in the memory 32 in advance, and the features of the bone conduction voice and the air conduction voice are extracted from the voices simultaneously picked up from the bone conduction microphone MC1 and the air conduction microphone MC2, and the features of the bone conduction voice are extracted. It is generated by mapping the amount to the feature amount of the air-conducted voice. The voice feature amount is, for example, information such as a fundamental frequency (voice pitch), a spectral characteristic of a voice signal (voice quality), and an aperiodic signal (voice faintness). The voice conversion unit 34 outputs the converted air conduction voice to the voice recognition unit 35. If the digital voice signal input by the voice discrimination unit 33 is air conduction voice, the voice conversion unit 34 outputs the digital voice signal to the voice recognition unit 35 as it is.

The voice recognition unit 35 is, for example, a voice recognition engine, and is included in the digital voice signal of the air conduction voice input from the voice conversion unit 34 by using an acoustic model using the voice collected by the air conduction microphone MC2 as a database. The phoneme to be used (for example, / a /, / k /, etc.) is determined. The acoustic model is generated in advance by statistically processing the frequency characteristics and time characteristics of the voices of thousands and thousands of hours picked up by the air conduction microphone MC2, and is stored in the memory 32.

In addition, the voice recognition unit 35 evaluates whether or not the character string or word string included in the digital voice signal of the air conduction voice input from the voice conversion unit 34 is appropriate as a language by using the language model. The language model is generated by collecting texts in each country's language and statistically processing them. Specifically, the language model executes natural language processing and formulates the part of speech and syntactic structure of sentences, the relationship between words or documents, and is probabilistic from a statistical point of view. It is decided. The language model is, for example, an N-gram model, a hidden Markov model, a maximum entropy model, or the like, and is stored in the memory 32.

The voice recognition unit 35 connects the phonemes contained in the digital voice signal determined by using the acoustic model and the character string or word string evaluated by using the language model with the phonemes based on the pronunciation dictionary to utter a word (word utterance). For example, / sakura /) is constructed, and the recognition decoder decodes the most acoustically and linguistically suitable linguistic expression and converts it into text information. The pronunciation dictionary is data for linking the acoustic model and the language model, and is stored in the memory 32. The recognition decoder is a so-called decoding device, and executes a process of decoding and converting a voice signal into a linguistic expression corresponding to the utterance content by using an acoustic model, a pronunciation dictionary, and a language model. The voice recognition unit 35 outputs the text information converted by the recognition decoder to the output unit 36.

Further, the process of converting the bone conduction voice converted into the air conduction voice described above into text information is regarded as the first voice recognition processing, and the voice recognition unit 35 converts the bone conduction voice into text information by the second voice recognition. The process may be executed. In this case, when the voice conversion unit 34 determines that the digital voice signal input from the voice discrimination unit 33 is the bone conduction voice, the voice recognition unit 35 converts the bone conduction voice and the air conduction voice obtained by voice conversion from the bone conduction voice. Output to.

The voice recognition unit 35 determines the reliability of the converted first text information and the second text information, and outputs the text information having a higher reliability to the output unit 36. The voice recognition unit 35 executes a determination of reliability based on word reliability for text information. Specifically, the speech recognition unit 35 trusts words based on the acoustic model and the language model used when the speech signal is converted into the first text information and the second text information by the recognition decoder. Determine the degree. The voice recognition unit 35 determines whether or not there is another candidate word that is close to each word contained in the text information, and if there is no other candidate having a score similar to that word, the reliability is high. It is determined that the more other candidates have the same score for the word, the lower the reliability. The voice recognition unit 35 outputs the text information having the higher reliability to the output unit 36.

The output unit 36 outputs the text information input from the voice recognition unit 35 to the storage unit 37 and the text / voice conversion unit 38, and outputs the text information to the communication unit 30. The communication unit 30 transmits text information to the terminal device 2 via the network NW1.

The storage unit 37 is a so-called storage, and stores text information converted by the voice recognition unit 35. Further, the storage unit 37 may store text information for each terminal device (that is, for each user).

The text-to-speech conversion unit 38 converts the text information input from the output unit 36 into a voice signal. The converted audio signal is transmitted to the terminal device 2 via the network NW1 and reproduced. As a result, the user can confirm by voice whether or not the utterance content is correctly converted into text information. Further, since this voice signal is generated as a noiseless voice signal once it is converted into text information, the voice becomes easier to hear. Therefore, the user can reproduce the sound with reduced noise.

Further, the text-to-speech conversion unit 38 may have a speech synthesis engine using an acoustic model generated based on the user's speech data. As a result, the text-to-speech conversion unit 38 can convert the voice to the user's voice and reproduce the voice signal.

As described above, the voice-text conversion system 100 can perform voice recognition and text conversion of voice according to the type of the connected microphone.

An example of using the sound receiver 1 will be described with reference to FIGS. 3A and 3B. FIG. 3A is a diagram showing a usage example of the bone conduction microphone MC1. FIG. 3B is a diagram showing a usage example of the air conduction microphone MC2.

The sound receiver 1 is used by connecting either the bone conduction microphone MC1 or the air conduction microphone MC2 to the microphone connection terminal 11. The bone conduction microphone MC1 is used by being worn in contact with the vicinity of the vocal cords of the user. Further, the air conduction microphone MC2 is arranged and used so as to be located in front of the user's mouth.

FIG. 4 is a sequence diagram showing an example of an operation procedure of the voice-text conversion system 100 according to the first embodiment. In FIG. 4, the network NW1 is not shown.

The sound receiver 1 picks up the user's voice and converts it into an analog voice signal by using either the bone conduction microphone MC1 or the air conduction microphone MC2 connected (T1).

The sound receiver 1 transmits an analog audio signal to the terminal device 2 (T2).

The terminal device 2 converts the received analog voice signal into a digital voice signal (T3). The terminal device 2 measures the dropped voltage value based on the voltage drop when the analog signal is received.

The terminal device 2 transmits the converted digital audio signal to the server 3 via the network NW1 (T4). Further, the terminal device 2 transmits the measured voltage value information to the server 3 via the network NW1.

The server 3 determines whether or not the voice that is the basis of the digital voice signal is the bone conduction voice based on the information of the voltage value based on the voltage drop received from the terminal device 2 (T5). Further, in the process of step T5, when the voice that is the basis of the digital voice signal is the bone conduction voice, the server 3 assigns an identifier to the digital voice signal.

As a result of the process of step T5, the server 3 converts the bone conduction voice into a digital voice signal of the air conduction voice (T6). Note that the server 3 does not execute any processing when it is an air-conducted voice.

The server 3 recognizes the air-conducted voice and converts it into text information (T7). Although not shown in FIG. 4, the server 3 may further generate an audio signal based on the text information.

The server 3 transmits the converted text information to the terminal device 2 via the network NW1 (T8). When the server 3 further generates an audio signal in the process of step T7, the server 3 transmits at least one of the generated audio signal and the text information to the terminal device 2.

The terminal device 2 receives the text information (T9). The received text information may be displayed on the terminal device 2, or may be further converted into a voice signal and transmitted to the sound receiver 1.

FIG. 5 is a flowchart showing an example of a voice discrimination procedure of the voice text conversion system 100 according to the first embodiment. The voice discrimination process shown in FIG. 5 is executed by the voice discrimination unit 33 in the server 3.

The voice discrimination unit 33 receives a digital voice signal from the terminal device 2 (St1). Further, the voice discrimination unit 33 receives information on the voltage value dropped due to the voltage drop at this time.

The voice discrimination unit 33 determines whether or not the voltage value is equal to or less than the threshold value Th based on the information of the voltage value dropped due to the voltage drop received from the terminal device 2 (St2).

In the process of step St2, the voice discrimination unit 33 assigns an identifier to the digital voice signal when the dropped voltage value is equal to or less than the threshold value Th (St2, YES) (St3).

The voice discrimination unit 33 determines whether or not the digital voice signal has an identifier (St4). As a result, the voice discrimination unit 33 may erroneously discriminate as the bone conduction voice when the dropped voltage value becomes large even though the voice on which the digital voice signal is based is the air conduction voice. Can be lowered.

In the process of step St4, the voice discrimination unit 33 determines that the voice that is the basis of the digital voice signal is the bone conduction voice (St5) when the identifier is given (St4, YES).

In the process of step St4, the voice discrimination unit 33 determines that the voice that is the basis of the digital voice signal is the air conduction voice when the identifier is not assigned (St4, NO) (St6).

As described above, the voice text conversion system 100 ends the voice discrimination process.

An example of the voice recognition result executed by the voice text conversion system 100 according to the first embodiment will be described with reference to FIGS. 6A and 6B. FIG. 6A is a diagram showing a voice recognition example 1 of the voice text conversion system 100 according to the first embodiment. FIG. 6B is a diagram showing a voice recognition example 2 of the voice text conversion system 100 according to the first embodiment. 6A and 6B show an example of the voice recognition result of voice recognition of each of the bone conduction voice, the air conduction voice, and the air conduction voice converted from the bone degree voice using the learning model. Each of the utterance contents U11 and U12 is a voice actually uttered by the user.

The utterance content U11 is "playing a video game or a game on a personal computer". The voice recognition result An11 is a result obtained by executing voice recognition based on the bone conduction voice, and is converted into text information "playing a game with a video game or a young sound". The voice recognition result An21 is a result obtained by executing voice recognition based on the air-conducted voice, and is converted into text information such as "That is a video game or a one-game play on a personal computer." The voice recognition result An31 is a result obtained by executing voice recognition based on the air conduction voice converted from the bone conduction voice using a learning model, and is called "playing a game on a video game or a personal computer". Converted to text information.

The utterance content U12 is "I twisted all the reality toward myself." Speech recognition result An12 is the result obtained by performing speech recognition based on bone conduction speech, and is converted into text information that "every present F is twisted toward oneself." .. The speech recognition result An22 is the result obtained by performing speech recognition based on the air-conducted speech, and is converted into the text information "Hmm, I wonder if every reality was twisted toward all housewives." To. The speech recognition result An32 is the result obtained by performing speech recognition based on the air conduction speech converted from the bone conduction speech using the learning model, and "twisted all the realities toward oneself." It is converted to the text information "Noda".

As described above, the voice text conversion system 100 can obtain a voice recognition result (text information) similar to the user's utterance content by converting the bone conduction voice into the air conduction voice using the learning model.

Further, the voice-text conversion system 100 can generate a voice signal with reduced noise by using the voice recognition result (text information).

In addition, other examples of the voice-text conversion system 100 according to the first embodiment will be described.

The terminal device 2 is not limited to the internal configuration example shown in FIG. The terminal device 2 may be configured including, for example, the configuration of the server 3. In this case, the voice-text converter 100A does not require the network NW1 and the server 3, and can be omitted. Hereinafter, description will be made with reference to FIG. 7.

FIG. 7 is a diagram showing an example of the voice text conversion device 100A. Since the configuration of the voice text conversion device 100A shown in FIG. 7 has substantially the same function as the function described in the voice text conversion system 100 according to the first embodiment, the same reference numerals are given to the same configuration. Is added to omit the description.

The voice-text conversion device 100A shown in FIG. 7 includes a sound receiver 1 and a terminal device 2. The terminal device 2 is further composed of a voice discrimination unit 33, a voice conversion unit 34, a voice recognition unit 35, an output unit 36, a storage unit 37, and a text voice conversion unit 38. The text-to-speech conversion unit 38 is not an essential configuration and may be omitted. Further, the terminal device 2 may further include a display unit (not shown) for displaying text information.

As described above, the voice text conversion system 100 according to the first embodiment can communicate between the terminal device 2 and the server 3 to which the sound receiver 1 for collecting the sound is connected, and the terminal device 2 receives the sound. The voice signal of the voice picked up by the sound device is transmitted to the server 3, and the server 3 transmits the bone conduction voice or air based on the vibration of the user's vocal band based on the voice signal received from the terminal device 2. It discriminates one of the air-conducted voices based on the vibration of the user's tympanic membrane, converts the bone-conducted voice into air-conducted voice, converts the air-conducted voice into text information, and outputs the converted text information.

As a result, the voice-to-text conversion system 100 according to the first embodiment can recognize the voice and convert the text according to the type of the connected microphone.

Further, when the voice is an air-conducted voice, the voice-text conversion system 100 converts the air-conducted voice into text information and outputs the converted text information. As a result, the voice-to-text conversion system 100 according to the first embodiment can recognize the voice and convert the text according to the type of the connected microphone.

Further, the sound receiver 1 includes either the bone conduction microphone MC1 for acquiring the bone conduction sound or the air conduction microphone MC2 for acquiring the air conduction sound. As a result, the voice-text conversion system 100 can connect a plurality of types of microphones, and can perform voice recognition and text conversion of voice according to the types of connected microphones.

Further, the server 3 determines whether the audio signal is a bone conduction voice or an air conduction voice based on the ratio of the high frequency component and the low frequency component in the spectral characteristics of the voice signal. As a result, the voice-to-speech conversion system 100 can discriminate whether the voice that is the basis of the voice signal is a bone conduction voice or an air conduction voice, and recognizes the voice according to the type of the connected microphone. Can be converted to text.

Further, the server 3 determines whether the voice signal is the bone conduction voice or the air conduction voice based on the voltage value (that is, the voltage drop value) that drops when the voice signal is received from the terminal device 2. As a result, the voice-to-speech conversion system 100 can discriminate whether the voice that is the basis of the voice signal is a bone conduction voice or an air conduction voice, and recognizes the voice according to the type of the connected microphone. Can be converted to text.

Further, when the voice picked up by the sound receiver 1 is a bone conduction voice, the server 3 assigns an identifier indicating that the voice is a bone conduction voice to the voice signal. As a result, the voice-to-speech conversion system 100 can further clarify that the voice that is the basis of the voice signal is the bone conduction voice and execute the subsequent processing.

Further, the server 3 determines whether the voice is a bone conduction voice or an air conduction voice based on whether or not an identifier is assigned. As a result, the voice-to-speech conversion system 100 can more reliably determine that the voice that is the basis of the voice signal is the bone conduction voice. Further, the voice-to-text conversion system 100 may erroneously determine that the voice that is the basis of the digital voice signal is the bone-conducted voice when the dropped voltage value becomes large even though the voice is the air-conducted voice. Can be lowered.

The identifier is a sound source having a frequency band different from that of voice. As a result, the voice-text conversion system 100 can assign an identifier without damaging the user's voice, and can further reduce the possibility of erroneous determination.

Further, the server 3 has a learning model for converting the bone conduction voice into the air conduction voice, and the learning model is based on the voice picked up from the bone conduction microphone and the air conduction microphone at the same time. And the feature amount of the air conduction voice is extracted respectively. The server 3 converts the feature amount of the extracted bone conduction voice into the feature amount of the air conduction voice. As a result, the voice-text conversion system 100 can perform efficient voice recognition and can remove noise peculiar to bone-conducted voice when converting to a feature amount of air-conducted voice.

Further, the server 3 recognizes the voice by using an acoustic model using the air conduction voice as a database. As a result, the voice-text conversion system 100 can perform efficient voice recognition.

Further, the server 3 is a first voice recognition that converts the air-conducted voice converted based on the bone-conducted voice into the first text information when the voice picked up by the sound receiver 1 is the bone-conducted voice. The process and the second speech recognition process of converting the bone conduction voice into the second text information are executed. The server 3 determines and compares the reliability of each of the first text information and the second text information, and outputs the text information having the higher reliability. As a result, the voice-text conversion system 100 can more accurately convert the voice picked up by the sound receiver 1 into text information.

The voice text conversion device 100A according to the modification of the first embodiment is a voice text conversion device 100A capable of communicating with a sound receiver 1 that collects voice, and is a voice picked up by the sound receiver. Is a voice discriminator that determines either a bone conduction voice based on the vibration of the user's voice band or an air conduction voice based on the vibration of the user's tympanic membrane via air, and a voice that converts the bone conduction voice into the air conduction voice. It includes a conversion unit, a voice recognition unit that converts air conduction voice into text information, and an output unit that outputs the converted text information.

As a result, the voice text conversion device 100A according to the modified example of the first embodiment can recognize the voice and convert the text according to the type of the connected microphone.

Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modification examples, modification examples, replacement examples, addition examples, deletion examples, and equal examples within the scope of claims. It is understood that it belongs to the technical scope of the present disclosure. In addition, each component in the various embodiments described above may be arbitrarily combined as long as the gist of the invention is not deviated.

The present disclosure presents a speech-to-speech conversion system and a speech-text converter that can recognize speech and perform text conversion according to the type of connected microphone in the presentation of the speech-to-speech conversion system and the speech-to-speech converter. It is useful.

1 Sound receiver 11 Microphone connection terminal 2 Terminal device 20, 30 Communication unit 21, 31 Processor 22, 32 Memory 23 A / D conversion unit 3 Server 33 Voice discrimination unit 34 Voice conversion unit 35 Voice recognition unit 36 Output unit 37 Storage unit 100 Voice-to-text conversion system 100A Voice-to-text conversion device NW1 Network MC1 Bone conduction microphone MC2 Air conduction microphone

Claims (12)

It is a voice-text conversion system that can communicate between a terminal device and a server to which a sound receiver that collects voice is connected.
The terminal device is
The voice signal of the voice picked up by the sound receiver is transmitted to the server.
The server
Based on the voice signal received from the terminal device, it is determined whether the voice is a bone conduction voice based on the vibration of the user's vocal cords or an air conduction voice based on the vibration of the eardrum of the user via air.
When the voice is the bone conduction voice,
The bone conduction voice is converted into the air conduction voice,
Convert the air conduction voice into text information and
Output the converted text information,
Speech-to-text conversion system. When the voice is the air conduction voice, the air conduction voice is converted into text information, and the converted text information is output.
The voice-to-text conversion system according to claim 1. The sound receiver comprises either a bone conduction microphone that acquires the bone conduction sound or an air conduction microphone that acquires the air conduction sound.
The voice-to-text conversion system according to claim 1 or 2. The server determines whether the voice signal is the bone conduction voice or the air conduction voice based on the ratio of the high frequency component and the low frequency component in the spectral characteristics of the voice signal.
The voice-to-text conversion system according to claim 1 or 2. When the server receives the voice signal from the terminal device, the server determines whether the voice signal is the bone conduction voice or the air conduction voice based on the voltage drop value in the sound receiver.
The voice-to-text conversion system according to claim 1 or 2. When the voice picked up by the sound receiver is the bone conduction voice, the server assigns an identifier indicating that the voice is the bone conduction voice to the voice signal.
The voice-to-text conversion system according to claim 4 or 5. The server determines whether the voice is the bone conduction voice or the air conduction voice based on the presence or absence of the identifier.
The voice-to-text conversion system according to claim 6. The identifier is an audio signal of a sound source having a frequency band different from that of the audio.
The voice-to-text conversion system according to claim 6. The server has a learning model for converting the bone conduction voice into the air conduction voice.
The learning model extracts the feature amounts of the bone conduction voice and the air conduction voice based on the voice picked up from the bone conduction microphone and the air conduction microphone at the same time, and the extracted bone. Converting the feature amount of the guide sound into the feature amount of the air guide voice,
The voice-text conversion system according to claim 3. The server recognizes voice using an acoustic model using the air conduction voice as a database.
The voice-to-text conversion system according to claim 1 or 2. The server
When the voice picked up by the sound receiver is the bone conduction voice, the first voice recognition process for converting the air conduction voice converted based on the bone conduction voice into the first text information. , A second voice recognition process that converts the bone conduction voice into a second text information is executed.
The reliability of each of the first text information and the second text information is determined and compared, and the text information having the higher reliability is output.
The voice-to-text conversion system according to claim 1 or 2. A voice-to-text converter that can communicate with a sound receiver that picks up sound.
A voice discrimination unit that determines whether the sound picked up by the sound receiver is a bone conduction sound based on the vibration of the user's vocal cords or an air conduction sound based on the vibration of the user's eardrum via air.
A voice conversion unit that converts the bone conduction voice into the air conduction voice,
A voice recognition unit that converts the air-conducted voice into text information,
An output unit that outputs the converted text information is provided.
Speech-to-text converter.

Images