JP7373739B2 - Speech-to-text conversion system and speech-to-text conversion device - Google Patents

Description

The present disclosure relates to a speech-to-text conversion system and a speech-to-text conversion device.

Patent Document 1 proposes an audio correction device that can reduce noise and generate an easily audible audio signal. This audio correction device consists of an air conduction microphone that collects air conduction sound using air vibration, a bone conduction microphone that collects bone conduction sound using the vibration of the user's bones, and a bone conduction microphone that collects air conduction sound using air vibration. a calculation unit that calculates the ratio of the voice to the noise; and a storage unit that stores a correction coefficient for matching the frequency spectrum of the bone-conducted sound to the frequency spectrum of the air-conducted sound when the ratio is equal to or greater than a first threshold value. a correction unit that corrects bone-conducted sound using a correction coefficient; and a generation unit that generates an output signal from the corrected bone-conducted sound when the ratio becomes smaller than a second threshold.

JP2014-239346A

The present disclosure was devised in view of the conventional circumstances described above, and an object of the present disclosure is to provide a speech-to-text conversion system and a speech-to-text conversion device that can recognize speech and convert it into text according to the type of connected microphone. shall be.

The present disclosure is a voice-to-text conversion system capable of communicating between a server and a terminal device connected to a sound receiver that picks up sound, the terminal device being able to transmitting an audio signal of audio to the server; Distinguish either bone conduction sound based on the vibration of the user's eardrum or air conduction sound based on the vibration of the eardrum of the user through the air, and if the sound is the bone conduction sound, convert the bone conduction sound into the air conduction sound. Provided is a speech-to-text conversion system that converts the air conduction speech into text information and outputs the converted text information.

The present disclosure also provides a speech-to-text conversion device that is capable of communicating with a sound receiver that picks up speech, and which includes high-frequency components and low-frequency components among the spectral characteristics of the speech picked up by the sound receiver. a voice discrimination unit that determines whether the voice is bone conduction voice based on the vibration of the user's vocal cords or air conduction voice based on the vibration of the user's eardrum through the air, based on a ratio with the frequency component; A voice-to-text converter comprising: a voice converter that converts bone conduction voice into the air conduction voice; a voice recognition unit that converts the air conduction voice into text information; and an output unit that outputs the converted text information. Provide equipment.

According to the present disclosure, speech can be recognized and converted into text depending on the type of connected microphone.

A diagram illustrating an example use case of the speech-to-text conversion system according to Embodiment 1. A block diagram showing an example of the internal configuration of the speech-to-text conversion system according to Embodiment 1. Diagram showing an example of how a bone conduction microphone is used Diagram showing an example of how to use an air conduction microphone Sequence diagram illustrating an example of an operation procedure of the speech-to-text conversion system according to Embodiment 1 Flowchart showing an example of a speech discrimination procedure of the speech-to-text conversion system according to Embodiment 1 A diagram showing example 1 of speech recognition of the speech-to-text conversion system according to the first embodiment A diagram showing example 2 of speech recognition of the speech-to-text conversion system according to the first embodiment Diagram showing an example of a speech-to-text conversion device

(Details leading to the content of Embodiment 1)
Patent Document 1 describes a sound correction method for correcting bone conduction sound picked up by a bone conduction microphone based on the ratio of the user's voice to noise (SNR (Signal to Noise Ratio)) in the collected air conduction sound. A device has been proposed. This audio correction device uses a correction coefficient (for example, a value obtained by dividing the signal intensity obtained from an air conduction microphone by the signal intensity obtained from a bone conduction microphone) when the ratio is equal to or higher than a first threshold value. Match the frequency spectrum of bone-conducted sound to the frequency spectrum of air-conducted sound. The audio correction device repeats the correction until the ratio becomes smaller than the second threshold value, and generates an output signal from the corrected bone conduction sound. However, the above-described sound correction device requires the simultaneous use of a bone conduction microphone and an air conduction sound microphone to collect sound, and it is difficult to correct the sound collected by one of the microphones.

Therefore, in the following various embodiments, examples of a speech-to-text conversion system and a speech-to-text conversion device that can recognize speech and convert it into text will be described depending on the type of connected microphone.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments specifically disclosing the configuration and operation of a speech-to-text conversion system and a speech-to-text conversion device according to the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of well-known matters or redundant explanations of substantially the same configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

(Embodiment 1)
FIG. 1 is a diagram showing an example use case of the speech-to-text conversion system 100 according to the first embodiment. The speech-to-text conversion system 100 includes a sound receiver 1, a terminal device 2, and a server 3. The sound receiver 1 is connected to either a bone conduction microphone MC1 or an air conduction microphone MC2.

The sound receiver 1 is a bone conduction headset that is connected to the terminal device 2 for wired communication, and includes a microphone connection terminal 11 and a speaker (not shown). The microphone connection terminal 11 is configured to be able to switch between the bone conduction microphone MC1 and the air conduction microphone MC2 for connection. The sound receiver 1 is based on an analog audio signal picked up by a connected microphone (for example, a bone conduction sound picked up by a bone conduction microphone MC1 or an air conduction sound picked up by an air conduction microphone MC2). The converted analog audio signal) is transmitted to the terminal device 2. Further, the speaker included in the sound receiver 1 is a bone conduction speaker (not shown).

The bone conduction microphone MC1 is mounted near the user's vocal cords and includes a piezoelectric element that picks up vibrations of the vocal cords (bone conduction sound). The bone conduction microphone MC1 generates an electric potential from mechanical stress accompanying the vibration of the vocal cords that have been picked up, and converts the electric potential into an audio signal (that is, an analog audio signal). Note that the bone conduction microphone MC1 is not limited to the vicinity of the vocal cords, but may be mounted, for example, on the zygomatic arch to pick up the vibrations of nasal sounds propagated by the vibrations of the vocal cords.

The bone conduction microphone MC1 includes an amplifier (not shown) that amplifies the collected bone conduction sound (vibration), and amplifies the vibration of the bone conduction sound. As a result, the gain of the analog audio signal converted by the bone conduction microphone MC1 is increased by the amplifier, so that the voltage drop during digital signal conversion becomes larger than that of the air conduction microphone MC2. Therefore, by detecting the voltage value of the received analog audio signal or digital audio signal, the terminal device 2 and the server 3 determine whether the audio signal is a converted bone conduction sound or a converted air conduction sound. It is possible to determine whether the

Furthermore, the bone conduction microphone MC1 is installed near the user's vocal cords and has better noise resistance than the air conduction microphone MC2. Therefore, the bone conduction microphone MC1 can pick up the user's voice even at a construction site or under an overpass where noise of 80 to 90 dB is generated, for example.

The air conduction microphone MC2 converts the user's air conduction voice propagating through the air into an audio signal (that is, an analog audio signal). Further, the air conduction microphone MC2 may be an omnidirectional microphone, a unidirectional microphone, or a phase directional microphone, or may be distinguished as each of a plurality of types of microphones by combining these microphones. .

A bone conduction speaker (not shown) converts an audio signal into mechanical vibrations, propagates the vibrations through the user's skin and skull, and transmits them to the auditory nerve. That is, while a normal speaker listens to sound transmitted by air vibrations (air conduction sound), a bone conduction speaker listens to sound transmitted by bone vibrations (bone conduction sound). Furthermore, the sound transmitted by bone conduction through the bone conduction speaker is hardly affected by external noise. That is, the sound receiver 1 equipped with the bone conduction speaker is less likely to pick up external noise, so that noise resistance can be improved. Furthermore, the mouth of the sound receiver 1 including the bone conduction microphone MC1 is completely open. Thereby, the sound receiver 1 can normally communicate even if the user also wears a dust-proof/gas-proof mask, for example.

The terminal device 2 is, for example, a smartphone, a tablet terminal, or a PC (Personal Computer), and is connected to the sound receiver 1 for wired communication. Further, the terminal device 2 is connected to the server 3 via the network NW1 so as to be able to communicate wirelessly. The terminal device 2 converts the analog audio signal received from the receiver 1 into a digital audio signal and transmits it to the server 3. The terminal device 2 also receives text information converted into text based on an analog audio signal, or an audio signal converted based on text information.

Network NW1 is a wireless network. Examples of wireless networks include wireless LAN (Local Area Network), wireless WAN (Wide Area Network), 4G (4th generation mobile communication system), LTE (Long Term Evolution), LTE-Advanced, and 5G (5th generation mobile communication system). ), Wi-fi (registered trademark), or WiGig (Wireless Gigabit).

The server 3 is connected to the terminal device 2 via the network NW1 so that they can communicate wirelessly. The server 3 converts the received digital audio signal into text information and transmits it to the terminal device 2. Furthermore, the server 3 converts the converted text information into a digital audio signal again and transmits it to the terminal device 2 .

FIG. 2 is a block diagram showing an example of the internal configuration of the speech-to-text conversion system 100 according to the first embodiment. Since the sound receiver 1 has been described with reference to FIG. 1, detailed description thereof will be omitted.

First, an example of the internal configuration of the terminal device 2 will be described. The terminal device 2 includes a communication section 20, a processor 21, a memory 22, and an A/D (Analog-to-Digital) conversion section 23.

The communication unit 20 is communicably connected to the server 3 via the network NW1. The communication unit 20 transmits the digital audio signal converted by the A/D conversion unit 23 to the server 3, and receives from the server 3 text information or a digital audio signal generated based on the text information.

The processor 21 is configured using, for example, a CPU (Central Processing unit) or an FPGA (Field Programmable Gate Array), and performs various processing and control in cooperation with the memory 22. Specifically, the processor 21 references programs and data held in the memory 22 and executes the programs to realize the functions of each part. The function of each part is, for example, a function of converting an analog audio signal received from the sound receiver 1 into a digital audio signal.

The memory 22 includes, for example, a RAM (Random Access Memory) as a work memory used when the processor 21 executes each process, and a ROM (Read Only Memory) that stores programs and data that define the operations of the processor 21. have Data or information generated or acquired by the processor 21 is temporarily stored in the RAM. A program that defines the operation of the processor 21 is written in the ROM. The memory 22 also stores digital audio signals sent to the server 3 and text information received from the server 3.

The A/D converter 23 converts the analog audio signal received from the sound receiver 1 into a digital audio signal. The A/D converter 23 transmits the converted digital audio signal to the server 3 via the network NW1. Further, the A/D converter 23 measures the voltage value dropped due to a voltage drop when receiving the analog audio signal from the sound receiver 1. Information on the measured voltage value is transmitted to the server 3.

Next, an example of the internal configuration of the server 3 will be described. The server 3 includes a communication section 30, a processor 31, a memory 32, a speech discrimination section 33, a speech conversion section 34, a speech recognition section 35, an output section 36, a storage section 37, and a text-to-speech conversion section 38. It consists of and. Note that the text-to-speech converter 38 is not an essential component, and may be omitted or provided in the terminal device 2.

The communication unit 30 is communicably connected to the terminal device 2 via the network NW1. The communication unit 30 receives a digital audio signal from the terminal device 2 and transmits text information or a digital audio signal generated based on the text information to the terminal device 2.

The processor 31 is configured using, for example, a CPU (Central Processing unit) or an FPGA (Field Programmable Gate Array), and performs various processing and control in cooperation with the memory 32. Specifically, the processor 31 references programs and data held in the memory 32 and executes the programs to realize the functions of each part. The functions of each part include, for example, a function to determine whether a digital voice signal is bone conduction voice or air conduction voice, and a function to convert a digital voice signal into text information based on pre-generated learning data. It is.

The memory 32 includes, for example, a RAM (Random Access Memory) as a work memory used when the processor 31 executes each process, and a ROM (Read Only Memory) that stores programs and data that define the operations of the processor 31. have Data or information generated or acquired by the processor 31 is temporarily stored in the RAM. A program that defines the operation of the processor 31 is written in the ROM. The memory 32 also stores learning data, acoustic models, pronunciation dictionaries, language models, recognition decoders, and the like.

The voice discrimination unit 33 receives the digital voice signal and information on the voltage value that has dropped from the terminal device 2, and determines whether the voice on which the digital voice signal is based is bone conduction voice or air conduction voice based on the voltage value information. Determine if there is.

When the audio discrimination unit 33 determines that the digital audio signal is an audio signal converted based on bone conduction audio, it assigns an identifier to the digital audio signal. The identifier is, for example, an artificial sound in a frequency band different from that of human speech, or a specific identification signal (eg, "110011"). When the audio discrimination unit 33 determines that the voltage value information is less than a predetermined threshold value and that an identifier has been added, the audio discrimination unit 33 converts the discrimination result indicating that the audio that is the basis of the digital audio signal is bone conduction audio into the audio. It is output to the converter 34.

Furthermore, the audio discrimination unit 33 determines the signal level (dB) in a low frequency band (for example, 0 to 1000 Hz) relative to the signal level (dB) in a high frequency band (for example, 1001 to 8000 Hz) among the spectral characteristics of the digital audio signal. The ratio may be calculated, and based on this ratio, it may be determined whether the sound on which the digital audio signal is based is bone conduction sound or air conduction sound. Bone conduction sound collects vibrations caused by the user's voice through the body, so the signal level becomes small in high frequency bands due to internal attenuation. Therefore, the voice discrimination unit 33 determines that the voice is air conduction voice when the signal level in the low frequency band is attenuated relative to the signal level in the high frequency band as the ratio value is smaller, and the relative attenuation is large. If so, it is determined that it is a bone conduction sound.

Note that the spectral characteristics obtained in the above-mentioned ratio-based voice discrimination method, which is the basis of the digital voice signal, vary depending on individual differences among users and environmental differences. Therefore, if the discrimination data generated based on a plurality of spectral characteristics collected in advance is stored in the memory 32, the speech discrimination section 33 uses the spectral characteristics and the discrimination data to determine the digital voice signal. It may also be determined whether the underlying sound is bone conduction sound or air conduction sound.

When the digital audio signal inputted from the audio discrimination unit 33 is a bone conduction sound, the voice conversion unit 34 maps the feature amount of the bone conduction sound to the feature amount of the air conduction sound using a learning model generated in advance. By doing so, a digital audio signal of bone conduction audio is converted into a digital audio signal of air conduction audio. The learning model is stored in the memory 32 in advance, and extracts the feature quantities of the bone conduction sound and the air conduction sound from the sounds simultaneously picked up from the bone conduction microphone MC1 and the air conduction microphone MC2, and extracts the features of the bone conduction sound. It is generated by mapping the amount to the feature amount of the air conduction sound. Note that the voice feature amount is information such as the fundamental frequency (voice pitch), the spectral characteristics of the voice signal (voice quality), and the aperiodic signal (voice hoarseness). The voice conversion unit 34 outputs the converted air conduction voice to the voice recognition unit 35. Note that, when the digital audio signal inputted by the audio discrimination unit 33 is air conduction audio, the audio conversion unit 34 outputs the digital audio signal as is to the audio recognition unit 35 .

The voice recognition unit 35 is, for example, a voice recognition engine, and uses an acoustic model whose database is the voice picked up by the air conduction microphone MC2, and uses the acoustic model that is included in the digital voice signal of the air conduction voice input from the voice conversion unit 34. The phoneme (for example, /a/, /k/, etc.) that is displayed is determined. Note that the acoustic model is generated in advance by statistically processing the frequency characteristics and time characteristics of the voices of several thousand people and for several thousand hours collected by the air conduction microphone MC2, and is stored in the memory 32.

Furthermore, the speech recognition section 35 uses the language model to evaluate whether the character string or word string included in the digital audio signal of the air conduction speech inputted from the speech conversion section 34 is appropriate as a language. Language models are generated by collecting texts in each country's languages and performing statistical processing. Specifically, a language model performs natural language processing to formulate the parts of speech and syntactic structure of sentences, relationships between words or documents, etc., and is calculated probabilistically from a statistical perspective. determined. The language model is, for example, an N-gram model, a hidden Markov model, a maximum entropy model, etc., and is stored in the memory 32.

The speech recognition unit 35 connects the phonemes included in the digital speech signal discriminated using the acoustic model and the character string or word string evaluated using the language model, based on the pronunciation dictionary, and generates a word utterance ( For example, /sakura/) is constructed, and a recognition decoder decodes the most acoustically and linguistically appropriate linguistic expression and converts it into text information. Note that the pronunciation dictionary is data for linking the acoustic model and the language model, and is stored in the memory 32. The recognition decoder is a so-called decoding device that uses an acoustic model, a pronunciation dictionary, and a language model to decode and convert an audio signal into a linguistic expression corresponding to the content of the utterance. The speech recognition unit 35 outputs the text information converted by the recognition decoder to the output unit 36.

Further, the voice recognition unit 35 performs a second voice recognition process in which the above-described process of converting the bone conduction voice converted into air conduction voice into text information is the first voice recognition process, and converts the bone conduction voice into text information. Processing may be executed. In this case, when the voice conversion unit 34 determines that the digital voice signal input from the voice discrimination unit 33 is a bone conduction voice, the voice recognition unit 34 converts the bone conduction voice and the air conduction voice converted from the bone conduction voice into a voice. Output to.

The speech recognition unit 35 determines the reliability of the converted first text information and second text information, and outputs text information having a higher reliability to the output unit 36. The speech recognition unit 35 performs reliability determination based on word reliability for text information. Specifically, the speech recognition unit 35 determines the word reliability based on the acoustic model and language model used when the recognition decoder converts the speech signal into the first text information and the second text information. Determine the degree. The speech recognition unit 35 determines whether there are other candidate words that are close to each word included in the text information, and if there is no other candidate word that has a score similar to that word, the reliability is high. It is determined that the reliability of the word is lower as the number of other candidates having similar scores for the word increases. The speech recognition unit 35 outputs the text information with higher reliability to the output unit 36.

The output unit 36 outputs the text information input from the speech recognition unit 35 to the storage unit 37 and the text-to-speech conversion unit 38, and outputs it to the communication unit 30. The communication unit 30 transmits text information to the terminal device 2 via the network NW1.

The storage unit 37 is a so-called storage, and stores text information converted by the speech recognition unit 35. Furthermore, the storage unit 37 may store text information for each terminal device (that is, for each user).

The text-to-speech conversion unit 38 converts the text information input from the output unit 36 into an audio signal. The converted audio signal is transmitted to the terminal device 2 via the network NW1 and played back. Thereby, the user can confirm by voice whether or not the content of the utterance has been correctly converted into text information. Moreover, since this audio signal is generated as a noiseless audio signal once converted into text information, the audio becomes easier to hear. Therefore, the user can reproduce audio with reduced noise.

Furthermore, the text-to-speech converter 38 may include a speech synthesis engine that uses an acoustic model generated based on the user's speech data. Thereby, the text-to-speech converter 38 can convert the audio signal into the user's voice and reproduce the audio signal.

As described above, the speech-to-text conversion system 100 can recognize speech and convert it into text depending on the type of connected microphone.

An example of how the sound receiver 1 is used will be described with reference to FIGS. 3A and 3B. FIG. 3A is a diagram showing an example of how the bone conduction microphone MC1 is used. FIG. 3B is a diagram showing an example of how the air conduction microphone MC2 is used.

The sound receiver 1 is used by connecting either the bone conduction microphone MC1 or the air conduction microphone MC2 to the microphone connection terminal 11. The bone conduction microphone MC1 is used by being placed in contact with the user's vocal cords. Furthermore, the air conduction microphone MC2 is used by being placed in front of the user's mouth.

FIG. 4 is a sequence diagram showing an example of the operation procedure of the speech-to-text conversion system 100 according to the first embodiment. Note that in FIG. 4, illustration of the network NW1 is omitted.

The sound receiver 1 uses either the connected bone conduction microphone MC1 or the air conduction microphone MC2 to collect the user's voice and convert it into an analog voice signal (T1).

The receiver 1 transmits an analog audio signal to the terminal device 2 (T2).

The terminal device 2 converts the received analog audio signal into a digital audio signal (T3). Note that the terminal device 2 measures the dropped voltage value based on the voltage drop when receiving the analog signal.

The terminal device 2 transmits the converted digital audio signal to the server 3 via the network NW1 (T4). Further, the terminal device 2 transmits information on the measured voltage value to the server 3 via the network NW1.

The server 3 determines whether or not the sound on which the digital audio signal is based is bone conduction sound based on the voltage value information based on the voltage drop received from the terminal device 2 (T5). Furthermore, in the process of step T5, the server 3 adds an identifier to the digital audio signal when the audio on which the digital audio signal is based is bone conduction audio.

If the result of the process in step T5 is bone conduction sound, the server 3 converts it into a digital sound signal of air conduction sound (T6). Note that the server 3 does not perform any processing if the sound is air conduction sound.

The server 3 recognizes the air conduction voice and converts it into text information (T7). Although not shown in FIG. 4, the server 3 may further generate an audio signal based on the text information.

The server 3 transmits the converted text information to the terminal device 2 via the network NW1 (T8). Note that when the server 3 further generates an audio signal in the process of step T7, it transmits at least one of the generated audio signal and text information to the terminal device 2.

The terminal device 2 receives the text information (T9). The received text information may be displayed on the terminal device 2 or may be further converted into an audio signal and transmitted to the sound receiver 1.

FIG. 5 is a flowchart illustrating an example of a speech discrimination procedure of the speech-to-text conversion system 100 according to the first embodiment. The voice discrimination process shown in FIG. 5 is executed by the voice discrimination unit 33 in the server 3.

The voice discrimination unit 33 receives a digital voice signal from the terminal device 2 (St1). In addition, the voice discrimination unit 33 receives information on the voltage value dropped due to the voltage drop at this time.

The voice determination unit 33 determines whether the voltage value is less than or equal to the threshold Th based on the information about the voltage value dropped due to the voltage drop received from the terminal device 2 (St2).

In the process of step St2, if the dropped voltage value is equal to or less than the threshold Th (St2, YES), the voice discrimination unit 33 adds an identifier to the digital voice signal (St3).

The audio determination unit 33 determines whether or not the digital audio signal has an identifier (St4). As a result, the sound discrimination unit 33 prevents the possibility that even though the sound on which the digital sound signal is based is air conduction sound, it may misjudge it as bone conduction sound if the dropped voltage value becomes large. It can be lowered.

In the process of step St4, if an identifier is assigned (St4, YES), the voice determining unit 33 determines that the voice that is the basis of the digital voice signal is a bone conduction voice (St5).

In the process of step St4, if no identifier is assigned (St4, NO), the voice determining unit 33 determines that the voice that is the basis of the digital voice signal is air conduction voice (St6).

With the above steps, the speech-to-text conversion system 100 ends the speech discrimination process.

An example of the result of the speech recognition performed by the speech-to-text conversion system 100 according to the first embodiment will be described with reference to FIGS. 6A and 6B. FIG. 6A is a diagram illustrating example 1 of speech recognition of the speech-to-text conversion system 100 according to the first embodiment. FIG. 6B is a diagram showing a second example of speech recognition by the speech-to-text conversion system 100 according to the first embodiment. 6A and 6B show examples of speech recognition results obtained by performing speech recognition on bone conduction speech, air conduction speech, and air conduction speech converted from bone conduction speech using a learning model. Each of the utterance contents U11 and U12 is a voice actually uttered by the user.

Utterance content U11 is "I play video games and games on my computer." The voice recognition result An11 is the result obtained by performing voice recognition based on the bone conduction voice, and is converted into text information such as "Play video games and games with young sounds". The voice recognition result An21 is the result obtained by performing voice recognition based on air conduction voice, and is converted into text information such as ``That should be played with a video game or one game on a computer.'' Speech recognition result An31 is the result obtained by performing speech recognition based on air conduction voice converted from bone conduction voice using a learning model, and is the result of performing voice recognition based on air conduction voice converted from bone conduction voice using a learning model. Converted to text information.

Utterance content U12 is ``I twisted all reality towards myself.'' Speech recognition result An12 is the result obtained by performing speech recognition based on bone conduction speech, and is converted into text information that says, ``I twisted all the current F towards myself.'' . Speech recognition result An22 is the result obtained by performing speech recognition based on air conduction speech, and is converted into text information that says, ``Hmm, I guess you twisted all reality to all the housewives.'' Ru. Speech recognition result An32 is the result obtained by performing speech recognition based on air conduction voice converted from bone conduction voice using a learning model, and is the result of performing voice recognition based on air conduction voice converted from bone conduction voice using a learning model. It is converted into text information such as "No."

As described above, the speech-to-text conversion system 100 can obtain speech recognition results (text information) similar to the content of the user's utterance by converting bone conduction speech into air conduction speech using the learning model.

Furthermore, the speech-to-text conversion system 100 can generate a speech signal with reduced noise by using the speech recognition result (text information).

Further, other examples of the speech-to-text conversion system 100 according to the first embodiment will be described.

The terminal device 2 is not limited to the internal configuration example shown in FIG. The terminal device 2 may be configured to include the configuration of the server 3, for example. In this case, the speech-to-text conversion device 100A does not require the network NW1 and the server 3, and can be omitted. This will be explained below with reference to FIG.

FIG. 7 is a diagram showing an example of the speech-to-text conversion device 100A. Note that the configuration of the speech-to-text conversion device 100A shown in FIG. 7 has substantially the same functions as the structure described in the speech-to-text conversion system 100 according to Embodiment 1, so the same components are denoted by the same reference numerals. will be added and the explanation will be omitted.

The speech-to-text conversion device 100A shown in FIG. 7 includes a sound receiver 1 and a terminal device 2. The terminal device 2 further includes a speech discrimination section 33, a speech conversion section 34, a speech recognition section 35, an output section 36, a storage section 37, and a text-to-speech conversion section 38. Note that the text-to-speech converter 38 is not an essential component and may be omitted. Furthermore, the terminal device 2 may further include a display unit (not shown) that displays text information.

As described above, in the speech-to-text conversion system 100 according to the first embodiment, communication is possible between the terminal device 2 to which the sound receiver 1 for collecting speech is connected and the server 3, and the terminal device 2 can communicate with the server 3. The audio signal of the voice collected by the sound device is transmitted to the server 3, and the server 3 determines whether the voice is bone conduction voice based on the vibration of the user's vocal cords or air based on the voice signal received from the terminal device 2. The device determines which of the air conduction sounds are based on the vibrations of the user's eardrum, converts the bone conduction sounds into air conduction sounds, converts the air conduction sounds into text information, and outputs the converted text information.

Thereby, the speech-to-text conversion system 100 according to the first embodiment can recognize speech and convert it into text depending on the type of connected microphone.

Furthermore, when the voice is air conduction voice, the speech-to-text conversion system 100 converts the air conduction voice into text information and outputs the converted text information. Thereby, the speech-to-text conversion system 100 according to the first embodiment can recognize speech and convert it into text depending on the type of connected microphone.

Further, the sound receiver 1 includes either a bone conduction microphone MC1 that acquires bone conduction sound or an air conduction microphone MC2 that acquires air conduction sound. Thereby, the speech-to-text conversion system 100 can connect a plurality of types of microphones, and can recognize speech and convert it into text according to the type of connected microphone.

Further, the server 3 determines whether the audio signal is bone conduction sound or air conduction sound based on the ratio of high frequency components to low frequency components among the spectral characteristics of the audio signal. Thereby, the speech-to-text conversion system 100 can determine whether the sound on which the sound signal is based is bone conduction sound or air conduction sound, and performs speech recognition of the sound according to the type of connected microphone. Can convert text.

Further, the server 3 determines whether the audio signal is bone conduction sound or air conduction sound based on the voltage value that drops when the audio signal is received from the terminal device 2 (that is, the voltage drop value). Thereby, the speech-to-text conversion system 100 can determine whether the sound on which the sound signal is based is bone conduction sound or air conduction sound, and performs speech recognition of the sound according to the type of connected microphone. Can convert text.

Further, when the sound picked up by the sound receiver 1 is a bone-conducted sound, the server 3 gives the audio signal an identifier indicating that the sound is a bone-conducted sound. Thereby, the speech-to-text conversion system 100 can execute subsequent processing while making it clearer that the audio that is the basis of the audio signal is bone conduction audio.

Furthermore, the server 3 determines whether the sound is bone conduction sound or air conduction sound based on whether an identifier is assigned. Thereby, the speech-to-text conversion system 100 can more reliably determine that the audio that is the basis of the audio signal is bone conduction audio. In addition, the speech-to-text conversion system 100 prevents the possibility that even though the sound on which the digital sound signal is based is air conduction sound, it may be misjudged as bone conduction sound if the dropped voltage value becomes large. It can be lowered.

Further, the identifier is a sound source in a frequency band different from that of the voice. Thereby, the speech-to-text conversion system 100 can assign an identifier without damaging the user's voice, and can further reduce the possibility of misidentification.

The server 3 also has a learning model for converting bone conduction sound into air conduction sound, and the learning model converts the bone conduction sound into bone conduction sound based on the sound picked up simultaneously from the bone conduction microphone and the air conduction microphone. and the features of the air conduction voice, respectively. The server 3 converts the extracted feature amount of the bone conduction sound into a feature amount of the air conduction sound. Thereby, the speech-to-text conversion system 100 can perform efficient speech recognition, and can also remove noise specific to bone conduction speech when converting it into a feature amount of air conduction speech.

Furthermore, the server 3 recognizes speech using an acoustic model that uses air conduction speech as a database. Thereby, the speech-to-text conversion system 100 can perform efficient speech recognition.

In addition, when the sound picked up by the sound receiver 1 is bone conduction sound, the server 3 performs first voice recognition that converts the air conduction sound converted based on the bone conduction sound into first text information. and a second voice recognition process that converts the bone conduction voice into second text information. The server 3 determines and compares the reliability of each of the first text information and the second text information, and outputs the text information with the higher reliability. Thereby, the speech-to-text conversion system 100 can more accurately convert the speech picked up by the sound receiver 1 into text information.

A speech-to-text conversion device 100A according to a modification of the first embodiment is a speech-to-text conversion device 100A that can communicate with a sound receiver 1 that collects speech, and is capable of communicating with a sound receiver 1 that collects speech. includes a voice discrimination unit that determines either bone conduction voice based on the vibration of the user's vocal cords or air conduction voice based on the vibration of the user's eardrum through the air; and a voice that converts the bone conduction voice into the air conduction voice. The apparatus includes a converting section, a speech recognition section that converts air conduction speech into text information, and an output section that outputs the converted text information.

Thereby, the speech-to-text conversion device 100A according to the modification of the first embodiment can recognize speech and convert it into text depending on the type of connected microphone.

Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that those skilled in the art can come up with various changes, modifications, substitutions, additions, deletions, and equivalents within the scope of the claims, and It is understood that it falls within the technical scope of the present disclosure. Further, each of the constituent elements in the various embodiments described above may be arbitrarily combined without departing from the spirit of the invention.

The present disclosure provides a speech-to-text conversion system and a speech-to-text conversion device that can recognize and convert speech into text according to the type of connected microphone. Useful.

1 Sound receiver 11 Microphone connection terminal 2 Terminal device 20, 30 Communication section 21, 31 Processor 22, 32 Memory 23 A/D conversion section 3 Server 33 Voice discrimination section 34 Voice conversion section 35 Voice recognition section 36 Output section 37 Storage section 100 Speech-to-text conversion system 100A Speech-to-text conversion device NW1 Network MC1 Bone conduction microphone MC2 Air conduction microphone

Claims (13)

A speech-to-text conversion system capable of communicating between a server and a terminal device connected to a receiver for collecting speech, the system comprising:
The terminal device is
transmitting an audio signal of the voice collected by the sound receiver to the server;
The server is
Based on the ratio of high frequency components to low frequency components among the spectral characteristics of the audio signal received from the terminal device, whether the audio is bone conduction audio based on the vibration of the user's vocal cords or the user's audio via the air is determined. Distinguish one of the air conduction sounds based on the vibration of the eardrum,
When the sound is the bone conduction sound,
converting the bone conduction sound into the air conduction sound;
converting the air conduction voice into text information;
outputting the converted text information;
Speech to text conversion system. A speech-to-text conversion system capable of communicating between a server and a terminal device connected to a receiver for collecting speech, the system comprising:
The terminal device is
transmitting an audio signal of the voice collected by the sound receiver to the server;
The server is
When receiving the audio signal from the terminal device, it is determined whether the audio is bone conduction audio based on the vibration of the user's vocal cords or the user's eardrum via the air, based on the voltage drop value in the sound receiver. Distinguish any of the air conduction sounds based on the vibrations of the
When the sound is the bone conduction sound,
converting the bone conduction sound into the air conduction sound;
converting the air conduction voice into text information;
outputting the converted text information;
Speech to text conversion system. If the voice is the air conduction voice, converting the air conduction voice into text information and outputting the converted text information;
The speech-to-text conversion system according to claim 1 or 2 . The sound receiver includes either a bone conduction microphone that acquires the bone conduction sound or an air conduction microphone that acquires the air conduction sound.
The speech-to-text conversion system according to any one of claims 1 to 3 . When the sound picked up by the sound receiver is the bone conduction sound, the server adds an identifier to the audio signal indicating that the sound is the bone conduction sound.
The speech-to-text conversion system according to claim 1 or 2 . The server determines whether the sound is the bone conduction sound or the air conduction sound based on the presence or absence of the identifier.
The speech-to-text conversion system according to claim 5 . The identifier is an audio signal of a sound source in a frequency band different from the audio,
The speech-to-text conversion system according to claim 5 . The server has a learning model for converting the bone conduction sound to the air conduction sound,
The learning model extracts the respective feature amounts of the bone conduction sound and the air conduction sound based on the sound picked up simultaneously from the bone conduction microphone and the air conduction microphone, and converting the feature amount of the guided voice into the feature amount of the air guided voice;
The speech-to-text conversion system according to claim 4 . The server recognizes speech using an acoustic model that uses the air conduction speech as a database.
The speech-to-text conversion system according to any one of claims 1 to 3 . A speech-to-text conversion system capable of communicating between a server and a terminal device connected to a receiver for collecting speech, the system comprising:
The terminal device is
transmitting an audio signal of the voice collected by the sound receiver to the server;
The server is
Based on the audio signal received from the terminal device, determine whether the audio is bone conduction sound based on the vibration of the user's vocal cords or air conduction sound based on the vibration of the user's eardrum through the air,
When the sound is the bone conduction sound,
a first voice recognition process that converts the air conduction voice converted based on the bone conduction voice into first text information; and a second voice recognition process that converts the bone conduction voice into second text information. and run
determining and comparing the reliability of each of the first text information and the second text information, and outputting the text information with the higher reliability;
Speech to text conversion system. A speech-to-text conversion device capable of communicating with a sound receiver that picks up speech,
Based on the ratio of high frequency components to low frequency components among the spectral characteristics of the voice picked up by the sound receiver, the voice is determined to be bone conduction voice based on the vibration of the user's vocal cords or bone conduction voice based on the user's voice via the air. a sound discrimination unit that discriminates one of the air conduction sounds based on the vibration of the eardrum;
a voice conversion unit that converts the bone conduction voice to the air conduction voice;
a voice recognition unit that converts the air conduction voice into text information;
an output unit that outputs the converted text information;
Speech-to-text converter. A speech-to-text conversion device capable of communicating with a sound receiver that picks up speech,
Based on the voltage drop value in the sound receiver when the sound collected by the sound receiver is received, it is determined whether the sound is bone conduction sound based on the vibration of the user's vocal cords or the user's sound through the air. a sound discrimination unit that discriminates between air conduction sounds based on vibrations of the eardrum;
a voice conversion unit that converts the bone conduction voice to the air conduction voice;
a voice recognition unit that converts the air conduction voice into text information;
an output unit that outputs the converted text information;
Speech-to-text converter. A speech-to-text conversion device capable of communicating with a sound receiver that picks up speech,
a voice discrimination unit that determines whether the voice picked up by the sound receiver is bone conduction voice based on the vibration of the user's vocal cords or air conduction voice based on the vibration of the user's eardrum through the air;
a voice conversion unit that converts the bone conduction voice to the air conduction voice;
a first voice recognition process that converts the air conduction voice into text information and converts the air conduction voice converted based on the bone conduction voice into first text information; a speech recognition unit that executes a second speech recognition process for converting into text information ;
an output unit that determines and compares the reliability of each of the first text information and the second text information, and outputs the text information with the higher reliability;
Speech-to-text converter.

Images