JP7512684B2 - Speech input system and input speech processing method - Google Patents

Description

The present invention relates to a voice input system and an input voice processing method.

Patent document 1 describes using a wireless headset equipped with a microphone and earphones as a voice input device to transmit the user's captured speech to an AI assistant. Patent document 2 describes wireless earphones with a microphone that serve as a voice input device.

JP 2020-030780 A JP 2019-195179 A

When a speaker speaks into a voice input device, such as a wireless earphone with a microphone, to be sent to an AI assistant in the presence of loud ambient noise, the loud ambient noise is picked up by the microphone of the voice input device along with the spoken voice and sent to the AI assistant. As a result, there is a risk that the AI assistant will not be able to recognize the user's voice and will not be able to respond appropriately.

The problem that the present invention aims to solve is to provide a voice input system and an input voice processing method that can achieve a high recognition rate for spoken voice by an AI assistant even when the surrounding noise is loud.

In order to solve the above problems, the present invention has the following configurations 1) to 4).
1) a first microphone that picks up a voice at a first position outside the ear canal of a speaker and outputs a first input voice signal;
a second microphone that picks up a voice at a second position outside the ear canal of the speaker that is closer to the mouth of the speaker than the first position and outputs a second input voice signal;
a third microphone that picks up a voice in the ear canal of the speaker and outputs a third input voice signal;
a control unit that detects a sound pressure of the first input audio signal, sets a reflection degree of each of the second input audio signal and the third input audio signal in accordance with the detected sound pressure, and generates an output audio signal including at least one of the second input audio signal and the third input audio signal based on the reflection degree;
a communication unit that transmits the output audio signal to an external device;
The audio input device includes a first audio input device and a second audio input device that can communicate with each other,
The control unit of the first voice input device
This is an audio input system that determines whether the sound pressure of the first input audio signal in the first audio input device is larger than the sound pressure of the first input audio signal in the second audio input device, and sets the output audio signal to be transmitted from the communication unit to the outside based on the determination result.
2) acquiring a first input audio signal based on a sound picked up at a first position outside the ear canal of a speaker, and detecting a sound pressure of the first input audio signal;
acquiring, as a second input audio signal, a sound picked up at a second position outside the ear canal of the speaker that is closer to the mouth of the speaker than the first position;
Acquiring a third input audio signal based on a sound picked up in the ear canal of the speaker;
a pair of audio input devices, a first audio input device and a second audio input device, which selectively set one of the second input audio signal and the third input audio signal as an output audio signal in response to a sound pressure of the first input audio signal and transmit the output audio signal to an outside;
The first audio input device and the second audio input device are attached to the left ear and the right ear, respectively;
This is an input audio processing method that determines whether the sound pressure of the first input audio signal in the first audio input device is larger than the sound pressure of the first input audio signal in the second audio input device, and sets the output audio signal to be transmitted from a communication unit to the outside based on the determination result.
3) acquiring a first input audio signal based on a sound picked up at a first position outside the ear canal of the speaker, and detecting a sound pressure of the first input audio signal;
acquiring, as a second input audio signal, a sound picked up at a second position outside the ear canal of the speaker that is closer to the mouth of the speaker than the first position;
Acquiring a third input audio signal based on a sound picked up in the ear canal of the speaker;
a first audio input device and a second audio input device which are a pair of audio input devices for mixing the second input audio signal and the third input audio signal at a sound pressure ratio corresponding to the sound pressure of the first input audio signal, setting the output audio signal as an output audio signal, and transmitting the output audio signal to an external device;
The first audio input device and the second audio input device are attached to the left ear and the right ear, respectively;
This is an input audio processing method that determines whether the sound pressure of the first input audio signal in the first audio input device is larger than the sound pressure of the first input audio signal in the second audio input device, and sets the output audio signal to be transmitted from a communication unit to the outside based on the determination result.

The present invention has the advantage that even when the surrounding noise is loud, the AI assistant can recognize the spoken voice with a high accuracy.

FIG. 1 is a schematic cross-sectional view showing an earphone 91 which is a first embodiment of a voice input device according to the present invention. FIG. 2 is a block diagram of the earphone 91. FIG. 3 is a diagram showing the operation of the earphone 91. FIG. 4 is a block diagram showing an earphone 91A which is a first variation of the earphone 91. FIG. 5 is a diagram showing the operation of the earphone 91A. FIG. 6 is a diagram showing the operation of the second modified example of the earphone 91. FIG. 7 is a diagram showing the operation of the third modified example of the earphone 91. FIG. 8 is a diagram showing the operation of the fourth modified example of the earphone 91. FIG. 9 is a block diagram showing an earphone system 91ST which is a second embodiment of a voice input system according to the present invention. FIG. 10 is a table showing the operation of the earphone system 91ST. FIG. 11 is a schematic cross-sectional view showing an example of the arrangement position of the third microphone M3 when the third microphone M3 is a bone conduction microphone.

Example 1
A voice input device according to an embodiment of the present invention will be described using an earphone 91 of Example 1 with reference to FIGS.

Fig. 1 is a vertical cross-sectional view showing an earphone 91 according to Example 1. In Fig. 1, the earphone 91 is shown in use, being attached to the auricle E of a speaker H.
FIG. 2 is a block diagram of the earphone 91.

The earphone 91 has a main body 1 and an insertion portion 2 that protrudes from the main body 1 and is inserted into the ear canal E1.
The main body 1 has a first microphone M1 and a second microphone M2, a control unit 3, a communication unit 4, a drive unit 5, and a speaker unit 6.
The insertion section 2 has a third microphone M3.
The control unit 3 includes a sound pressure detection unit 3a and an input selection unit 3b.

The main body 1 has an air chamber 1 a on the sound output side of the speaker unit 6 .
The insertion portion 2 has a sound emission path 2a that opens at the tip and communicates with the air chamber 1a.
When the earphone 91 is in use, sound output from the speaker unit 6 by the operation of the drive unit 5 passes through the air chamber 1a and the sound output path 2a and is released into the ear canal E1.
This allows the earphone 91 to receive an audio signal wirelessly transmitted from an external audio playback device via the communication unit 4 and play the signal in the speaker unit 6 via the control unit 3 and drive unit 5 .

The first microphone M1 is disposed at a first position on the main body 1 that is farther from the mouth of the speaker H, and picks up sounds around the main body 1.
The second microphone M2 is disposed at a second position on the main body 1 that is closer to the mouth of the speaker H, and mainly collects the voice uttered by the speaker H as air-conducted sound. In other words, when the earphones 91 are in use, the second microphone M2 is located closer to the mouth of the speaker H than the first microphone M1.
Hereinafter, the sound around the main body 1 will also be simply referred to as ambient sound.
The third microphone M3 is an air conduction sound microphone and is arranged at a third position facing the sound emission path 2a of the insertion portion 2. The third microphone M3 picks up air conduction sound generated when the sound emitted by the speaker H reaches the external auditory canal E1 as bone conduction sound and reverberates in the internal space Ev of the external auditory canal E1 and the sound emission path 2a.
That is, the first position of the first microphone M1 is outside the ear canal of the speaker H. The second position of the second microphone M2 is outside the ear canal of the speaker H and closer to the mouth of the speaker H than the first position. The third microphone M3 is inside the ear canal of the speaker H.

The sound pressure detector 3a of the control unit 3 detects the sound pressure of the input sound signal SN1, which is the first input sound signal received from the first microphone M1, and outputs it as a detected first sound signal SN1a.
The sound pressure is detected, for example, as an equivalent continuous-tone acoustic level (LAeq). The sound pressure of the detected first sound signal SN1a detected as the equivalent continuous-tone acoustic level (LAeq) by the sound pressure detector 3a is hereinafter referred to as a sound pressure Va.
As described above, the first microphone M1 mainly picks up ambient sound, and therefore the sound pressure Va can be regarded as the sound pressure of the ambient sound.

As shown in Figure 2, the input selection unit 3b receives an input audio signal SN2 which is the second input audio signal from the second microphone M2, an input audio signal SN3 which is the third input audio signal from the third microphone M3, and a detected first audio signal SN1a from the sound pressure detection unit 3a.
The input selection unit 3 b of the control unit 3 generates and outputs an output audio signal SNt to the communication unit 4 .
At this time, the input selection unit 3b of the control unit 3 sets the reflection degrees RF1 and RF2 of the input audio signals SN2 and SN3 in the output audio signal SNt, respectively, based on the sound pressure Va of the detected first audio signal SN1a.
The reflection degree RF1 and the reflection degree RF2 are indexes indicating the degree to which the input audio signal SN2 and the input audio signal SN3 are reflected in the output audio signal SNt, respectively.
The index is, for example, the magnitude of sound pressure. In this example, the input selection unit 3b sets the reflection degree RF1 and the reflection degree RF2 as an alternative, one being reflected and the other not being reflected.
That is, the input selection unit 3b selectively sets and generates the output audio signal SNt to switch to either the input audio signal SN2 or the input audio signal SN3 based on the sound pressure Va of the detected first audio signal SN1a. As a result, the output audio signal SN1 includes at least one of the input audio signal SN2 and the input audio signal SN3.
The communication unit 4 wirelessly transmits the output audio signal SNt output from the input selection unit 3b to the outside of the earphone 91. The wireless transmission is performed by, for example, Bluetooth (registered trademark).

Next, the input voice processing method by the operation of the input selection unit 3b will be described in detail with reference to FIG.
FIG. 3 is a diagram in which the horizontal axis represents sound pressure Va and the vertical axis represents the input audio signal SN2 of the second microphone M2 and the input audio signal SN3 of the third microphone M3, which are alternatively selected as the output audio signal SNt.
As arbitrary values of the sound pressure Va, a first sound pressure, a lower switching sound pressure Va1, and a second sound pressure, an upper switching sound pressure Va2, which is higher than the lower switching sound pressure Va1, are preset.

When the sound pressure Va is less than the switching lower sound pressure Va1, the input selection unit 3b selects and sets the input sound signal SN2 as the output sound signal SNt to be generated. When the sound pressure Va is greater than the switching upper sound pressure Va2, the input selection unit 3b selects and sets the input sound signal SN3 as the output sound signal SNt to be generated.

When the input audio signal SN2 is selected and set as the output audio signal SNt, if the sound pressure Va increases and exceeds the upper switching sound pressure Va2, the input selection unit 3b switches the input audio signal SN2 to the input audio signal SN3.
When the input audio signal SN3 is selected and set as the output audio signal SNt, if the sound pressure Va decreases to less than the switching lower sound pressure Va1, the input selection unit 3b switches the input audio signal SN3 to the input audio signal SN2.

That is, when the ambient sound is low, the earphone 91 transmits the voice of the speaker H, which is picked up by the second microphone M2 as air-conducted sound outside the ear canal E1, to the outside as an output audio signal SNt.
Furthermore, when the ambient sound is loud, the earphone 91 transmits to the outside the voice of the speaker H picked up by the third microphone M3 as air-conducted sound that has passed through bone conduction in the external auditory canal E1 as an output audio signal SNt.

The voice of speaker H picked up in the ear canal E1 as air-conducted sound via bone conduction or as bone conduction sound is less clear than the voice picked up outside the ear canal E1 as air-conducted sound, but is obtained with a stable sound pressure with almost no influence from ambient sounds.
Therefore, even if the surrounding sound is loud, the earphone 91 can output the output audio signal SNt with a high sound pressure of the voice of the speaker H without being drowned out by the surrounding sound.
Furthermore, when the ambient sound is low, the voice of the speaker H is picked up outside the ear canal E1 as air-conducted sound, and the sound pressure of the voice of the speaker H is relatively high, allowing a clearer output voice signal SNt to be sent.

As shown in FIG. 3, in the earphone 91, the upper switching sound pressure Va2 at which the input selection unit 3b switches the output audio signal SNt from the input audio signal SN2 to the input audio signal SN3 and the lower switching sound pressure Va1 at which the input audio signal SN3 switches to the input audio signal SN2 are set to different values. Specifically, the upper switching sound pressure Va2 is set higher than the lower switching sound pressure Va1.

By making the value of the upper switching sound pressure Va2 different from the value of the lower switching sound pressure Va1, when the ambient sound picked up by the first microphone M1 frequently rises and falls between the lower switching sound pressure Va1 or the upper switching sound pressure Va2, it is possible to avoid a phenomenon in which the input audio signal SN2 and the input audio signal SN3 frequently switch and the sound pressure or sound quality of the output audio signal SNt becomes unstable. This prevents a deterioration in the voice recognition of the AI assistant 81 due to fluctuations in the sound pressure of the ambient sound of the earphone 91.

In addition, by setting the upper switching sound pressure Va2 to be greater than the lower switching sound pressure Va1, a problem is prevented in which the input audio signal to be selected is not switched to when the increase/decrease fluctuation of the sound pressure Va is reversed between the lower switching sound pressure Va1 and the upper switching sound pressure Va2.

The lower switching sound pressure Va1 and the upper switching sound pressure Va2 are appropriately set by the manufacturer in accordance with the environment in which the earphones 91 are used, etc., so that the recognition rate of the AI assistant 81 is maintained high. In addition, the present invention is not limited to this, and the speaker H may be able to adjust the lower switching sound pressure Va1 and the upper switching sound pressure Va2 in accordance with the environment in which the earphones 91 are used.

As described above, the earphone 91 maintains the voice of the speaker H at a high sound pressure in the output audio signal SNt generated by the control unit 3 and sent from the communication unit 4, regardless of the volume of the sound around the main body 1.
As a result, the AI assistant 81 that receives the output voice signal SNt can improve the recognition rate of the voice uttered by the speaker H.

The above detailed description of Example 1 is not limited to the above-mentioned configuration and procedures, and may be modified without departing from the spirit and scope of the present invention.

(Variation 1)
FIG. 4 is a block diagram showing an earphone 91A which is a variation of the earphone 91, and FIG. 5 is a diagram showing the operation of the earphone 91A.
As shown in FIG. 4, the earphone 91A is the same as the earphone 91 except that the input selection unit 3b is replaced with an input mixing unit 3c.

The input mixer 3c receives the input audio signal SN2 from the second microphone M2, the input audio signal SN3 from the third microphone M3, and the detected first audio signal SN1a from the sound pressure detector 3a.
The input mixer 3c mixes the input audio signal SN2 and the input audio signal SN3 at a sound pressure ratio according to the sound pressure Va of the detected first audio signal SN1a, and outputs the mixed signal to the communication unit 4 as an output audio signal SNt.
The input mixing unit 3c of the control unit 3 sets the reflection degrees RF2 and RF3 of the input audio signals SN2 and SN3 in the output audio signal SNt based on the ratio of their sound pressures. The sound pressure ratio is the ratio of the sound pressures of the input audio signals SN2 and SN3 included in the output audio signal SNt.

The input voice processing method by the operation of the input mixer 3c will be described with reference to FIG.
In Figure 5, the horizontal axis is a linear axis of sound pressure Va, the left vertical axis is a linear axis of mixed sound pressure V of input audio signal SN2 and input audio signal SN3, and the right vertical axis is a linear axis of total sound pressure Vt of output audio signal SNt.
The total sound pressure Vt is the sound pressure of the sound signal obtained by mixing the input sound signal SN2 and the input sound signal SN3, and includes the case where either one of them is 0 (zero).

5, a mixed lower limit sound pressure Va3 and a mixed upper limit sound pressure Va4 that is greater than the mixed lower limit sound pressure Va3 are set in advance as arbitrary values for the sound pressure Va. Hereinafter, the range of the sound pressure Va that is equal to or greater than the mixed lower limit sound pressure Va3 and equal to or less than the mixed upper limit sound pressure Va4 is also referred to as the mixed range R of the sound pressure Va.
Furthermore, a minimum mixed sound pressure Vmin, which is the minimum sound pressure to be mixed, and a maximum mixed sound pressure Vmax, which is the maximum sound pressure to be mixed, are set in advance for the input audio signal SN2 and the input audio signal SN3. The minimum mixed sound pressure Vmin may be 0 (zero).

When the sound pressure Va is less than the mixing lower limit sound pressure Va3, the input mixing unit 3c sets the input sound signal SN2 to the mixing maximum sound pressure Vmax and sets the input sound signal SN3 to the mixing minimum sound pressure Vmin.
When the sound pressure Va exceeds the mixed upper limit sound pressure Va4, the input mixing unit 3c sets the input sound signal SN2 to the mixed minimum sound pressure Vmin and sets the input sound signal SN3 to the mixed maximum sound pressure Vmax.
The input mixer 3c decreases the mixed sound pressure V for the input audio signal SN2 as the sound pressure Va increases within the mixing range R of the sound pressure Va, and increases the mixed sound pressure V for the input audio signal SN3 as the sound pressure Va increases.
That is, as the sound pressure Va increases, the input mixing unit 3c decreases the reflection degree FR2 of the input sound signal SN2 and increases the reflection degree FR3 of the input sound signal SN3.
The input mixer 3c increases or decreases the mixed sound pressure V relative to the sound pressure Va, for example, linearly, within the mixing range R of the sound pressure Va.

As a result, the input mixing unit 3c mixes the input audio signal SN2 and the input audio signal SN3 at an arbitrary sound pressure Vax in the mixing range R of the sound pressure Va with mixed sound pressures V2x and V3x corresponding to the sound pressure Vax, respectively, to generate an output audio signal SNt, and outputs the generated output audio signal SNt to the communication unit 4.

By the operation of the input mixing unit 3c described above, the total sound pressure Vt of the output audio signal SNt becomes a constant total sound pressure Vtc, regardless of the magnitude of the sound pressure Va.

The mixed lower limit sound pressure Va3, the mixed upper limit sound pressure Va4, the mixed minimum sound pressure Vmin, and the mixed maximum sound pressure Vmax are appropriately set by the manufacturer depending on the usage environment of the earphones 91A, etc., so that the voice recognition rate of the AI assistant 81 is maintained high.
The lower mixed sound pressure limit Va3, the upper mixed sound pressure limit Va4, the minimum mixed sound pressure Vmin, and the maximum mixed sound pressure Vmax may be adjustable by the speaker H.

According to the earphone 91A of the first modified example, when the sound pressure Va of the ambient sound is within the mixing range R between the lower mixing limit sound pressure Va3 and the upper mixing limit sound pressure Va4, the input audio signal SN2 and the input audio signal SN3 are mixed at sound pressures with reflection degrees FR2, FR3 corresponding to the sound pressure Va, and the ratio of the mixed sound pressures gradually changes linearly in accordance with the increase or decrease in the sound pressure of the ambient sound of the main body 1.
For example, the reflection degree FR2 in the output sound signal SNt is represented by Vmax/Vmin when the sound pressure Va is Va3, V2x/V3x when the sound pressure Vax, and Vmin/Vmax when the sound pressure Va4.
The reflection degree FR3 in the output audio signal SNt is expressed as Vmin/Vmax when the sound pressure Va is the sound pressure Va3, as V3x/V2x when the sound pressure Vax, and as Vmax/Vmin when the sound pressure Va4.
As a result, the change in sound quality of the output audio signal SNt in response to an increase or decrease in the ambient sound becomes gradual and smooth, and the recognition rate of the voice uttered by the speaker H by the AI assistant 81 is maintained high regardless of the sound pressure of the ambient sound of the main body 1.
In addition, since the total sound pressure Vt of the output audio signal SNt of the earphone 91A remains constant and does not change suddenly regardless of whether the ambient sound increases or decreases, the recognition rate of the voice uttered by the speaker H by the AI assistant 81 is maintained at a higher level.

(Variation 2)
The earphone 91A may be an earphone 91B (see FIG. 4) of variant example 2 in which, instead of the input mixing unit 3c that keeps the total sound pressure Vt of the output audio signal SNt constant regardless of the sound pressure Va, an input mixing unit 3cB (see FIG. 4) that changes the total sound pressure Vt according to the sound pressure Va, as shown in FIG. 6 .

For example, in the mixing range R of the sound pressure Va, the input mixer 3cB increases the total sound pressure Vt as the sound pressure Va increases.
Specifically, the input mixer 3cB performs a mixing operation with the maximum mixed sound pressure V2max of the input audio signal SN2 and the maximum mixed sound pressure V3max of the input audio signal SN3 set to different values, as shown in Fig. 6. For example, the maximum mixed sound pressure V3max is set to be greater than the maximum mixed sound pressure V2max.
As a result, the sound pressure of the output sound signal SNt increases and decreases between a total sound pressure Vt1 at the mixed lower limit sound pressure Va3 and a total sound pressure Vt2 that is greater than the total sound pressure Vt1 at the mixed upper limit sound pressure Va4.

When the total sound pressure Vt is constant, if the sound pressure Va is large, that is, the ambient sound is large, the sound pressure ratio of the ambient sound contained to a certain extent as background noise in the input audio signal SN2 becomes high. Therefore, the sound pressure ratio of the ambient sound in the total sound pressure Vt of the output audio signal SNt becomes relatively high.
In contrast, in the earphone 91B, the mixing ratio of the sound pressure of the input audio signal SN3 to the input audio signal SN2 increases as the sound pressure Va increases in the mixing range R of the sound pressure Va. This suppresses an increase in the sound pressure ratio of the ambient sound in the total sound pressure Vt of the output audio signal SNt.
This allows the voice recognition rate by the AI assistant 81 that receives the output voice signal SNt to be stably maintained.

(Variation 3)
The earphone 91A of the first modification may be modified to an earphone 91C of a third modification (see FIG. 4) in which the input mixer 3c is replaced with an input mixer 3cC (see FIG. 4) that nonlinearly decreases and increases the input signal.

As shown in Figure 7, in the input mixing unit 3cC, in the mixing range R at the sound pressure Va, the sound pressure Va5 that mixes the input audio signal SN2 and the input audio signal SN3 at an equal sound pressure when the sound pressure Va decreases over time is set closer to the mixing lower limit sound pressure Va3 than the midpoint between the mixing lower limit sound pressure Va3 and the mixing upper limit sound pressure Va4.
That is, the input mixing unit 3cC mixes the input audio signal SN2 and the input audio signal SN3 when the sound pressure Va decreases, based on the nonlinear characteristic lines LN2b and LN3b.

On the other hand, the input mixing unit 3cC has a sound pressure Va6 that mixes the input audio signal SN2 and the input audio signal SN3 at an equal sound pressure when the sound pressure Va increases over time, and is set closer to the mixed upper limit sound pressure Va4 than the midpoint between the mixed lower limit sound pressure Va3 and the mixed upper limit sound pressure Va4.
That is, the input mixing unit 3cC mixes the input audio signal SN2 and the input audio signal SN3 when the sound pressure Va increases, based on the nonlinear characteristic lines LN2a and LN3a.

In addition, when the sound pressure Va increases but does not reach the upper mixed sound pressure Va4 and starts to decrease, the input mixing unit 3cC changes the mixing ratio on the characteristic lines LN2a and LN3a.In addition, when the sound pressure Va decreases but does not reach the lower mixed sound pressure Va3 and starts to increase, the input mixing unit 3cC changes the mixing ratio on the characteristic lines LN3b and LN2b.

Moreover, the input mixer 3cC controls the mixing ratio of the input audio signal SN2 and the input audio signal SN3 so that the total sound pressure Vt of the output audio signal SNt becomes a constant total sound pressure Vtc regardless of the magnitude of the sound pressure Va.
The nonlinear characteristics of the input audio signals SN2 and SN3 in FIG. 7 are preset by the manufacturer of the earphone 91 or are set by adjustment of the speaker H.

When the sound pressure Va of the ambient sound is maintained relatively small and is close to the lower limit sound pressure Va3 of the mixing range R, the earphone 91C of variant example 3 generates an output audio signal SNt by mixing so that the ratio of the input audio signal SN2 to the input audio signal SN3 is higher, thereby prioritizing clarity of the sound.
In addition, when the sound pressure Va of the ambient sound is maintained relatively high and is close to the upper mixed sound pressure Va4 of the mixing range R, the earphone 91C generates the output audio signal SNt by mixing so that the ratio of the input audio signal SN3 to the input audio signal SN2 is higher, thereby prioritizing the increase in sound pressure of the audio.

In this way, the earphone 91C of the third modification generates an output audio signal SNt that is more suitable for voice recognition according to the tendency of the sound pressure Va of the ambient sound. Therefore, the recognition rate of the voice uttered by the speaker H by the AI assistant 81 is maintained at a higher level.

(Variation 4)
The earphone 91C may be modified to an earphone 91D (see FIG. 4) of variant 4 in which the input mixing unit 3cC is replaced with an input mixing unit 3cD (see FIG. 4) that changes the total sound pressure Vt in response to the sound pressure Va, as shown in FIG. 8.

For example, in a mixing range R for a sound pressure Va, the input mixing unit 3cD increases the total sound pressure Vt as the sound pressure Va increases.
8, the input mixer 3cD performs a mixing operation with the maximum mixed sound pressure V2max of the input audio signal SN2 and the maximum mixed sound pressure V3max of the input audio signal SN3 set to different values. For example, the maximum mixed sound pressure V3max is set to be greater than the maximum mixed sound pressure V2max.
As a result, the sound pressure of the output sound signal SNt increases and decreases between a total sound pressure Vt1 at the mixed lower limit sound pressure Va3 and a total sound pressure Vt2 that is greater than the total sound pressure Vt1 at the mixed upper limit sound pressure Va4.

As a result, as in variant example 2, the higher the sound pressure Va, the higher the mixing ratio of the sound pressure of the input sound signal SN3 to the input sound signal SN2. This suppresses an increase in the sound pressure ratio of the ambient sound in the total sound pressure Vt of the output sound signal SNt, and the voice recognition rate of the AI assistant 81 that receives the output sound signal SNt is stably maintained.

When the earphones 91, 91A to 91C are sold as products, they may be sold not only as single items but also as a set of two or more. If the earphones 91, 91A to 91C are designed to be wearable on both the left and right ears, they may be sold as a set of two for the left and right ears.
Furthermore, the earphones 91, 91A to 91C may be sold as a set of any number of earphones with a microphone for one ear to be worn by multiple employees of a large store.

Example 2
Second Embodiment A voice input system according to an embodiment of the present invention will be described below using an earphone system 91ST according to a second embodiment with reference mainly to FIGS.
Fig. 9 is a block diagram of the earphone system 91ST, and Fig. 10 is a table showing the operation of the earphone system 91ST. As shown in Fig. 9, the earphone system 91ST is configured as a pair of an earphone 91L which is a first voice input device and an earphone 91R which is a second voice input device. The earphone 91L is worn on the left ear of a speaker H, and the earphone 91R is worn on the right ear of the speaker H.

As shown in FIG. 1, the earphone 91L has a main body 1L and an insertion portion 2, and the earphone 91R has a main body 1R and an insertion portion 2.
In the earphones 91L and 91R, the configurations and positions of the first to third microphones M1 to M3, the driver 5, and the speaker unit 6 are the same as those of the earphones 91 of the first embodiment.
Hereinafter, the same reference numerals will be used to designate the same components as those in the earphone 91, and different components will be distinguished by adding L and R to the end of the reference numerals, respectively.

As shown in FIG. 1 and FIG. 9, earphone 91L and earphone 91R have control units 3L and 3R, respectively, instead of control unit 3 of earphone 91, and communication units 4L and 4R, respectively, instead of communication unit 4 of earphone 91.

In the earphone 91L, the main body 1L has a first microphone M1, a second microphone M2, a control unit 3L, a communication unit 4L, a drive unit 5, and a speaker unit 6. The insertion portion 2 has a third microphone M3.
In the earphone 91R, the main body 1R has a first microphone M1, a second microphone M2, a control unit 3L, a communication unit 4L, a drive unit 5, and a speaker unit 6. The insertion portion 2 has a third microphone M3.

As shown in FIG. 1, the main body 1L and the main body 1R have an air chamber 1a on the sound output side of the speaker unit 6.
The insertion portion 2 has a sound emission path 2a that opens at the tip and communicates with the air chamber 1a.
When the earphones 91L and 91R are in use, sound output from the speaker unit 6 by the operation of the drive unit 5 passes through the air chamber 1a and the sound output path 2a and is output into the external ear canals E1 of each of the left and right ears.
As a result, the earphones 91L, 91R can receive audio signals wirelessly transmitted from an external audio playback device via the communication units 4L, 4R, respectively, and play the signals in the speaker unit 6 via the control units 3L, 3R and drive unit 5.
The earphones 91L and 91R are capable of communicating with each other via the communication units 4L and 4R.

The first microphone M1 provided on each of the main bodies 1L and 1R is disposed at a portion of the main body 1L or 1R farther from the mouth of the speaker H, and picks up sounds around the main body 1L or 1R.
The second microphone M2 provided in each of the main bodies 1L and 1R is disposed at a portion of the main bodies 1L and 1R that is closer to the mouth of the speaker H. In other words, when the earphones 91L and 91R are in use, the second microphone M2 is located closer to the mouth of the speaker H than the first microphone M1.
The third microphone M3 is an air conduction sound microphone and is arranged at a third position facing the sound emission path 2a of the insertion portion 2. The third microphone M3 picks up air conduction sound generated when the sound emitted by the speaker H reaches the external auditory canal E1 as bone conduction sound and reverberates in the internal space Ev of the external auditory canal E1 and the sound emission path 2a.
That is, the first position of the first microphone M1 is outside the ear canal of the speaker H. The second position of the second microphone M2 is outside the ear canal of the speaker H and closer to the mouth of the speaker H than the first position. The third microphone M3 is inside the ear canal of the speaker H.

As shown in FIG. 9, a control unit 3L of an earphone 91L has a sound pressure detection unit 3aL, an input selection unit 3bL, and a sound pressure difference evaluation unit 3d.
The control unit 3R of the earphone 91R has a sound pressure detection unit 3aR, an input selection unit 3bR, and an output control unit 3e.

In the earphone 91L, the sound pressure detector 3aL detects the sound pressure of the audio signal SN1L received from the first microphone M1, and outputs it as a detected first audio signal SNL to both the input selector 3bL and the sound pressure difference evaluator 3d.
In the earphone 91R, the sound pressure detector 3aR detects the sound pressure of the audio signal SN1R received from the first microphone M1, and outputs it as a detected first audio signal SNR to both the input selector 3bR and the output controller 3e.

The sound pressure of the audio signals SN1L and SN1R is detected, for example, as an equivalent noise level (LAeq). The sound pressures of the detected first audio signals SNL and SNR detected by the sound pressure detectors 3aL and 3aR as equivalent noise levels (LAeq) are hereinafter referred to as sound pressures VL and VR, respectively.

The first microphone M1 provided in the main body 1L picks up ambient sound around the main body 1L, and the first microphone M1 provided in the main body 1R picks up ambient sound around the main body 1R.
Therefore, the sound pressure VL can be regarded as the sound pressure of the ambient sound of the left earphone 91L, and the sound pressure VR can be regarded as the sound pressure of the ambient sound of the right earphone 91R.

The output control unit 3e of the earphone 91R sends sound pressure information JR1, which includes the sound pressure VR of the incoming detected first audio signal SNR, and communication control information JR2 (details of which will be described later), to the communication unit 4R.

The input sound processing method by the operation of the input selection units 3bL and 3bR of the earphones 91L and 91R is similar to that of the input selection unit 3b of the first embodiment.
3, the input selection unit 3bL of the earphone 91L selects and sets the input audio signal SN2L as the output audio signal SNtL when the sound pressure VL of the detected first audio signal SNL output from the sound pressure detection unit 3aL is less than at least a preset lower switching sound pressure Va1, and selects and sets the input audio signal SN3L as the output audio signal SNtL when the sound pressure VR exceeds at least an upper switching sound pressure Va2.
The input selection unit 3bL outputs the output audio signal SNtL set as described above to the communication unit 4L.
In this manner, the input selection unit 3bL sets the reflection degrees RF1L and RF2L of the input sound signals SN2L and SN3L in the output sound signal SNtL, respectively, based on the sound pressure Va of the detected first sound signal SN1a.
In this example, the input selection unit 3bL sets the reflection degree RF1L and the reflection degree RF2L as an alternative between one being reflected and the other not being reflected.

In addition, the input selection unit 3bR of the earphone 91R selects and sets the input audio signal SN2R as the output audio signal SNtR when the sound pressure VR of the detected first audio signal SNR output from the sound pressure detection unit 3aR is less than at least the preset lower switching sound pressure Va1, and selects and sets the input audio signal SN3R as the output audio signal SNtR when the sound pressure VR exceeds at least the upper switching sound pressure Va2.
The input selection unit 3bR outputs the output audio signal SNtR set as described above to the communication unit 4R.
In this manner, the input selection unit 3bR sets the reflection degrees RF1R and RF2R of the input sound signals SN2R and SN3R in the output sound signal SNtR, respectively, based on the sound pressure Va of the detected first sound signal SN1a.
In this example, the input selection unit 3bR sets the reflection degree RF1R and the reflection degree RF2R as an alternative between one being reflected and the other not being reflected.

The communication unit 4R wirelessly transmits the sound pressure information JR1 received from the output control unit 3e to the outside of the earphone 91R. The wireless transmission method is, for example, Bluetooth (registered trademark).
Here, whether or not the communication section 4R wirelessly transmits the output audio signal SNtR output from the input selection section 3bR is controlled by communication control information JR2 input from the output control section 3e.
That is, the communication control information JR2 includes a command to either permit or prohibit wireless transmission of the output audio signal SNtR, and based on this command, the wireless transmission of the output audio signal SNtR in the communication unit 4R is controlled.

The communication unit 4L of the earphone 91L receives the sound pressure information JR1 wirelessly transmitted from the communication unit 4R of the earphone 91R, and sends it to the sound pressure difference evaluation unit 3d.
The sound pressure difference evaluation unit 3d acquires the sound pressure VR from the sound pressure information JR1 sent from the communication unit 4L, and compares the sound pressure VR with the sound pressure VL of the detected first sound signal SNL acquired from the sound pressure detection unit 3aL.

The sound pressure difference evaluation unit 3d sets one of the output sound signal SNtL and the output sound signal SNtR, which are preset in accordance with the magnitude relationship between the sound pressures VL and VR, as an output sound signal SNst to be wirelessly transmitted to the outside as the earphone system 91ST.
Next, the sound pressure difference evaluation unit 3d outputs communication control information JL2 that identifies the signal set in the output sound signal SNtL to the communication unit 4L, and the communication unit 4L wirelessly transmits the communication control information JL2 to the communication unit 4R of the earphone 91R.
The communication unit 4R receives the communication control information JL2 and sends the received communication control information JL2 to the output control unit 3e.

The operation of the sound pressure difference evaluation unit 3d will be described in detail with reference to FIG.
FIG. 10 is a table showing the relationship between the magnitude of the sound pressure VL and the sound pressure VR and the output audio signal SNst wirelessly transmitted to the outside as the earphone system 91ST.
As shown in FIG. 10, when the sound pressure difference evaluation unit 3d determines that the sound pressure VR is greater than the sound pressure VL, it sets the output sound signal SNtL to the output sound signal SNst to be wirelessly transmitted by the earphone system 91ST.

Moreover, the sound pressure difference evaluation unit 3d sends the communication control information JL2 to the communication unit 4L, including an instruction to the communication unit 4L to wirelessly transmit the output sound signal SNtL.
When the sound pressure difference evaluation unit 3d determines that the sound pressure VR is smaller than the sound pressure VL, it sends the communication control information JL2 to the communication unit 4L including an instruction to the communication unit 4L to stop wireless transmission of the output sound signal SNtL.

The communication unit 4L transmits the communication control information JL2 to the communication unit 4R, and executes or stops wireless transmission of the output audio signal SNtL based on an instruction for the communication unit 4L contained in the communication control information JL2.
On the other hand, the communication section 4R receives the communication control information JL2 transmitted from the communication section 4L and sends it to the output control section 3e.

When the output control unit 3e causes the communication unit 4L to wirelessly transmit the output audio signal SNtL based on the communication control information JL2, the output control unit 3e generates the communication control information JR2 including an instruction to stop wireless transmission of the output audio signal SNtR in the communication unit 4R.
Furthermore, when the output control unit 3e does not want the communication unit 4L to wirelessly transmit the output audio signal SNtL, the output control unit 3e generates communication control information JR2 including an instruction to wirelessly transmit the output audio signal SNtR in the communication unit 4R.
The output control unit 3e sends the generated communication control information JR2 to the communication unit 4R.
The communication unit 4R executes or stops wireless transmission of the output audio signal SNtR based on the communication control information JR2 sent from the output control unit 3e.

As is apparent from the above-described audio input device and input audio processing method, the earphone system 91ST selects the output audio signal of the earphone 91L, 91R, whichever has the smaller ambient sound, and wirelessly transmits it to the outside.
As a result, the earphone system 91ST improves the recognition rate of the AI assistant 81 for the voice uttered by the speaker H.

The second embodiment described above is not limited to the above-mentioned configuration and procedure, and may be modified without departing from the spirit and scope of the present invention.

As described in Modification 1 of the first embodiment, the earphones 91L and 91R may have, instead of the input selection units 3bL and 3bR, input mixing units 3cL and 3cR (see FIG. 9 ) that perform the same operation as the input mixing unit 3c.
For example, in the input mixing unit 3cL, the input audio signal SN2L and the input audio signal SN3 are mixed at a sound pressure ratio of the reflection degrees FR2L and FR3L, respectively, which correspond to the sound pressure Va, and the ratio of the mixed sound pressures gradually changes linearly in response to the increase or decrease in the sound pressure of the ambient sound of the main body unit 1.
The reflection degree RF1L and the reflection degree RF2L are indexes indicating the degree of reflection of the input audio signal SN2L and the input audio signal SN3L to the output audio signal SNtL, respectively. The indexes are, for example, the magnitude of sound pressure. Therefore, the sound pressure ratio is the ratio of the sound pressure of the input audio signal SN2L and the input audio signal SN3L contained in the output audio signal SNtL.
For example, as shown in FIG. 5, the reflection degree FR2L in the output audio signal SNtL is represented by Vmax/Vmin when the sound pressure Va is Va3, V2x/V3x when the sound pressure Vax, and Vmin/Vmax when the sound pressure Va4.
The reflection degree FR3 in the output audio signal SNtL is expressed as Vmin/Vmax when the sound pressure Va is the sound pressure Va3, as V3x/V2x when the sound pressure Vax, and as Vmax/Vmin when the sound pressure Va4.
When the earphones 91L and 91R have the input mixing units 3cL and 3cR instead of the input selection units 3bL and 3bR, respectively, the change in sound quality of the output audio signal SNst in response to an increase or decrease in the ambient sound becomes gradual and smooth.
This maintains a high recognition rate by the AI assistant 81 of the voice uttered by the speaker H, regardless of the sound pressure of the surrounding sound of the main body unit 1L or the main body unit 1R.
In addition, since the earphone 91L and the earphone 91R have the input mixing units 3cL and 3cR, respectively, the total sound pressure of the output audio signal SNst remains constant and does not change suddenly regardless of an increase or decrease in the ambient sound, so that the recognition rate by the AI assistant 81 of the voice uttered by the speaker H is maintained at a higher level.

In the case where earphone 91L and earphone 91R have input mixing unit 3cL and input mixing unit 3cR, respectively, the earphone system 91ST can apply variants 2 to 4 of Example 1 to the extent possible.

The communication method used by the communication units 4, 4L, and 4R is not limited to the above-mentioned Bluetooth (registered trademark), and various methods can be applied. Furthermore, the communication units 4, 4L, and 4R are not limited to wireless communication with the outside world, and may be wired.

In the earphones 91, 91A to 91D, 91L, and 91R that are the voice input devices of the first and second embodiments, the third microphone M3 is not limited to the air conduction sound microphone described above, and may be a bone conduction microphone that picks up bone conduction sound.
Fig. 11 is a diagram illustrating an example of the arrangement position of the third microphone M3 when the third microphone M3 is a bone conduction microphone. As shown in Fig. 11, the third microphone M3 is a bone conduction microphone, and when the insertion portion 2 is inserted into the ear canal E1, the third microphone M3 is located at a third position where it is in close contact with the inner surface of the ear canal E1, and picks up the bone conduction sound of the voice uttered by the speaker H.

In the voice input system 91ST of the second embodiment, the use of the earphone 91L as the first voice input device and the earphone 91R as the second voice input device is not limited to being worn on one ear and the other ear of the speaker H. For example, the earphone 91L may be worn on the ear of the first speaker, and the earphone 91R may be worn on the ear of a second speaker other than the first speaker.

1 Main body 1a Air chamber 2 Insertion section 2a Sound emission path 3, 3L, 3R Control section 3a, 3aL, 3aR Sound pressure detection section 3b, 3bL, 3bR Input selection section 3c, 3cB, 3cC, 3cD Input mixing section 3d Sound pressure difference evaluation section 3e Output control section 4, 4L, 4R Communication section 5 Drive section 6 Speaker unit 81 AI assistant 91, 91A, 91B, 91C, 91D, 91L, 91R Earphones (voice input device)
91ST Earphone System (Audio Input System)
E Pinna E1 Ear canal Ev Internal space FR2, FR3 Degree of reflection H Speaker JR1 Sound pressure information JL2, JR2 Communication control information M1 First microphone M2 Second microphone M3 Third microphone LN2b, LN3b, LN2a, LN3a Characteristic line R Mixing range SN1 to SN3, SN1L to SN3L, SN1R to SN3R Input audio signal SN1a, SNL, SNR Detected first audio signal SNt, SNtL, SNtR, SNst Output audio signal V, V2x, V3x Mixed sound pressure Va, Vax, Va5, Va6 Sound pressure Va1 Switching lower sound pressure Va2 Switching upper sound pressure Va3 Mixed lower limit sound pressure Va4 Mixed upper limit sound pressure Vmax Mixed maximum sound pressure Vmin Mixed minimum sound pressure VL, VR Sound pressure Vt, Vt1, Vt2, Vtc Total sound pressure V2max, V3max Maximum mixed sound pressure

Claims (7)

a first microphone that picks up a voice at a first position outside the ear canal of a speaker and outputs a first input voice signal;
a second microphone that picks up a voice at a second position outside the ear canal of the speaker that is closer to the mouth of the speaker than the first position and outputs a second input voice signal;
a third microphone that picks up a voice in the ear canal of the speaker and outputs a third input voice signal;
a control unit that detects a sound pressure of the first input audio signal, sets a reflection degree of each of the second input audio signal and the third input audio signal in accordance with the detected sound pressure, and generates an output audio signal including at least one of the second input audio signal and the third input audio signal based on the reflection degree;
a communication unit that transmits the output audio signal to an external device;
The audio input device includes a first audio input device and a second audio input device that can communicate with each other,
The control unit of the first voice input device
An audio input system that determines whether the sound pressure of the first input audio signal in the first audio input device is larger than the sound pressure of the first input audio signal in the second audio input device, and sets the output audio signal to be transmitted from the communication unit to the outside based on the determination result. In the first voice input device and the second voice input device,
The control unit selects the reflection degree as either "reflected" or "not reflected,"
2. The voice input system according to claim 1, wherein one of the second input voice signal and the third input voice signal is selectively set as an output voice signal in response to the detected sound pressure. In the first voice input device and the second voice input device,
The control unit is
When a sound pressure of the detected first input sound signal is smaller than a preset first sound pressure, the second input sound signal is set as the output sound signal;
3. The voice input system according to claim 2, wherein the third input voice signal is set as the output voice signal when the sound pressure of the first input voice signal exceeds a second sound pressure that is greater than the first sound pressure. In the first voice input device and the second voice input device,
The control unit is
The reflection degree is a sound pressure ratio,
2. The voice input system according to claim 1, wherein the second input voice signal and the third input voice signal are mixed at the sound pressure ratio according to the detected sound pressure, and the result is set as an output voice signal. The control unit of the first voice input device
The second input audio signal and the third input audio signal,
When the detected sound pressure of the first input audio signal is smaller than a preset first sound pressure, the sound pressure of the second input audio signal is mixed to be greater than the sound pressure of the third input audio signal;
When the sound pressure of the detected first input audio signal exceeds a second sound pressure that is greater than the first sound pressure, a sound pressure of the third input audio signal is mixed to be greater than a sound pressure of the second input audio signal;
5. The voice input system according to claim 4, characterized in that, when the detected sound pressure of the first input voice signal is in a range equal to or greater than the first sound pressure and equal to or less than the second sound pressure, the sound pressure is mixed so that the ratio of the sound pressure of the third input voice signal to the sound pressure of the second input voice signal increases as the sound pressure increases. acquiring a first input audio signal based on a sound picked up at a first position outside the ear canal of a speaker, and detecting a sound pressure of the first input audio signal;
acquiring, as a second input audio signal, a sound picked up at a second position outside the ear canal of the speaker that is closer to the mouth of the speaker than the first position;
Acquiring a third input audio signal based on a sound picked up in the ear canal of the speaker;
a pair of audio input devices, a first audio input device and a second audio input device, which selectively set one of the second input audio signal and the third input audio signal as an output audio signal in response to a sound pressure of the first input audio signal and transmit the output audio signal to an outside;
The first audio input device and the second audio input device are attached to the left ear and the right ear, respectively;
An input audio processing method that determines whether the sound pressure of the first input audio signal in the first audio input device is larger than the sound pressure of the first input audio signal in the second audio input device, and sets the output audio signal to be transmitted from a communication unit to the outside based on the determination result. acquiring a first input audio signal based on a sound picked up at a first position outside the ear canal of a speaker, and detecting a sound pressure of the first input audio signal;
acquiring, as a second input audio signal, a sound picked up at a second position outside the ear canal of the speaker that is closer to the mouth of the speaker than the first position;
Acquiring a third input audio signal based on a sound picked up in the ear canal of the speaker;
a first audio input device and a second audio input device, which are a pair of audio input devices, mix the second input audio signal and the third input audio signal at a sound pressure ratio corresponding to the sound pressure of the first input audio signal, set the output audio signal, and transmit the output audio signal to an outside;
The first audio input device and the second audio input device are attached to the left ear and the right ear, respectively;
An input audio processing method that determines whether the sound pressure of the first input audio signal in the first audio input device is larger than the sound pressure of the first input audio signal in the second audio input device, and sets the output audio signal to be transmitted from a communication unit to the outside based on the determination result.

Images