JP7257834B2 - Speech processing device, speech processing method, and speech processing system - Google Patents

Description

The present disclosure relates to an audio processing device, an audio processing method, and an audio processing system.

A phenomenon (howling) is known in which an offensive noise is generated when a microphone picks up sound output from a speaker. Howling interferes with listening to voice. Therefore, various devices and methods for suppressing howling have been proposed.

For example, the howling suppression device of Patent Document 1 is used in a karaoke machine. The howling suppression device of Patent Literature 1 switches the analog dry sound path to the digital dry sound path during a period in which howling is detected.

JP 2018-056893 A

However, in the technique of Patent Document 1, the voice input to the microphone is the voice uttered by a person present on the spot, or a sound other than the utterance of the speaker (for example, noise, voice via telephone, etc.). does not distinguish between Therefore, in the technique disclosed in Patent Document 1, uniform audio processing is performed for both human voice in which howling is unlikely to occur and audio other than human voice. However, it is desirable that the audio processing for applying acoustic effects such as howling suppression effects is controlled according to the speaking state of the speaker (for example, whether the speaker is speaking or not).

An object of the present disclosure is to provide a speech processing device, a speech processing method, and a speech processing system that control speech processing for applying sound effects according to the utterance state of a speaker.

The sound processing device of the present disclosure includes a microphone that outputs a sound signal based on sound input from the outside, a concentration measurement unit that measures the carbon dioxide concentration, and a sound effect that is applied to the sound signal based on the carbon dioxide concentration. and an audio signal processing unit that applies the sound effect to the audio signal based on the control signal.

The audio processing method of the present disclosure includes the steps of outputting an audio signal based on an externally input audio, measuring a carbon dioxide concentration, and controlling a sound effect applied to the audio signal based on the carbon dioxide concentration. and applying the sound effect to the audio signal based on the control signal.

A sound processing system of the present disclosure includes a sound collecting device that collects sound and a sound output device. The sound collecting device includes a microphone that outputs a sound signal based on sound input from the outside, and a carbon dioxide concentration and an audio signal control unit that generates a control signal for controlling a sound effect applied to the audio signal based on the carbon dioxide concentration, the sound collecting device and / or the audio output The device comprises an audio signal processing unit that applies the sound effect to the audio signal based on the control signal, and the audio output device includes an audio output unit that outputs the audio signal to which the sound effect has been applied as sound. Prepare.

Advantageous Effects of Invention According to the present disclosure, it is possible to provide a speech processing device, a speech processing method, and a speech processing system that control speech processing for applying sound effects according to the utterance state of a speaker.

1 is a schematic configuration diagram showing an example of a speech processing device according to a first embodiment of the present invention; FIG. 2 is a flowchart for explaining the operation of the speech processing device of FIG. 1; FIG. 2 is a diagram showing a loudspeaker equipped with an example of the audio processing device of FIG. 1; FIG. FIG. 5 is a schematic configuration diagram showing an example of a speech processing device according to a second embodiment of the present invention; 5 is a flowchart for explaining the operation of the audio processing device of FIG. 4; FIG. 1 is a schematic configuration diagram showing an example of a speech processing system according to an embodiment of the present invention; FIG.

Embodiments of the present disclosure will be described below with reference to the drawings.

(sound processing device)
(First embodiment)
First, the first embodiment will be explained. FIG. 1 is a schematic configuration diagram showing an example of a speech processing device 10 according to this embodiment. The speech processing device 10 performs speech processing on speech signals in order to apply sound effects according to the utterance state of the speaker. Here, the speech processing device 10 detects the user's exhalation and determines whether or not the input information is the user's natural voice. The speech processing device 10 changes speech processing to be executed according to the determination result of the input information.

The audio processing device 10 includes a microphone 20 , a density measuring section 30 , an audio signal control section 40 and an audio signal processing section 50 . The audio processing device 10 may further include an audio output unit 60 .

For example, the audio processing device 10 may be installed in a loudspeaker (see FIG. 3), a handheld microphone, a hands-free communication device in a car, or the like.

(microphone)
The microphone 20 outputs an audio signal based on a sound input from the outside of the audio processing device 10 to the audio signal processing unit 50 . The microphone 20 is not particularly limited as long as it can acquire sound, and may be, for example, a condenser microphone, a piezoelectric microphone, or a dynamic microphone.
The microphone 20 may be arranged inside the audio processing device 10, and the microphone 20 and the outside of the audio processing device 10 may be communicated through an opening of the audio processing device 10, or may be exposed from the audio processing device 10. It may be arranged in a state where
Here, the speech processing device 10 acquires input information input from the outside. Input information includes speech and breath. The microphone 20 acquires the voice of the input information. Speech may include user-generated voice, noise, voice over the phone, and the like.

(Concentration measuring part)
The concentration measurement unit 30 measures the concentration of carbon dioxide in the air, which changes based on the exhaled breath among the input information. The concentration measuring unit 30 is not limited as long as it can measure the carbon dioxide concentration, and for example, a gas sensor using a non-dispersive infrared analysis method can be used. Further, the carbon dioxide concentration detection method in the concentration measurement unit 30 may be either absolute value detection or relative value detection.
The arrangement of the concentration measurement unit 30 in the sound processing device 10 is not particularly limited, but for example, when the microphone 20 is arranged inside the sound processing device 10, the concentration measurement unit 30 can be placed at the opening of the sound processing device 10. It can be installed near the part. Further, when the microphone 20 is arranged in a state exposed from the sound processing device 10, the density measuring section 30 can be provided near the exposed microphone 20. FIG. The concentration of carbon dioxide in the air measured by the concentration measurement unit 30 changes when air containing exhalation accompanying the user's vocalization is taken into the voice processing device 10 through the opening, is blown into the microphone 20 and changes. When the user speaks into the microphone 20, the concentration measuring unit 30 allows the microphone 20 and the concentration measuring unit 30 to be separated if the carbon dioxide concentration can be measured by the concentration measuring unit 30. be done.
The concentration measurement unit 30 may operate constantly or intermittently at appropriate intervals (for example, 1 second).

(Audio signal control unit)
The audio signal control section 40 acquires the carbon dioxide concentration measured by the concentration measurement section 30 . Also, the audio signal control unit 40 generates a control signal for controlling the acoustic effect applied to the audio signal based on the acquired carbon dioxide concentration. The signal generation by the audio signal control unit 40 is realized by an arithmetic processing device such as a microprocessor, a microcontroller, or a CPU (Central Processing Unit).
The audio signal control unit 40 determines whether or not the user is speaking based on the carbon dioxide concentration. Details of the determination will be described later. The audio signal control unit 40 generates different control signals depending on the determination result, that is, whether or not the user is speaking. The generated control signal is output to the audio signal processing section 50 . In this embodiment, the audio signal control unit 40 generates the first control signal when determining that the user is speaking. Also, the audio signal control unit 40 generates a second control signal when it is determined that the user does not speak.

(Audio signal processor)
The audio signal processing section 50 applies sound effects to the audio signal acquired from the microphone 20 based on the control signal acquired from the audio signal control section 40 . As described above, the audio signal control unit 40 generates different control signals depending on whether the user is speaking or not. The audio signal processing unit 50 can apply sound effects to the audio signal according to the control signal.
In this embodiment, the acoustic effect applied to the audio signal by the audio signal processing unit 50 can be a frequency filter effect. Here, the acoustic effect applied to the audio signal is not limited to the frequency filter effect. As another example, the audio signal processing unit 50 may apply acoustic effects such as noise removal to the audio signal. As still another example, the audio signal processing unit 50 may apply a sound effect (amplifier) for amplifying the audio signal. As another example, the audio signal processing unit 50 may apply sound effects such as so-called voice effects such as pitch correction processing and distortion processing to the audio signal.
The audio signal processing unit 50 is realized by an arithmetic processing device such as a microprocessor, a microcontroller, or a CPU (Central Processing Unit).

When the acoustic effect applied by the audio signal processing unit 50 is the above frequency filter effect, the audio signal processing unit 50 includes a first howling suppression filter and a second howling suppression filter. The audio signal processing unit 50 applies the first howling suppression filter to the audio signal when the first control signal (generated when it is determined that the user is speaking) is obtained. That is, the audio signal processing unit 50 applies the howling suppression effect in the first frequency band to the audio signal according to the first control signal. The first frequency band is, for example, a frequency band from 100 Hz to 1000 Hz, which is the main frequency band of human voice, or from 100 Hz to 1000 Hz, in which howling is likely to occur depending on the acoustic device used or the surrounding environment. When it is determined that the user is speaking, the audio signal processing unit 50 reduces the volume of the first frequency band with respect to the audio signal to suppress howling. Further, the audio signal processing unit 50 applies the second howling suppression filter to the audio signal when the second control signal (generated when it is determined that the user is not vocalizing) is acquired. do. That is, the audio signal processing unit 50 applies the howling suppression effect in the second frequency band to the audio signal according to the second control signal. The second frequency band is wider than the first frequency band. The second frequency band is, for example, within the human audible range of 20 Hz to 20000 Hz, in which howling that is likely to occur depends on the acoustic device used or the surrounding environment. When it is determined that the user does not speak, the audio signal processing unit 50 reduces the volume of the second frequency band with respect to the audio signal to suppress howling. Therefore, according to the audio processing device 10 of the present embodiment, it is possible to effectively suppress howling according to how likely it is to occur.
Here, as another example, the audio signal processing section 50 may comprise three or more howling suppression filters, each of which is applied according to the first control signal or the second control signal.
Further, if necessary, another sound effect may be applied to the audio signal to which the howling suppression effect has been applied. Specifically, for example, a sound effect that amplifies an audio signal to which the howling suppression effect has been applied may be applied.

Further, in the modified example of the first embodiment, when the acoustic effect applied by the audio signal processing unit 50 is an effect of amplifying the audio signal, the audio signal processing unit 50, when acquiring the first control signal, A sound effect for amplifying the signal is applied, and an effect for amplifying the audio signal is not applied when the second control signal is acquired. Alternatively, the audio signal processing unit 50 suppresses howling by stopping the output of the audio signal when the second control signal is acquired.

(Audio output part)
The audio output unit 60 outputs the audio signal to which the sound effect has been applied by the audio signal processing unit 50 as audio. The audio output unit 60 may be, for example, a speaker, and the audio output unit 60 is limited to this type as long as it can output the audio signal to which the sound effect has been applied by the audio signal processing unit 50 as audio. not something.

FIG. 2 is a flow chart showing an example of a processing procedure of the speech processing device 10. As shown in FIG. The speech processing device 10 implements the speech processing method by executing the processes described in the flowcharts.

The microphone 20 of the audio processing device 10 acquires audio (input audio) from input information input from the outside. Then, the microphone 20 outputs an audio signal to the audio signal processing section 50 according to the input audio (step S1).

The concentration measurement unit 30 of the speech processing device 10 acquires air containing exhaled breath from the input information. Then, the concentration measurement unit 30 measures the concentration of carbon dioxide in the air, which changes based on exhalation (step S2). Step S2 may be executed at timing preset by the user or at arbitrary timing designated by the user. The preset timing may be determined at regular intervals. Also, the preset timing may be when the voice processing device 10 is powered on. Also, the preset timing may be when the distance between the user, who is the speaker, and the microphone 20 fluctuates and the volume of the voice in the input information exceeds a preset threshold.

The audio signal control unit 40 of the audio processing device 10 determines that the user is speaking when the carbon dioxide concentration measured by the concentration measurement unit 30 is equal to or higher than a preset threshold. Also, when determining that the carbon dioxide concentration is lower than the preset threshold value, the audio signal control unit 40 determines that the user does not speak (step S3). In this embodiment, the threshold value is a fixed value and is stored in the audio signal control section 40 .

When it is determined that the user is speaking (YES in step S3), the audio signal processing unit 50 of the audio processing device 10 processes the audio signal with the first howling suppression filter (step S4).

When it is determined that the user does not speak (NO in step S3), the audio signal processing unit 50 of the audio processing device 10 processes the audio signal with the second howling suppression filter (step S5).

When the user is not speaking into the microphone 20, voice such as noise having a wider frequency band than the human voice is input to the microphone 20. - 特許庁At this time, if the audio signal of the input audio is not processed by the second howling suppression filter, howling may occur in the human audible range of 20 Hz to 20000 Hz. Also, when the user speaks into the microphone 20, the volume of the main frequency band of human voice from 100 Hz to 1000 Hz is relatively louder than the volume of other frequency bands. Therefore, howling is less likely to occur in a frequency band other than 100 Hz to 1000 Hz. At this time, if the audio signal of the input audio were to be processed by the second howling suppression filter, the volume of the frequency band in which howling is unlikely to occur would also be reduced at the same time. As a result, the sound quality of the audio signal output from the audio signal processing unit 50, that is, the audio signal to which the sound effect has been applied by the audio signal processing unit 50 is deteriorated. Therefore, when the user speaks into the microphone 20, if the audio signal is processed by the first howling suppression filter that reduces the volume only in the frequency band from 100 Hz to 1000 Hz where howling is likely to occur, the sound quality deteriorates. howling can be suppressed without

After step S4 or step S5 is executed, the audio signal output from the audio signal processing unit 50 is used to generate output audio by the audio output unit 60 (step S6).
Note that after step S4 or step S5 is executed and before step S6, an acoustic effect for amplifying the audio signal may be further applied to the audio signal to which the howling suppression effect has been applied.

(Second embodiment)
Next, a second embodiment will be described. FIG. 4 is a schematic configuration diagram showing an example of the speech processing device 10 in this embodiment. The speech processing device 10 according to the present embodiment further includes a density storage unit 100 in addition to the configuration of the speech processing device 10 according to the first embodiment. The density storage unit 100 stores a threshold value used by the audio signal control unit 40 to determine the vocalization state of the user.

The concentration of carbon dioxide in human exhaled breath is usually about 4%, but since emitted exhaled breath diffuses in the air, the carbon dioxide concentration decreases as the distance from the mouth increases. Also, the carbon dioxide concentration in fresh air is 0.04% or less, but the carbon dioxide concentration in indoor air with poor ventilation is about 0.4%. Thus, the carbon dioxide concentration varies depending on the distance between the user and the concentration measurement unit 30 and the usage environment. For example, when the concentration of carbon dioxide measured by the concentration measuring unit 30 is about 0.4% due to poor ventilation in the room, comparison with a fixed threshold may erroneously determine that it is the user's utterance. Therefore, it is possible to improve the accuracy of detection by varying the carbon dioxide concentration threshold for determining whether the user is speaking or not according to the usage environment.

The audio signal control unit 40 compares the concentration of carbon dioxide in the air measured by the concentration measurement unit 30 with the threshold value stored in the concentration storage unit 100 to determine whether or not the user is speaking. . The carbon dioxide concentration threshold value stored in the concentration storage unit 100 is the carbon dioxide concentration measured by the concentration measurement unit 30 or a value obtained by calculating the carbon dioxide concentration measured by the concentration measurement unit 30 . For example, the value obtained by calculating the carbon dioxide concentration measured by the concentration measurement unit 30 is measured for a certain period of time while the speaker, who is the user, is using the device 10, and the minimum carbon dioxide concentration during that period is may be used as the threshold value, or the measurement may be performed for a certain period of time without the user, the speaker, and the maximum carbon dioxide concentration during that period may be used as the threshold value. For example, the calculated value of the carbon dioxide concentration measured by the concentration measuring unit 30 may be a value obtained by multiplying the carbon dioxide concentration by a predetermined ratio. It may be possible to determine whether or not the user is speaking even if the user is away from the user.

The threshold value stored in the density storage unit 100 is updated at preset timing or arbitrary timing designated by the user. The preset timing may be determined at regular intervals. Also, the preset timing may be when the voice processing device 10 is powered on. Also, the preset timing may be when the distance between the user, who is the speaker, and the microphone 20 fluctuates and the volume of the voice in the input information exceeds a preset threshold. The preset timing may be when the distance between the user and the microphone 20 fluctuates and the carbon dioxide concentration measured by the concentration measuring unit 30 among the input information exceeds a preset threshold value. Also, the preset timing may be when the carbon dioxide concentration in the usage environment fluctuates and the carbon dioxide concentration within a certain range is measured by the concentration measuring unit 30 for a certain period of time or longer. By updating the threshold, even if there is a change in the distance between the sound processing device 10 and the user's mouth and a change in the carbon dioxide concentration in the usage environment, the sound signal control unit 40 can accurately detect the user's A vocalization state can be determined. For example, when a carbon dioxide concentration of about 0.4% is continuously measured by the concentration measuring unit 30 for one minute or more, it is determined that the carbon dioxide concentration in the air in the usage environment is about 0.4%, The carbon dioxide concentration may be measured at the timing when the user is speaking or at any timing specified by the user, and a carbon dioxide concentration threshold may be determined based on which the user's speaking state is determined. .

(Concentration storage unit)
The concentration storage unit 100 is not particularly limited as long as it can store a carbon dioxide concentration threshold for determining the vocalization state of the user. For example, main storage devices such as semiconductor memories typified by DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory), magnetic disks typified by HDD (Hard Disk Drive), CD-ROMs and DVD- Auxiliary storage devices such as optical discs represented by ROMs, USB memories, various memory cards, and flash memories mounted on SSDs (Solid State Drives) can be applied.

FIG. 5 is a flowchart showing an example of a processing procedure of the speech processing device 10. As shown in FIG. The speech processing device 10 first determines whether or not it is time to update the carbon dioxide concentration threshold for determining the vocalization state of the user (step S11).

If it is time to update the carbon dioxide concentration threshold (YES in step S11), the carbon dioxide concentration threshold stored in the concentration storage unit 100 is deleted (step S12). On the other hand, if it is not the timing to update the carbon dioxide concentration threshold (NO in step S11), it is determined whether or not the carbon dioxide concentration threshold is stored (step S13).

Then, when the threshold value of the carbon dioxide concentration is stored in the concentration storage unit 100 (YES in step S13), the process corresponding to the carbon dioxide concentration measured by the concentration measurement unit 30 is performed as in the first embodiment. (Step S16 to Step S21). On the other hand, if the concentration storage unit 100 does not store a carbon dioxide concentration threshold value (NO in step S13), the concentration measurement unit 30 can store carbon dioxide at a preset timing or at an arbitrary timing designated by the user. Carbon concentration is measured (step S14). The preset timing may be when the user speaks into the microphone 20 and the sound volume of the input information exceeds a preset threshold.

Then, based on the carbon dioxide concentration measured in step S14, the carbon dioxide concentration threshold for determining the vocalization state of the user is stored in the concentration storage unit 100 (step S15).

By the way, in the speech processing device 10 according to the first embodiment and the second embodiment, each component and function of the speech processing device 10 may be rearranged. For example, part or all of the configuration and functions of the audio processing device 10, particularly the audio signal processing section 50, may be included in another device.

(Voice processing system)
Next, the voice processing system according to this embodiment will be described.
FIG. 6 is a schematic configuration diagram showing an example of the audio processing system 200 according to this embodiment. The audio processing system 200 performs audio processing on audio signals to suppress howling. Here, the speech processing system 200 detects the user's exhalation and determines whether or not the input information is the user's natural voice. The speech processing system 200 changes speech processing to be executed according to the determination result of the input information.
The audio processing system 200 may be installed, for example, in a karaoke system in a karaoke store, audio equipment in a concert hall, or the like.

The audio signal processing system in this embodiment includes a sound pickup device 210 that picks up sound and an audio output device 220 .
The sound collection device 210 also includes a microphone 20 that outputs an audio signal based on a sound input from the outside, a concentration measurement unit 30 that measures the carbon dioxide concentration, and a sound effect applied to the audio signal based on the carbon dioxide concentration. and an audio signal control unit 40 that generates a control signal for controlling the . Furthermore, the sound pickup device 210 and/or the audio output device 220 include an audio signal processing unit 50 that applies sound effects to the audio signal based on the control signal (in FIG. 6, the sound pickup device 210 includes the audio signal processing unit 50). ). The audio output device 220 also includes an audio output unit 60 that outputs an audio signal to which sound effects have been applied as audio.
In the present embodiment, the sound processing system 200 may further include a concentration storage unit 100 in which the sound collection device 210 stores the carbon dioxide concentration threshold.

In the present embodiment, the microphone 20, the concentration measurement unit 30 and the audio signal control unit 40 provided in the sound collection device 210, the concentration storage unit 100, the sound collection device 210 and/or the sound signal processing unit 50 provided in the sound output device 220, In addition, the audio output unit 60 included in the audio output device 220 can be the same as each component of the audio processing device 10 of the present embodiment described above.

Also, the sound collection device 210 and the audio output device 220 can transmit and receive information via the network 230, for example. Network 230 may include any communication path, for example wireless or wired.

As described above, according to the audio processing system 200 of the present embodiment, it is possible to effectively suppress howling according to how likely it is to occur.

Although the present invention has been described with reference to drawings and embodiments, it should be noted that those skilled in the art can easily make various changes and modifications based on this disclosure. Therefore, please note that these variations and modifications are included in the scope of the present invention. For example, the functions included in each means, each step, etc. can be rearranged so as not to be logically inconsistent, and it is possible to combine a plurality of means, steps, etc. into one or divide them. .
Furthermore, the present invention may be used for purposes other than howling suppression.

10 Audio processing device 20 Microphone 30 Density measurement unit 40 Audio signal control unit 50 Audio signal processing unit 60 Audio output unit 100 Density storage unit 200 Audio processing system 210 Sound pickup device 220 Audio output device 230 Network

Claims (5)

a microphone that outputs an audio signal based on audio input from the outside;
a concentration measuring unit that measures carbon dioxide concentration;
an audio signal control unit that generates a control signal for controlling a sound effect applied to the audio signal based on the carbon dioxide concentration;
an audio signal processing unit that applies the sound effect to the audio signal based on the control signal ,
The audio signal control unit determines whether or not the user is speaking based on the carbon dioxide concentration, generates the control signal according to whether the user is speaking,
The audio signal processing unit applies a howling suppression effect in a first frequency band to the audio signal when it is determined that the user is speaking, and it is determined that the user is not speaking. and applying a howling suppression effect to the audio signal in a second frequency band wider than the first frequency band . further comprising a concentration storage unit that stores the threshold value of the carbon dioxide concentration;
2. The speech processing apparatus according to claim 1 , wherein said speech signal control section compares said carbon dioxide concentration with said threshold to determine whether said user is speaking. 3. The audio processing device according to claim 1 , further comprising an audio output unit that outputs, as audio, the audio signal to which the sound effect has been applied. a step of outputting an audio signal based on an externally input audio;
measuring carbon dioxide concentration;
generating a control signal for controlling a sound effect applied to the audio signal based on the carbon dioxide concentration;
applying the sound effect to the audio signal based on the control signal ;
The step of generating the control signal includes determining whether or not the user is vocalizing based on the carbon dioxide concentration, generating the control signal according to whether the user is vocalizing,
The step of applying a sound effect applies a howling suppression effect in a first frequency band to the audio signal when it is determined that the user is speaking, and when it is determined that the user is not speaking. and applying a howling suppression effect to the audio signal in a second frequency band wider than the first frequency band . A sound processing system comprising a sound collecting device for collecting sound and a sound output device,
The sound collecting device includes a microphone that outputs an audio signal based on a sound input from the outside, a concentration measurement unit that measures the carbon dioxide concentration, and a sound effect applied to the audio signal based on the carbon dioxide concentration. and an audio signal control unit that generates a control signal to
The sound collecting device and/or the audio output device comprises an audio signal processing unit that applies the sound effect to the audio signal based on the control signal,
The audio output device includes an audio output unit that outputs the audio signal to which the sound effect has been applied as audio,
The audio signal control unit determines whether or not the user is speaking based on the carbon dioxide concentration, generates the control signal according to whether the user is speaking,
The audio signal processing unit applies a howling suppression effect in a first frequency band to the audio signal when it is determined that the user is speaking, and it is determined that the user is not speaking. and applying a howling suppression effect to the audio signal in a second frequency band wider than the first frequency band .

Images