JP2021135311A - Voice processing device and voice processing method - Google Patents

Abstract

To adaptively suppress acoustic crosstalk components caused by utterance voices of other speakers mixed in an utterance voice of a main speaker depending on the states of a plurality of speakers present in a closed space, and to improve the sound quality of the utterance voice of the main speaker.SOLUTION: An acoustic crosstalk suppression device comprises: a single talk detection part that detects a single talk state in which any one of a plurality of persons, including a main speaker, who are present in a closed space is uttering, on the basis of the voice signals collected by a plurality of microphones; a mixing ratio estimation part that estimates a mixing ratio indicating a ratio of a voice signal of a main speaker mixed in the voice signals of other persons on the basis of the sound pressure ratio of the voice signal collected in the single talk state of the main speaker and the sound pressure ratio of the voice signal collected in the single talk state of any other; and a determination part that determines the necessity of crosstalk components caused by utterances of the other persons mixed in the voice signal of the main speaker on the basis of the estimation result of the mixing ratio.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a voice processing device and a voice processing method.

In Patent Document 1, an arrangement pattern of occupants is assumed in advance as a situation in the vehicle interior, sound transmission characteristics are measured for each arrangement pattern, and each transmission characteristic obtained by the measurement and stored in a memory or the like is described. A sound removing device that estimates and removes sound contained in an audio signal output from a speaker is disclosed. According to this sound removal device, sound can be removed or suppressed as long as the arrangement of the occupants satisfies any of the arrangement patterns.

JP-A-2009-216835

In the configuration of Patent Document 1, only one microphone is arranged in front of the driver for the purpose of picking up the voice spoken by the driver, and the voice of the driver can be picked up at high sound pressure. In some cases, it may be difficult to pick up the voice of a passenger (that is, another occupant) in the same vehicle with the same microphone at a high sound pressure. This is because the location of the microphone is biased toward the driver, so the distance from the driver to the microphone and the distance from the passenger to the microphone are different. Therefore, even if the driver and the passenger want to suppress the voice signal of another speaker (for example, the passenger) included in the voice signal of the main speaker (for example, the driver) as a crosstalk component when the driver and the passenger speak at almost the same time. If the audio signals of other speakers are not picked up at high sound pressure, the effect of suppressing crosstalk does not appear, and the sound quality of the audio signals of the main speaker may deteriorate. This is because it is difficult for the driver's microphone to pick up the voice of another speaker (for example, a passenger) at a high sound pressure, and learning the filter coefficient of the adaptive filter for suppressing the voice signal of the passenger as a crosstalk component. Is difficult.

The present disclosure has been devised in view of the above-mentioned conventional situations, and depending on the situation of a plurality of speakers existing in a closed space, the sound produced by the uttered voices of other speakers that may be included in the uttered voices of the main speaker. It is an object of the present invention to provide a voice processing device and a voice processing method for adaptively suppressing a typical cross-talk component and improving the sound quality of the spoken voice of the main speaker.

The present disclosure is based on audio signals connected to a plurality of microphones arranged in the closed space and picked up by each of the plurality of microphones, and a plurality of persons including a main speaker existing in the closed space. The sound pressure ratio of the single talk detector that detects the single talk state in which one of them is speaking, the sound pressure ratio of the sound signal picked up by each of the plurality of microphones in the single talk state of the main speaker, and the above. Based on the sound pressure ratio of the sound signals picked up by each of the plurality of microphones in the single talk state of another person other than the main speaker, the sound signal of the main speaker is compared with the sound signal of the other person. Based on the mixing ratio estimation unit that estimates the mixing ratio indicating the proportion of the mixture and the estimation result of the mixing ratio, it is necessary to suppress the crosstalk component by the speech of the other person included in the voice signal of the main speaker. Provided is a sound processing device including a determination unit for determining whether or not to use the sound.

Further, the present disclosure is a voice processing method executed by a voice processing device connected to a plurality of microphones arranged in a closed space, and is based on a sound signal picked up by each of the plurality of microphones. , A single talk state in which any one of a plurality of persons including the main speaker existing in the closed space is speaking is detected, and the sound is picked up by each of the plurality of microphones in the single talk state of the main speaker. Based on the sound pressure ratio of the voice signal, and the sound pressure ratio of the voice signal picked up by each of the plurality of microphones in the single talk state of another person other than the main speaker, the voice of the other person. A mixing ratio indicating the ratio of the voice signal of the main speaker to the signal is estimated, and based on the estimation result of the mixing ratio, a cross due to the speech of the other person included in the voice signal of the main speaker is used. Provided is a sound processing method for determining the necessity of suppressing the talk component.

According to the present disclosure, the acoustic cross-talk component due to the utterance voice of another speaker that may be included in the utterance voice of the main speaker is adaptively suppressed according to the situation of a plurality of speakers existing in the closed space. It is possible to improve the sound quality of the utterance voice of the main speaker.

A block diagram showing a functional configuration example of the acoustic crosstalk suppression device according to the first embodiment. A flowchart showing an example of an acoustic crosstalk suppression operation procedure according to the first embodiment. A block diagram showing a functional configuration example of the acoustic crosstalk suppression device according to the second embodiment. A flowchart showing an example of an acoustic crosstalk suppression operation procedure according to the second embodiment. A block diagram showing a functional configuration example of the acoustic crosstalk suppression device according to the modified example of the second embodiment. A diagram showing an example of an image captured by an omnidirectional camera on which a sound pressure heat map is superimposed. Diagram showing an example of a situation where a microphone array is placed in the middle of a clerk and a customer FIG. 7 is a diagram illustrating an example of acoustic crosstalk suppression processing for voice collected by forming directivity in each direction of a store clerk and a customer in the situation of FIG. 7. A diagram showing an example of a situation where the microphone array is placed near the store clerk and away from the customer. FIG. 9 is a diagram illustrating an example of acoustic crosstalk suppression processing for voice collected by forming directivity in each direction of a store clerk and a customer in the situation of FIG.

(Background to this disclosure)
As a scene where the acoustic crosstalk suppression device is used, for example, a situation where two people talk with each other is assumed. The acoustic cross-talk suppressor, for example, as disclosed in Japanese Patent No. 6635394, when the voice spoken by one person includes the voice spoken by the other person as the cross-talk component, the acoustic cross-talk component is used. By generating a suppression signal for suppression (in other words, subtraction) and suppressing the suppression signal from the voice signal uttered by one of the persons, it is possible to output a voice signal in which the crosstalk component is suppressed. The situation where two people talk is, for example, a situation where a prison officer and a resident such as a criminal talk face-to-face in a prison, a situation where a clerk and a customer talk across a table in a store, etc. There is a situation where an employee and a boss discuss at a meeting, but it does not have to be limited to the above-mentioned situation. The content of the utterance may be recorded as a log, converted into text and saved, or the voice signal of the utterance may be input as a voice recognition process.

Below, the situation where the clerk and the customer interact in the store is shown as an example. The acoustic crosstalk suppression device is connected to each of a plurality of microphones arranged on a round table table installed in the store, for example, and uses a voice uttered by one of the clerk and the customer as the main speaker as the target sound. Suppresses the voices spoken by other speakers that are mixed with the main speaker's voice as disturbing sounds.

FIG. 7 is a diagram showing an example of a situation in which the microphone array mA is placed in the middle of the clerk hm1 and the customer hm2. The microphone array mA has a housing that accommodates a plurality of omnidirectional microphones, and each omnidirectional microphone picks up ambient sound. The sound picked up by the microphone array mA is directional in each direction of the clerk hm1 and the customer hm2 by a known method (for example, a beamforming process performed by a PC (not shown) connected to the microphone array mA). Is formed and audio output becomes possible. The microphone is not limited to the microphone array mA, and may be one or a plurality of omnidirectional microphones.

In FIG. 7, the distance from the microphone array mA to the clerk hm1 and the distance from the microphone array mA to the customer hm2 are almost equal, and the direction d1 from the microphone array mA to the clerk hm1 and the direction d2 from the microphone array mA to the customer hm2. However, when the microphone array mA is at substantially the same angle from the surface of the table on which the microphone array mA is placed, the microphone array mA can separate the voice of the clerk hm1 and the voice of the customer hm2 at a high ratio and collect the sound.

FIG. 8 is a diagram illustrating an example of acoustic crosstalk suppression processing for voice picked up by forming directivity in each direction of the clerk hm1 and the customer hm2 in the situation of FIG. 7. The microphone array mA has, for example, four omnidirectional microphone elements m1 to m4. Although not shown, the microphone array mA or the PC connected to the microphone array mA inputs the voice signal picked up by the microphone array mA and forms directivity in each direction of the clerk hm1 and the customer hm2 ( Output sound (performing beamforming processing). The voice V1 of the clerk hm1 and the voice V2 of the customer hm2, which are picked up by the four microphone elements m1 to m4, have a sound pressure ratio of 5: 5.

When directivity is formed in the direction d1 of the clerk hm1 by the beamforming process, the voice V1 of the clerk hm1 and the voice V2 of the customer hm2 are assumed to be, for example, 7: 3 in sound pressure ratio. Similarly, when directivity is formed in the direction d2 of the customer hm2 by the beamforming process, it is assumed that the voice V1 of the clerk hm1 and the voice V2 of the customer hm2 have, for example, a sound pressure ratio of 3: 7.

When the acoustic crosstalk suppression processing is performed using the voice signal of the voice V1 of the clerk hm1 after beamforming as the main signal and the voice signal of the voice V2 of the customer hm2 after the beamforming processing as a reference signal, after the crosstalk is suppressed. The voice V1 of the clerk hm1 and the voice V2 of the customer hm2 are, for example, 9: 1 in sound pressure ratio. Therefore, the voice V1 of the clerk hm1 is relatively emphasized. Similarly, when the acoustic crosstalk suppression processing is performed using the voice signal of the voice V1 of the clerk hm1 after the beamforming process as a reference signal and the voice signal of the customer hm2's voice V2 after the beamforming process as the main signal. The voice V1 of the clerk hm1 and the voice V2 of the customer hm2 after the cross talk is suppressed are, for example, 1: 9 in sound pressure ratio. Therefore, the voice V2 of the customer hm2 is emphasized. The voice recognition engine egg can accurately recognize both the voice V1 of the clerk hm1 and the voice V2 of the customer hm2 after suppressing the acoustic crosstalk.

FIG. 9 is a diagram showing an example of a situation in which the microphone array mA is placed at a position close to the clerk hm1 and away from the customer hm2. Normally, the microphone array mA is often placed on either side rather than being placed in the middle of the clerk hm1 and the customer hm2, or physically placed between the clerk hm1 and the customer hm2. Even if it is, the directivity characteristics may vary due to the influence of the spatial characteristics. Considering the former as an example, the distance from the microphone array mA to the clerk hm1 and the distance from the microphone array mA to the customer hm2 are significantly different. Therefore, there is a difference between the sound pressure of the audio signal of the clerk hm1 and the sound pressure of the audio signal of the customer hm2 received (picked up) by the microphone array mA (see FIG. 10). For example, as shown in FIG. 10, there is a difference so that the sound pressure ratio of the audio signals of the clerk hm1 and the customer hm2 is 7: 3 for each microphone constituting the microphone array mA. Therefore, unlike the situation shown in FIG. 7, the microphone array mA cannot separate the voice of the clerk hm1 and the voice of the customer hm2 at a high rate and cannot collect the sound. The microphone array mA may be attached to a human body or clothing. In this case, the voice of the person to whom the microphone array mA is attached is predominantly picked up, and cannot be further separated and picked up.

FIG. 10 is a diagram for explaining an example of acoustic crosstalk suppression processing for a sound collected by forming directivity in each direction of a clerk hm1 and a customer hm2 in the situation of FIG. The voice V1 of the clerk hm1 and the voice V2 of the customer hm2, which are picked up by the four microphone elements m1 to m4, have a sound pressure ratio of 5: 5.

When the directivity is formed in the direction d1 of the clerk hm1 by the beamforming process, the microphone array mA is arranged near the clerk hm1, so that the voice V1 of the clerk hm1 can be predominantly picked up. The voice V1 of the clerk hm1 and the voice V2 of the customer hm2 have, for example, a sound pressure ratio of 9: 1. On the other hand, when the directivity is formed in the direction d2 of the customer hm2 by beamforming, the microphone array mA is arranged far from the customer hm2, so that the voice V2 of the customer hm2 cannot be sufficiently picked up. The voice V1 of the clerk hm1 and the voice V2 of the customer hm2 have, for example, a sound pressure ratio of 4: 6.

In such a case, the audio signal of the voice V1 of the clerk hm1 after beamforming is used as a reference signal, and the audio signal of the voice V2 of the customer hm2 after beamforming is used as the main signal, and the acoustic crosstalk suppression process is performed. Since the voice of the signal clerk hm1 is clear, the crosstalk suppression performance is high. Therefore, the voice V2 of the customer hm2 is relatively sufficiently emphasized. The voice recognition engine egg can accurately recognize the voice V2 of the customer hm2.

On the other hand, when the acoustic crosstalk suppression process is performed using the voice signal of the clerk hm1's voice V1 after beamforming as the main signal and the voice signal of the customer hm2's voice V2 after beamforming as the reference signal, the voice of the clerk hm1 is performed. Since the sound pressure ratio of V1 and the voice V2 of the customer hm2 is almost the same as 4: 6, the performance of the acoustic crosstalk suppression processing is low. As a result, instead of suppressing the voice V2 of the customer hm2, which is a disturbing sound, the voice V2 of the customer hm2 is added, and the voice V1 of the clerk hm1 may become more and more unclear.

Therefore, in the following embodiment, the acoustic crosstalk suppression device as an example of the voice processing device outputs the sound as it is without performing the acoustic crosstalk suppression processing when the suppression performance of the crosstalk component is low due to the reference signal. The first embodiment shows a case where an omnidirectional microphone is used, and the second embodiment shows a case where a microphone array capable of forming directivity is used.

Hereinafter, embodiments in which the voice processing apparatus and the voice processing method according to the present disclosure are specifically disclosed will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations for substantially the same configuration may be omitted. This is to avoid unnecessary redundancy of the following description and to facilitate the understanding of those skilled in the art. It should be noted that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

(Embodiment 1)
FIG. 1 is a block diagram showing a functional configuration example of the acoustic crosstalk suppression device 5 according to the first embodiment. The acoustic crosstalk suppression device 5 as an example of the voice processing device suppresses the disturbing sound mixed with the target sound, and has a configuration including a DSP (Digital Signal Processor) 10 and a memory 50. Two microphones mc1 and mc2 are connected to the acoustic crosstalk suppression device 5 as input devices, and a voice recognition engine (not shown) is connected as an output device.

The microphone mc1 as an example of the sound collecting device is one omnidirectional microphone. For example, the sound uttered by the main speaker is arranged so as to be able to collect the sound mainly, and the sound uttered by the main speaker is picked up. Acquires the voice signal (main signal). Similarly, the microphone mc2 as an example of the sound collecting device is one omnidirectional microphone, for example, the voice uttered by another speaker who is not the main speaker is arranged so as to be able to collect the sound mainly, and the other talk. Acquires a voice signal (reference signal) in which the voice spoken by a person is picked up. The microphone mc1 may pick up the voice uttered by another speaker to acquire the reference signal, and the microphone mc2 may pick up the voice uttered by the main speaker to acquire the main signal. The microphones mc1 and mc2 are composed of, for example, a high-quality sound compact electret condenser microphone (ECM).

The voice recognition engine performs voice recognition processing based on the voice signal after crosstalk suppression or the voice signal in which crosstalk suppression is not performed, which is output from the acoustic crosstalk suppression device 5, and the voice signal is processed as a result of the processing. Generate text data showing the contents. As the output device, instead of the voice recognition engine, a cloud server that performs processing such as voice recognition via a network (not shown) or a speaker capable of outputting voice may be connected. Further, the microphones mc1 and mc2 and the voice recognition engine may be built in the acoustic crosstalk suppression device 5.

In the acoustic crosstalk suppression device 5, for example, when two speakers (each of a plurality of speakers including the main speaker) are talking, one of the two voices spoken at the same time is used as the target sound and the other as the disturbing sound. , Suppresses the cross-talk component caused by disturbing sounds and converts the target sound into clear voice. Specifically, the acoustic crosstalk suppression device 5 performs a predetermined (see below) signal processing using an audio signal including an interfering sound as a reference signal to reproduce a pseudo crosstalk signal (suppression signal) that reproduces an acoustic crosstalk component. An example) is generated. The acoustic crosstalk suppression device 5 is clear after crosstalk suppression (that is, by removing (specifically subtracting) the pseudo crosstalk signal from the audio signal of the target sound picked up by the microphone mc1 or the microphone mc2. Generates an audio signal (with improved sound quality).

The memory 50 stores a clear voice signal of the voice (that is, the disturbing sound) uttered by the customer hm2 in the past when the microphone mc1 picks up the voice (that is, the target sound) uttered by the clerk hm1. Similarly, when the microphone mc1 picks up the voice (that is, the target sound) uttered by the customer hm2, the memory 50 stores a clear voice signal of the voice (that is, the disturbing sound) uttered by the clerk hm1 in the past. The audio signal stored in the memory 50 is used as a reference signal for reproducing acoustic crosstalk (that is, generating the pseudo-crosstalk signal described above).

The DSP 10 is a processor that performs acoustic crosstalk suppression processing on the voice signal of the voice picked up by the microphone mc1. The DSP 10 includes a single talk detection unit 45, a sound pressure comparison unit 46, a disturbing sound mixing ratio estimation unit 41, a signal processing selection unit 42, a switching unit 43, and a suppression unit 20.

The single talk detection unit 45 detects the single talk state spoken by either the clerk hm1 or the customer hm2 based on the audio signals picked up by the microphone mc1 and the microphone mc2, respectively. For example, when the single talk detection unit 45 has a speech, the sound pressure of only one of the sounds picked up by the microphone mc1 or the microphone mc2 is higher than the sound pressure of the other voice. It is determined that a single talk state has been detected. Further, the single talk detection unit 45 may determine that the single talk state has been detected when the tones of the sounds picked up by the microphone mc1 or the microphone mc2 are the same. Further, when the microphone mc1 is arranged near the clerk hm1 and the microphone mc2 is arranged near the customer hm2, the sound pressure of the sound picked up by the microphone mc1 is high during the single talk spoken by the clerk hm1, and the microphone It is determined that the sound pressure of the sound picked up by the mc2 becomes low. On the other hand, at the time of double talk spoken by both the clerk hm1 and the customer hm2, it is determined that the sound pressure of the voice picked up by the microphone mc1 and the microphone mc2 is high. Therefore, the single talk detection unit 45 detects the single talk state based on the sound pressure difference between the sound picked up by the microphone mc1 and the sound picked up by the microphone mc2.

The sound pressure comparison unit 46 collects the sound pressure of the sound picked up by the microphone mc1 and the sound picked up by the microphone mc2 in the single talk state detected by the single talk detection unit 45 and spoken by the clerk hm1 who is the main speaker. Compare with the sound pressure of the sound. The sound pressure comparison unit 46 obtains a sound pressure ratio (that is, a value indicating the ratio of the sound pressure of the sound picked up by the microphone mc1 to the sound pressure of the sound picked up by the microphone mc2) by comparison. Similarly, the sound pressure comparison unit 46 is the sound pressure of the sound picked up by the microphone mc1 and the microphone mc2 in the single talk state detected by the single talk detection unit 45 and spoken by the customer hm2 who is another speaker. Compare with the sound pressure of the sound picked up by. The sound pressure comparison unit 46 obtains a sound pressure ratio (that is, a value indicating the ratio of the sound pressure of the sound picked up by the microphone mc2 to the sound pressure of the sound picked up by the microphone mc1) by comparison. The same applies when the main speaker is the customer hm2 and the other speaker is the clerk hm1.

The disturbing sound mixing rate estimation unit 41 as an example of the mixing rate estimation unit is a main speaker that collects sound with the microphone mc1 or the microphone mc2 based on the sound pressure ratio at the time of single talk obtained by the sound pressure comparison unit 46. Estimate the mixing ratio of disturbing sounds contained in the voice signal (in other words, the reference signal) of the voice of another speaker who is not. The mixing ratio here is the ratio of the disturbing sound (in other words, the main signal of the main speaker) included in the reference signal to the reference signal. Specifically, when the main speaker is the clerk hm1, the mixing ratio is the voice (interference sound) uttered by the clerk hm1 included in the voice signal (reference signal) of the voice spoken by the customer hm2 who is another speaker. This is the ratio of the voice spoken by the customer hm2 to the voice signal (reference signal). Similarly, when the main speaker is the customer hm2, the mixing ratio is the voice (interference sound) spoken by the customer hm2 included in the voice signal (reference signal) of the voice spoken by the clerk hm1 who is another speaker. , The ratio of the voice spoken by the clerk hm1 to the voice signal (reference signal).

As an example, the sound pressure comparison unit 46 compares the sound pressure ratios of the microphone mc1 and the microphone mc2 when only the clerk hm1 who is the main speaker is speaking. At this time, it is assumed that the microphone mc1: microphone mc2 = 2: 1. Subsequently, the sound pressure comparison unit 46 compares the sound pressure ratios of the microphone mc1 and the microphone mc2 when only the customer hm2 who is the main speaker is speaking. At this time, it is assumed that the microphone mc1: microphone mc2 = 1:10. Analysis of these sound pressure ratios reveals the following.

Specifically, when the clerk hm1 speaks, the sound pressure of the voice of the clerk hm1 picked up by the microphone mc2 is relatively large, 1/3. Therefore, regarding whether or not the voice picked up by the microphone mc2 can be used as a reference signal, the voice picked up by the microphone mc2 includes the target sound (main signal) spoken by the clerk hm1 who is the main speaker (interfering sound). Since the ratio is high, the mixing ratio of the voice of the clerk hm1 becomes large. Therefore, the sound picked up by the microphone mc2 is inappropriate as a reference signal.

On the other hand, when the customer hm2 speaks, the sound pressure of the voice of the customer hm2 picked up by the microphone mc1 is as small as 1/11. Therefore, regarding whether or not the voice picked up by the microphone mc1 can be used as a reference signal, the voice picked up by the microphone mc1 includes the target sound (main signal) uttered by the customer hm2 who is the main speaker (interfering sound). Since the ratio is low, the mixing ratio of the voice of the customer hm2 becomes small. Therefore, the sound picked up by the microphone mc1 is suitable as a reference signal.

The signal processing selection unit 42 as an example of the determination unit instructs the switching unit 43 to switch based on the mixing ratio estimated by the disturbing sound mixing ratio estimation unit 41. Specifically, the signal processing selection unit 42 determines the crosstalk component when the reference signal is inappropriate, based on the comparison between the mixing ratio estimated by the disturbing sound mixing ratio estimation unit 41 and the threshold value (see FIG. 2). Instruct the switching unit 43 not to suppress the above. Further, the signal processing selection unit 42 suppresses the crosstalk component when the reference signal is appropriate, based on the comparison between the mixing ratio estimated by the disturbing sound mixing ratio estimation unit 41 and the threshold value (see FIG. 2). Instruct the switching unit 43 to do so.

The switching unit 43 has a first terminal 43a that transmits the input voice signal of the main speaker to the output stage of the acoustic crosstalk suppression device 5 without going through the suppression unit 20, and the input voice signal of the main speaker. Has a second terminal 43b that transmits the signal to the output stage of the acoustic crosstalk suppression device 5 via the suppression unit 20. The switching unit 43 switches the input of the audio signal of the main speaker to the first terminal 43a or the second terminal 43b according to the instruction from the signal processing selection unit 42. The changeover unit 43 is, for example, a mechanical, electrical or magnetic changeover switch.

The suppression unit 20 includes an adder 22, a filter update unit 25, and a delay 29. In the suppression unit 20, the adder 22 as an example of the crosstalk suppression unit subtracts the pseudo crosstalk signal generated by the convolution signal generation unit 23 from the audio signal of the sound picked up by the microphone mc1. As a result, the adder 22 can suppress the crosstalk component included in the sound picked up by the microphone mc1. In the suppression unit 20, the adder 22 outputs an audio signal after the crosstalk component is suppressed. Strictly speaking, the process performed by the adder 22 is subtraction, but it may be a process of subtracting a pseudo-crosstalk signal or a process of adding an inverted pseudo-crosstalk signal, and it can be added as a subtraction. Can also be realized. Therefore, in the present specification, this process is described as a process performed by the adder 22.

Hereinafter, in order to make the explanation easier to understand, the voice spoken by the clerk hm1 is used as the target sound (voice of the main speaker), and the voice spoken by the customer hm2 is used as the disturbing sound (voice of another person who is not the main speaker). Is illustrated. The same applies to the case where the voice uttered by the customer hm2 is used as the target sound and the voice uttered by the clerk hm1 is used as the disturbing sound.

The cross-talk component to be suppressed by the suppression unit 20 is a voice that the voice uttered by the customer hm2 in the past reaches the microphone mc1 with respect to the voice uttered by the clerk hm1 that is picked up by the microphone mc1. That is, the crosstalk component picked up by the microphone mc1 is a voice mixed by the time required for the voice spoken by the customer hm2 to reach the clerk hm1. Therefore, the suppression unit 20 holds the voice of the voice spoken by the customer hm2 in the past, and performs signal processing on the voice to generate a pseudo crosstalk signal that reproduces the mixed voice.

The filter update unit 25 includes a convolution signal generation unit 23, an update amount calculation unit 26, a non-linear conversion unit 27, and a norm calculation unit 28.

The convolution signal generation unit 23 as an example of the filter is an adaptive filter that performs a process of generating a pseudo crosstalk signal from a reference signal, and specifically, FIR (FIR) described in JP-A-2007-19595 and the like. Use the Finite Adaptive Response) filter. The convolution signal generation unit 23 reproduces the transmission characteristics between the clerk hm1 and the customer hm2 with respect to the microphone mc1 and processes the reference signal to generate a pseudo crosstalk signal. However, since the transmission characteristics of the place where the clerk hm1 and the customer hm2 face each other are not steady, it is necessary to change the characteristics of the convolution signal generation unit 23 as needed. Therefore, in the first embodiment, the filter updating unit 25 controls the coefficient or the number of taps of the FIR filter so that the characteristic of the convolution signal generation unit 23 is the latest between the clerk hm1 and the customer hm2 with respect to the microphone mc1. Change to approach the transmission characteristics. Hereinafter, the update of the adaptive filter may be referred to as learning.

Here, as described above, the voice of the clerk hm1 picked up by the microphone mc1 is delayed by the time when the voice of the customer hm2 reaches the microphone mc1. When the microphone mc1 picks up the voice of the clerk hm1, the voice of the customer hm2 is held in the memory 50 immediately before the clerk hm1 speaks, so that the reference signal is until the voice of the customer hm2 reaches the microphone mc1. The delay between them is not reflected. Therefore, in the first embodiment, the delay 29 absorbs this time difference, and the filter update unit 25 obtains a reference signal that matches the timing picked up by the microphone mc1. That is, by delaying the reference signal by the delay 29 for the time obtained by dividing the distance between the microphone mc1 and the customer hm2 by the speed of sound, the reproduced sound at the timing actually picked up by the microphone mc1 is reproduced. The value of the delay 29 can be obtained by actually measuring the distance between the microphone mc1 and the customer hm2 and dividing it by the speed of sound.

The non-linear conversion unit 27 performs non-linear conversion on the signal after suppressing the acoustic crosstalk. This non-linear conversion is a process of converting the signal after suppressing the acoustic crosstalk into information indicating the direction (positive or negative) to be updated of the filter. The non-linear conversion unit 27 outputs the signal after the non-linear conversion to the update amount calculation unit 26.

The norm calculation unit 28 calculates the norm of the voice signal of the voice spoken by the customer hm2 in the past. The norm of the voice signal of the voice spoken by the customer hm2 is the sum of the loudness of the voice signal of the voice spoken by the customer hm2 within a predetermined time in the past, and is a value indicating the degree of the magnitude of the signal within this time. Is. The norm is used by the update amount calculation unit 26 to normalize the influence of the volume of the voice of the voice spoken by the customer hm2. In general, the louder the volume, the larger the update amount of the filter is calculated. Therefore, the characteristics of the convolution signal generation unit 23 are excessively affected by the characteristics of the large voice unless normalization is performed. Therefore, in the first embodiment, the update amount of the convolution signal generation unit 23 is stabilized by normalizing the audio signal output from the delay 29 using the norm calculated by the norm calculation unit 28.

The update amount calculation unit 26 updates the filter characteristics of the convolution signal generation unit 23 (specifically, the coefficient of the FIR filter or the number of taps) from the signals received from the nonlinear conversion unit 27, the norm calculation unit 28, and the delay 29. Update amount) is calculated. Specifically, the voice of the voice received by the customer hm2 in the past received from the delay 29 is normalized based on the norm calculated by the norm calculation unit 28. Then, the update amount is determined by adding positive or negative information based on the information obtained from the nonlinear conversion unit 27 to the result of normalizing the voice of the voice spoken by the customer hm2 in the past. In the first embodiment, the update amount calculation unit 26 calculates the update amount of the filter characteristics by the ICA (Independent Component Analysis) algorithm or the NLMS (Normalized Last Mean Square) algorithm.

By executing the processes of the update amount calculation unit 26, the nonlinear conversion unit 27, and the norm calculation unit 28 at any time, the filter update unit 25 uses the characteristics of the convolution signal generation unit 23 as a microphone for collecting the voice of the clerk hm1. The transmission characteristics between mc1 and customer hm2 can be approached. When the voice uttered by the customer hm2 is used as the target sound and the voice uttered by the clerk hm1 is used as the disturbing sound, the filter updating unit 25 picks up the characteristics of the convolution signal generation unit 23 and the voice of the customer hm2. The transmission characteristics between the microphone mc1 and the clerk hm1 are brought closer.

Next, the operation of the acoustic crosstalk suppression device 5 according to the first embodiment will be shown.

FIG. 2 is a flowchart showing an example of an acoustic crosstalk suppression operation procedure according to the first embodiment. In the description of FIG. 2, this process is executed for each sample with respect to the audio signal of the audio picked up by the microphones mc1 and mc2.

In FIG. 2, the microphone mc1 picks up the voice uttered by the clerk hm1 who is the main speaker, and acquires the main signal to be recognized by the voice (S1). Further, the microphone mc2 picks up the voice uttered by the customer hm2 who is not the main speaker and acquires the reference signal (S2). The DSP 10 stores this reference signal in the memory 50.

The single talk detection unit 45 detects the single talk state spoken by either the clerk hm1 or the customer hm2 based on the voice picked up by the microphones mc1 and mc2 (S3). When the single talk state is detected, the sound pressure comparison unit 46 picks up the sound pressure of the sound picked up by the microphone mc1 and the sound picked up by the microphone mc2 in the single talk state spoken by the clerk hm1 who is the main speaker. The sound pressure ratio (see above) is obtained by comparing with the sound pressure of the voice (S4). Similarly, the sound pressure comparison unit 46 sets the sound pressure of the sound picked up by the microphone mc1 and the sound pressure of the sound picked up by the microphone mc2 in a single talk state spoken by the customer hm2 who is another speaker. To obtain the sound pressure ratio (see above).

The disturbing sound mixing ratio estimation unit 41 uses the sound pressure ratio at the time of single talk obtained by the sound pressure comparison unit 46 as the sound signal (reference signal) of the sound picked up by the microphone mc2 (or microphone mc1). The mixing ratio of the contained interfering sounds (see above) is estimated (S5).

The disturbing sound mixing rate estimation unit 41 determines whether or not the estimated mixing rate is equal to or less than the threshold value (S6). The threshold is the disturbing sound contained in the reference signal (in other words, the voice of the main speaker), which is said to not deteriorate the voice of the main speaker (that is, the disturbing sound does not increase) when the acoustic crosstalk suppression processing is performed. ) Is set.

When the mixing ratio exceeds the threshold value (S6, NO), DSP10 ends the present process shown in FIG. That is, in this case, since the crosstalk component is not suppressed, the main signal (voice signal) of the clerk hm1 who is the main speaker is output as it is to the output stage of the acoustic crosstalk suppression device 5.

On the other hand, when the mixing ratio is equal to or less than the threshold value (S4, YES), the filter update unit 25 reads the corresponding filter coefficient stored in the memory (not shown) built in the filter update unit 25 and generates a convolution signal. It is set in the unit 23 (S7). The convolution signal generation unit 23 generates a crosstalk suppression signal (an example of the suppression signal) corresponding to a pseudo crosstalk signal by using the reference signal picked up by the microphone mc2 and delayed by the delay 29. That is, the convolution signal generation unit 23 uses the latest filter coefficient updated by the update amount calculation unit 26 to perform convolution processing on the reference signal deviated by the delay time, and crosstalks from the reference signal deviated by the delay time. Generate a suppression signal (see above).

The adder 22 subtracts the crosstalk suppression signal generated by the convolution signal generation unit 23 from the audio signal of the sound picked up by the microphone mc1, and obtains the crosstalk component included in the sound picked up by the microphone mc1. Suppress (S8).

DSP10 determines whether or not it is the filter learning period (S9). The filter learning period is a period in which the customer hm2, who is another speaker, speaks to the clerk hm1 who is the main speaker. Further, the period other than the filter learning period is a period during which the customer hm2, who is another speaker, does not speak. In the case of the filter learning period (S9, YES), the filter update unit 25 updates the filter coefficient of the convolution signal generation unit 23 with the filter coefficient calculated by the update amount calculation unit 26, and is built in the filter update unit 25. It is stored in a memory (not shown) (S10). On the other hand, when it is not the filter learning period (S9, NO), DSP10 ends the present process shown in FIG.

As described above, the acoustic crosstalk suppression device 5 according to the first embodiment is connected to, for example, two microphones mc1 and mc2 arranged in a closed space such as a store where the clerk hm1 and the customer hm2 interact with each other. The acoustic crosstalk suppression device 5 is either a clerk hm1 or a customer hm2 (a plurality of people including the main speaker) existing in the store based on the audio signals picked up by the two microphones mc1 and mc2, respectively. The single talk detection unit 45 detects the single talk state spoken by one person). The acoustic cross-talk suppression device 5 includes the sound pressure ratio of the audio signal picked up by each of the two microphones mc1 and mc2 in the single talk state of the clerk hm1 who is the main speaker, and the customer hm2 who is the other speaker. Based on the sound pressure ratio of the audio signal picked up by each of the two microphones mc1 and mc2 in the single talk state of (an example of another person other than the main speaker), the other speaker (other than the main speaker) The interfering sound mixing ratio estimation unit 41 estimates the mixing ratio indicating the ratio of the main speaker's voice signal to the voice signal of another person). Based on the estimation result of the mixing ratio, the acoustic crosstalk suppression device 5 determines the necessity of suppressing the crosstalk component due to the utterance of another speaker included in the voice signal of the main speaker by the signal processing selection unit 42. ..

As a result, the acoustic crosstalk suppression device 5 is included in the utterance voice of the main speaker (for example, clerk hm1) according to the situation of a plurality of speakers (for example, clerk hm1 and customer hm2) existing in a closed space such as a store. The acoustic cross-talk component due to the spoken voice of another possible speaker (for example, customer hm2) can be adaptively suppressed. Therefore, the acoustic crosstalk suppression device 5 can improve the sound quality of the uttered voice of the main speaker.

Further, when the signal processing selection unit 42 determines that the estimation result of the mixing ratio is equal to or less than a predetermined threshold value, the signal processing selection unit 42 suppresses the crosstalk component due to the utterance of another speaker included in the audio signal of the main speaker. To decide. As a result, the acoustic crosstalk suppression device 5 can effectively suppress the crosstalk component when the voice signal of the voice uttered by another speaker is used as the reference signal.

Further, when the signal processing selection unit 42 determines that the estimation result of the mixing ratio is larger than a predetermined threshold value, the signal processing selection unit 42 does not suppress the crosstalk component due to the utterance of another speaker included in the audio signal of the main speaker. To decide. As a result, the acoustic crosstalk suppression device 5 suppresses the crosstalk component, so that the voices of other speakers mixed with the voices of the main speaker increase, and the voices of the main speaker are suppressed from becoming unclear. can. Further, by omitting the crosstalk suppression processing, the processing load by the DSP 10 can be reduced.

Further, the acoustic crosstalk suppression device 5 has a convolution signal generation unit 23 that generates a suppression signal of the crosstalk component due to the speech of another speaker included in the voice signal of the main speaker, and suppresses the crosstalk component. Using the filter update unit 25 that updates the parameters of the convolution signal generation unit 23 for the purpose and holds the update result in the memory, and the suppression signal of the crosstalk component generated by the convolution signal generation unit 23, the main speaker It further includes an adder 22 that suppresses a crosstalk component contained in an audio signal. As a result, the acoustic cross talk suppression device 5 can be included in the utterance voice of the main speaker (for example, the clerk hm1) according to the speaker situation of the clerk hm1 and the customer hm2 in the store, and the acoustic cross talk by the customer hm2. The talk component can be suppressed adaptively, and the sound quality of the utterance voice of the clerk hm1 can be improved. Therefore, even if the sound field in the store changes, for example, even if the clerk hm1 or the customer hm2 stands up after leaving their seats, the suppression performance of the crosstalk component can be gradually improved in accordance with the change in the sound field.

Further, the convolution signal generation unit 23 generates a crosstalk component suppression signal by using the latest parameter update result of the convolution signal generation unit 23 held in the memory. As a result, when the same speaker situation continues, the acoustic crosstalk suppression device 5 can continuously obtain the adaptive crosstalk component already calculated according to the speaker situation. The crosstalk component contained in the voice of the main speaker can be effectively suppressed.

Further, the acoustic crosstalk suppression device 5 has a first terminal 43a that transmits the input voice signal of the main speaker to the output stage of the acoustic crosstalk suppression device 5 without going through the adder 22, and the input main. It has a second terminal 43b that transmits the speaker's voice signal to the output stage of the acoustic crosstalk suppression device 5 via the adder 22, and is the key to suppressing the crosstalk component determined by the signal processing selection unit 42. A switching unit 43 for switching the input of the audio signal of the main speaker to the first terminal 43a or the second terminal 43b is provided according to the determination result of whether or not. As a result, the acoustic crosstalk suppression device 5 can easily switch between the voice signal with crosstalk suppression and the voice signal without crosstalk suppression by using a mechanical, electrical, or magnetic changeover switch. Can be output.

(Embodiment 2)
In the acoustic crosstalk suppression device 5A according to the second embodiment, a case where a microphone array capable of forming directivity in an arbitrary direction is used is shown. FIG. 3 is a block diagram showing a functional configuration example of the acoustic crosstalk suppression device 5A according to the second embodiment. In the acoustic crosstalk suppression device 5A according to the second embodiment, the same components as those in the first embodiment are used with the same reference numerals, the description thereof will be omitted, and only the different parts will be described here. The acoustic crosstalk suppression device 5A has a configuration including a microphone array mA instead of the microphones mc1 and mc2, as compared with the first embodiment.

The microphone array mA as an example of the sound collecting device has a plurality of (for example, 16) omnidirectional microphone elements m11, m12, ... M1n and a microphone array processing unit md, and is described in the first embodiment. It is a directional microphone capable of forming directivity (beamforming processing) in each direction of a person's speaker (for example, clerk hm1 and customer hm2). The microphone array mA as an example of the directional processing unit can form the directivity in the microphone array processing unit md in a predetermined direction by using a plurality of omnidirectional microphone elements. The technique for forming this directivity is a known technique, for example, as shown in Japanese Patent Application Laid-Open No. 2015-292241. The microphone array processing unit md may be configured to be included in the DSP 10.

The memory 50 stores the voice signal of the voice spoken by the customer hm2 in the past when the microphone array mA forms a directivity in the direction d1 in which the clerk hm1 is present and collects the voice. Similarly, the memory 50 stores the voice signal of the voice spoken by the clerk hm1 in the past when the microphone array mA forms a directivity in the direction d2 where the customer hm2 is present and picks up the voice. These signals are used as reference signals for reproducing acoustic crosstalk (that is, generating the pseudo-crosstalk signal described above).

Further, the acoustic crosstalk suppression device 5A is different from the single talk detection unit 45, the sound pressure comparison unit 46 and the disturbing sound mixing ratio estimation unit 41 according to the first embodiment, and is different from the single talk detection unit 45A and the sound pressure comparison unit 46A. The configuration includes the disturbing sound mixing ratio estimation unit 41A.

The single talk detection unit 45A is based on the voice in which the microphone array mA forms the first directivity in the direction d1 of the clerk hm1 and the voice in which the microphone array mA forms the second directivity in the direction d1 of the customer hm2. Similar to the single talk detection unit 45 according to the first embodiment, the single talk state in which either the clerk hm1 or the customer hm2 is speaking is detected.

The sound pressure comparison unit 46A forms the first directivity (see above) in the direction d1 from the microphone array mA to the clerk hm1 (main speaker) in the single talk state of the clerk hm1, and then the sound pressure of the audio signal of the clerk hm1. To get. The sound pressure comparison unit 46A acquires the sound pressure difference of the audio signal of the clerk hm1 before and after forming the first directivity (see above) in the direction d1 of the clerk hm1 from the microphone array mA in the single talk state of the clerk hm1. You may.

Further, the sound pressure comparison unit 46A acquires the sound pressure of the voice signal of the customer hm2 after forming the second directivity (see above) from the microphone array mA in the direction d2 of the customer hm2 in the single talk state of the customer hm2. .. The sound pressure comparison unit 46A acquires the sound pressure difference of the voice signal of the customer hm2 before and after forming the second directivity (see above) from the microphone array mA to the direction d2 of the customer hm2 in the single talk state of the customer hm2. You may.

Interfering sound as an example of the mixing rate estimation unit The mixing rate estimation unit 41A is a sound picked up by the microphone mc1 or the microphone mc2 based on the sound pressure or the sound pressure difference at the time of single talk obtained by the sound pressure comparison unit 46A. Estimate the mixing ratio (see above) of the disturbing sounds contained in the voice signal (reference signal) of.

The signal processing selection unit 42 as an example of the determination unit instructs the switching unit 43 to switch based on the mixing ratio estimated by the disturbing sound mixing ratio estimation unit 41A.

As an example, when the microphone array mA is placed at a position offset to the clerk hm1 side, when the microphone array mA forms a directivity in the direction d1 where the clerk hm1 is and picks up the sound, it is mixed with the voice of the clerk hm1. The ratio of voices of customer hm2 is small. Therefore, when the microphone array mA forms directivity in the direction d2 where the customer hm2 who is the main speaker is present and the suppression unit 20 acquires the suppressed sound of the crosstalk component, the microphone array mA picks up the sound. The sound that forms directivity and collects sound in the direction d1 in which the clerk hm1 who is the speaker is present is suitable for the reference signal used for suppressing the acoustic crosstalk component. Therefore, the signal processing selection unit 42 instructs the switching unit 43 to suppress the crosstalk component.

On the other hand, when the microphone array mA forms a directivity in the direction d2 where the customer hm2 is present and collects the sound, the ratio of the voice of the clerk hm1 mixed with the voice of the customer hm2 is large. Therefore, when the microphone array mA forms a directivity in the direction d1 in which the clerk hm1 who is the main speaker is present and the suppression unit 20 acquires the suppressed sound of the cross talk component, the microphone array mA picks up the sound. The sound that forms directivity and collects sound in the direction d2 in which the customer hm2 who is the speaker is present is not suitable for the reference signal used for suppressing the acoustic crosstalk component. Therefore, the signal processing selection unit 42 instructs the switching unit 43 not to suppress the crosstalk component.

The switching unit 43 receives the audio signal input from the microphone array mA on the first terminal 43a, assuming that the acoustic crosstalk is not suppressed for the sound collected by forming directivity in the direction of the clerk hm1 who is the main speaker. Switch to. On the other hand, when acoustic crosstalk suppression is performed on the sound collected by forming directivity in the direction of the customer hm2 who is the main speaker, the switching unit 43 inputs the sound signal input from the microphone array mA to the second terminal. Switch to 43b.

FIG. 4 is a flowchart showing an example of an acoustic crosstalk suppression operation procedure according to the second embodiment. In the description of FIG. 4, the same step processing as that of the first embodiment is attached, and the description thereof will be omitted.

In FIG. 4, the microphone array mA picks up the voice spoken in the store where the clerk hm1 and the customer hm2 are present (S01). The microphone array mA forms the first directivity in the direction d1 where the clerk hm1 is present with respect to the audio signal of the picked-up sound, and acquires the audio signal (main signal) of the clerk hm1 who is the main speaker (S1A). .. Similarly, the microphone array mA forms a second directivity in the direction d2 where the customer hm2 is present with respect to the audio signal of the picked-up voice, and acquires the audio signal (reference signal) of the customer hm2 who is another speaker. (S2A). The processing after step S3 is the same as that of the first embodiment.

As described above, the acoustic crosstalk suppression device 5A is based on the voice signals picked up by each of the plurality of omnidirectional microphone elements m11 to m1n of the microphone array mA, from the microphone array mA to the main speaker and others. A microphone array processing unit md, which forms different directivities in each direction of the speaker, is provided. The disturbing sound mixing ratio estimation unit 41A determines the sound pressure of the voice signal of the clerk hm1 after forming the first directivity in the direction d1 of the clerk hm1 who is the main speaker from the microphone array mA in the single talk state of the clerk hm1. The mixing ratio is estimated based on the sound pressure of the voice signal of the customer hm2 after forming the second directivity in the direction d2 of the customer hm2 who is another speaker from the microphone array mA in the single talk state of the customer hm2. ..

Thereby, the acoustic crosstalk suppression device 5A can determine whether or not to perform the acoustic crosstalk suppression processing in consideration of the directivity performance of the microphone array mA. Further, by collecting the voice having the directivity formed in the direction d2 of the customer hm2, the ratio (mixing ratio) of the voice (interfering sound) of the clerk hm1 mixed with the voice of the customer hm2 used as the reference signal is reduced. Can be done. Therefore, it is possible to increase the probability that the crosstalk component is suppressed with respect to the voice of the voice spoken by the clerk hm1.

(Modified Example of Embodiment 2)
In the modified example of the first embodiment, the acoustic crosstalk suppression device 5B shows a case where the single talk state is detected based on the sound source direction information. FIG. 5 is a block diagram showing a functional configuration example of the acoustic crosstalk suppression device 5B according to the modified example of the second embodiment. In the acoustic crosstalk suppression device 5B, the same components as those of the acoustic crosstalk suppression device 5A according to the second embodiment are designated by the same reference numerals, and the description thereof will be omitted. Here, different components will be described.

The acoustic crosstalk suppression device 5B has a single talk detection unit 45B, a sound pressure comparison unit 46B, a disturbing sound mixing ratio estimation unit 41B, and a memory 53, which are different from those in the second embodiment. The single talk detection unit 45B inputs the sound source direction information stored in the memory 53 and detects the single talk state. The sound source direction information is, for example, a sound pressure value calculated to correspond to the position of each pixel constituting a fisheye image having a 360-degree orientation taken by an omnidirectional camera (not shown). It is a sound pressure heat map created by being assigned in association with. This sound pressure heat map is created by an external device (not shown) different from the acoustic crosstalk suppression device 5B, and is stored in the memory 53 in advance. The external device has a microphone array with an omnidirectional camera, for example to generate a sound pressure heatmap. A microphone array with an omnidirectional camera has a plurality of (for example, 16) microphone elements arranged in a ring shape, and is coaxial with the omnidirectional camera so that the microphone array including the plurality of microphone elements surrounds the omnidirectional camera. It is a configuration provided in. The analysis of the sound source direction is a known technique, for example, as disclosed in Japanese Patent Application Laid-Open No. 2020-12704. When the microphone array with an omnidirectional camera is installed on the ceiling or a wall surface near the ceiling, for example, it forms directivity in each direction with respect to the image captured by the omnidirectional camera and collects sound. The sound pressure in the direction is acquired as a sound pressure heat map.

FIG. 6 is a diagram showing an image GZ1 captured by an omnidirectional camera on which a sound pressure heat map is superimposed. When a person in an image captured by an omnidirectional camera is identified, the microphone array forms directivity in that direction and is capable of picking up the voice spoken by that person. In FIG. 6, the microphone array with an omnidirectional camera performs beamforming in a range including the store clerk hm1 and the customer hm2 in the captured image to generate a sound pressure heat map.

The single talk detection unit 45B detects the single talk state when there is one place on the sound pressure heat map where the sound pressure of the voice spoken by the speaker is equal to or higher than a predetermined value. That is, if there is one place (dark dot display in FIG. 6) where a sound pressure equal to or higher than a predetermined value appears on the sound pressure heat map, it is determined that the single talk state has been detected.

The sound pressure comparison unit 46B acquires the sound pressure ratio of the corresponding audio signal in which the directivity of the clerk hm1 is formed by the microphone array processing unit md in the single talk state of the clerk hm1. Further, the sound pressure comparison unit 46B acquires the sound pressure ratio with the corresponding voice signal in which the directivity of the customer hm2 is formed by the microphone array processing unit md in the single talk state of the customer hm2.

The disturbing sound mixing ratio estimation unit 41B includes sound source direction information, the sound pressure ratio of the corresponding audio signal in which the directivity of the clerk hm1 is formed by the microphone array processing unit md in the single talk state of the clerk hm1, and the single of the customer hm2. The mixing ratio (see above) is estimated based on the sound pressure ratio with the corresponding audio signal in which the directivity of the customer hm2 is formed by the microphone array processing unit md in the talk state.

When the detection of the single talk state is performed using the sound source direction information, the camera image may be used as the sound source direction information. Further, when the camera image is used, for example, if there is only one person moving the mouth in the image captured by the omnidirectional camera, it is determined that the single talk state is detected.

As described above, the acoustic crosstalk suppression device 5B is based on the voice signals picked up by each of the plurality of omnidirectional microphone elements m11 to m1n of the microphone array mA, from the microphone array mA to the main speaker and others. A microphone array processing unit md, which forms different directivities in each direction of the speaker, is provided. The single talk detection unit 45B acquires sound source direction information indicating directions to each of the clerk hm1 who is the main speaker in the store and the customer hm2 who is another speaker, and the single talk state is based on the sound source direction information. Is detected. The disturbing sound mixing ratio estimation unit 41B includes sound source direction information, the sound pressure ratio of the corresponding audio signal in which the directivity of the clerk hm1 is formed by the microphone array processing unit md in the single talk state of the clerk hm1, and the single of the customer hm2. The mixing ratio is estimated based on the sound pressure ratio with the corresponding audio signal in which the directivity of the customer hm2 is formed by the microphone array processing unit md in the talk state.

As a result, in the acoustic crosstalk suppression device 5B, since the single talk detection unit 45B acquires the sound source direction information, the single talk state can be quickly detected and the mixing ratio can be acquired. In addition, the single talk state detection process can be reduced.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modifications, modifications, substitutions, additions, deletions, and equality within the scope of the claims. It is understood that it naturally belongs to the technical scope of the present disclosure. Further, each component in the various embodiments described above may be arbitrarily combined as long as the gist of the invention is not deviated.

For example, in the first embodiment described above, two microphones, a microphone mc1 for the clerk hm1 and a microphone mc2 for the customer hm1, are provided, but at least one of these microphones may be built in the headset. .. As a result, the sound pressure of the disturbing sound included in the audio signal used as the reference signal is lowered, and the suppression of acoustic crosstalk is easily executed.

Further, in any of the above-described first and second embodiments, when the mixing ratio estimated by the parameter mixing rate estimation unit 41 is equal to or less than the threshold value, the update amount calculation unit 26 determines the algorithm (N) based on the mixing ratio value. The parameters of the adaptive filter may be calculated by changing the LMS algorithm, the ICA algorithm, etc.), and the parameters can be set to more suitable values.

Further, the acoustic crosstalk suppression device may be used for a howling canceller. In a karaoke box or the like, the howling canceller suppresses a sound in which a voice emitted by itself is reproduced by a speaker and picked up by a microphone as an interfering sound. Further, the acoustic crosstalk suppression device may be used for an echo canceller. The echo canceller suppresses the sound that the voice spoken by another speaker is output from the speaker and picked up by the microphone of the main speaker as a disturbing sound in the vehicle interior or the like.

The present disclosure adaptively suppresses the acoustic cross-talk component of the speech speech of another speaker that may be included in the speech speech of the main speaker, depending on the situation of a plurality of speakers existing in the closed space. It is useful as a voice processing device and a voice processing method for improving the sound quality of the uttered voice of the main speaker.

5,5A, 5B Acoustic crosstalk suppression device 22 Adder 23 Folding signal generation unit 25 Filter update unit 26 Update amount calculation unit 27 Non-linear conversion unit 28 Norm calculation unit 29 Delay 41, 41A, 41B Interference sound mixing rate estimation unit 42 Signal Processing selection unit 43 Switching unit 43a 1st terminal 43b 2nd terminal 45 Single talk detection unit 46 Sound pressure comparison unit mA Microphone array mc1, mc2 Microphone

Claims (9)

Connected to multiple microphones located in a closed space,
Single talk detection that detects a single talk state in which any one of a plurality of people including the main speaker existing in the closed space is speaking based on the audio signals picked up by each of the plurality of microphones. Department and
The sound pressure ratio of the audio signal picked up by each of the plurality of microphones in the single talk state of the main speaker and the sound picked up by each of the plurality of microphones in the single talk state of another person other than the main speaker. A mixing ratio estimation unit that estimates a mixing ratio indicating the ratio of the main speaker's voice signal to the voice signal of the other person based on the sound pressure ratio of the voice signal.
Based on the estimation result of the mixing ratio, a determination unit for determining whether or not the crosstalk component is suppressed by the utterance of the other person included in the audio signal of the main speaker is provided.
Voice processing device. When the determination unit determines that the estimation result of the mixing ratio is equal to or less than a predetermined threshold value, it determines that the crosstalk component is suppressed by the utterance of the other person included in the audio signal of the main speaker. ,
The voice processing device according to claim 1. When the determination unit determines that the estimation result of the mixing ratio is larger than a predetermined threshold value, the determination unit determines that the crosstalk component is not suppressed by the utterance of the other person included in the audio signal of the main speaker. ,
The voice processing device according to claim 1. It has a filter that generates a suppression signal of a crosstalk component due to the utterance of the other person included in the audio signal of the main speaker, updates the parameters of the filter for suppressing the crosstalk component, and updates the result. And the filter updater that holds
Using the suppression signal generated by the filter, a crosstalk suppression unit that suppresses the crosstalk component included in the audio signal of the main speaker is further provided.
The voice processing device according to claim 1. Directivity that forms different directivity in the direction from the sound collecting device to the main speaker and the other person based on the sound signal picked up by the sound collecting device accommodating each of the plurality of microphones. Further equipped with a processing unit,
The mixing ratio estimation unit includes the sound pressure of the voice signal of the main speaker after forming the first directivity from the sound collecting device in the direction of the main speaker in the single talk state of the main speaker, and the sound pressure of the main speaker. The mixing ratio is estimated based on the sound pressure of the voice signal of the other person after forming the second directivity in the direction of the other person from the sound collecting device in the single talk state of the other person.
The voice processing device according to claim 1. Directivity that forms different directivity in the direction from the sound collecting device to the main speaker and the other person based on the sound signal picked up by the sound collecting device accommodating each of the plurality of microphones. Further equipped with a processing unit,
The single talk detection unit acquires sound source direction information indicating directions to each of the main speaker and the other person in the closed space, detects the single talk state based on the sound source direction information, and detects the single talk state.
The mixing ratio estimation unit uses the audio signal in which the directivity of the main speaker is formed by the directivity processing unit in the single talk state of the main speaker and the directivity processing unit in the single talk state of the other person. The mixing ratio is estimated based on the sound pressure ratio with the audio signal in which the directivity of the other person is formed.
The voice processing device according to claim 1. The filter generates the suppression signal by using the latest update result of the parameter of the filter held in the memory.
The voice processing device according to claim 4. The first terminal that transmits the input audio signal of the main speaker to the output stage of the audio processing device without passing through the cross talk suppression unit, and the input audio signal of the main speaker are crossed. It has a second terminal that transmits to the output stage of the voice processing device via the talk suppression unit, and inputs the voice signal of the main speaker based on the determination result of the necessity of suppressing the cross talk component. A switching unit for switching to the first terminal or the second terminal is further provided.
The voice processing device according to claim 4. A voice processing method executed by a voice processing device connected to a plurality of microphones arranged in a closed space.
Based on the audio signals picked up by each of the plurality of microphones, a single talk state in which any one of a plurality of persons including the main speaker existing in the closed space is speaking is detected.
The sound pressure ratio of the audio signal picked up by each of the plurality of microphones in the single talk state of the main speaker and the sound picked up by each of the plurality of microphones in the single talk state of another person other than the main speaker. Based on the sound pressure ratio of the voice signal, the mixing ratio indicating the ratio of the voice signal of the main speaker to the voice signal of the other person is estimated.
Based on the estimation result of the mixing ratio, it is determined whether or not it is necessary to suppress the crosstalk component due to the utterance of the other person included in the audio signal of the main speaker.
Voice processing method.

Images