JP5251808B2 - Noise removal device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately remove noise from input voice. <P>SOLUTION: A removing part removes the component of a second voice from a first signal obtained from a first input part to which first voice is mainly input, by using a filter coefficient which is updated according to a second signal obtained from a second input part to which second voice is mainly input. A mixing detection part detects that the first voice component is mixed in the second signal, when it is determined that a correlation value for indicating correlation between the first signal and the second signal exceeds a predetermined threshold, and that the difference of an output level of the first signal from that of the second signal exceeds a predetermined threshold. A control part controls to stop updating of the filter coefficient, when it is detected that the first voice component is included in the second signal, by the mixing detection part. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a noise removal device.

  Conventionally, for the purpose of improving the recognition accuracy of an audio signal acquired by an audio input device such as a microphone, for example, there is a noise removal technique for removing a signal corresponding to ambient noise from the acquired audio signal. Note that a noise removal device that implements the above-described noise removal technology may be applied to a personal service providing robot that performs a personal service using voice communication in a public space such as a shopping center or a hands-free telephone.

  The conventional noise removal apparatus has a voice acquisition microphone (MIC_S) and a noise acquisition microphone (MIC_N). The conventional noise removing device applied to the hands-free telephone is a received voice that is emitted using a sound reproduction speaker included in the noise removing device itself as a sound source, and an environmental sound such as an announcement or BGM that flows in the environment surrounding the noise removing device. Are processed as noise removal targets.

  For example, when a received noise is removed from a voice acquired by a voice acquisition microphone, a conventional noise removing device provides an echo that removes a received voice signal from a voice input signal by feeding back a voice reproduction output signal to an echo cancellation unit. Cancel processing is performed. That is, the conventional noise removal apparatus executes a process of subtracting the noise component from the voice input signal using an echo canceling adaptive filter.

  In addition, when removing the environmental sound from the sound acquired by the sound acquisition microphone, the conventional noise removal device inputs the sound of the environmental sound acquired by the noise acquisition microphone (MIC_N) to the noise cancellation unit. A noise cancellation process is performed to remove the environmental sound signal from the input signal. That is, the conventional noise removal apparatus performs a process of subtracting the noise component from the voice input signal using an adaptive filter for noise cancellation.

  By the way, the conventional noise removal apparatus has the following two problems when a noise acquisition microphone is used. The first problem is that the coefficient estimation of the adaptive filter for noise cancellation or echo cancellation becomes unstable because the received voice is input not only to the voice acquisition microphone but also to the noise acquisition microphone. is there. The second problem is that the sound input signal is deteriorated by the noise cancellation processing when the sound input to the sound acquisition microphone wraps around the noise acquisition microphone.

  Therefore, in order to cope with the above-described two problems, there is a technique for controlling the coefficient update of the adaptive filter using a voice output detector that detects the presence or absence of a received voice and a voice input detector that detects the presence or absence of a voice input. Proposed. For example, when it is determined by the audio output detector that the power of the received voice exceeds a predetermined threshold, it is detected that the received voice has been input. Similarly, when it is determined by the voice input detector that the power of voice input exceeds a predetermined threshold, it is detected that there is voice input. Then, when voice input is detected, control is performed so as to stop the coefficient update of the adaptive filter, thereby preventing deterioration of the voice input signal.

JP-A-6-14101

  In the above-described technology for controlling the update of the adaptive filter, it is necessary to set an appropriate threshold value for detecting the voice input in accordance with the volume of the received voice, the volume of the voice input, and the change in the noise level. It becomes. That is, since the volume of the received voice, the volume of the voice input, and the noise level constantly change, it is extremely difficult to set an appropriate threshold value that can cope with any situation surrounding the noise removal apparatus. For this reason, the detection accuracy of the voice input is greatly lowered, and as a result, the voice input signal is deteriorated.

  The disclosed technology has been made in view of the above, and an object of the present invention is to provide a noise removing device capable of accurately removing noise from input speech.

  In one aspect, the technology disclosed in the present application mainly uses a filter coefficient that is updated based on a second signal acquired from a second input unit to which second audio is input. Between the first signal and the second signal, and a removal unit that removes the component of the second sound from the first signal acquired from the first input unit to which the sound is input And calculating whether or not the calculated correlation value exceeds a predetermined threshold, and comparing the output level of the first signal with the output level of the second signal Determining whether a difference between an output level of the first signal and an output level of the second signal exceeds a predetermined threshold; determining that the correlation value exceeds a predetermined threshold; and outputting the output When it is determined that the level difference exceeds a predetermined threshold, the second signal A mixing detection unit for detecting that the first audio component is mixed, and when the mixing detection unit detects that the first audio component is mixed in the second signal And a control unit that controls to stop the update of the filter coefficient.

  According to one aspect of the technology disclosed in the present application, noise can be accurately removed from input speech.

FIG. 1 is a diagram illustrating the noise removal device according to the first embodiment. FIG. 2 is a diagram illustrating a configuration according to the second embodiment. FIG. 3 is a diagram illustrating the configuration of the voice input detector according to the second embodiment. FIG. 4 is a diagram illustrating an example of a voice input flag determination table according to the second embodiment. FIG. 5 is a diagram illustrating an example of a filter coefficient update execution determination table according to the second embodiment. FIG. 6 is a diagram illustrating an example of a filter coefficient update execution determination table according to the second embodiment. FIG. 7 is a diagram illustrating a flow of processing by the noise removal device according to the second embodiment. FIG. 8 is a diagram illustrating a process flow of the noise removal apparatus according to the second embodiment. FIG. 9 is a diagram illustrating a flow of processing performed by the noise removal apparatus according to the second embodiment. FIG. 10 is a diagram illustrating a configuration according to the third embodiment. FIG. 11 is a diagram illustrating a configuration according to the fourth embodiment. FIG. 12 is a diagram illustrating the configuration of the received voice input detector according to the fourth embodiment. FIG. 13 is a diagram illustrating a process flow of the noise removal apparatus according to the fourth embodiment. FIG. 14 is a diagram illustrating a flow of processing by the noise removal device according to the fourth embodiment. FIG. 15 is a diagram illustrating a process flow of the noise removal device according to the fourth embodiment. FIG. 16 is a diagram illustrating a configuration according to the fifth embodiment. FIG. 17 is a diagram illustrating a configuration according to the sixth embodiment. FIG. 18 is a diagram for explaining the operation of the noise removal device according to the sixth embodiment.

  Hereinafter, an embodiment of a noise removal device disclosed in the present application will be described in detail with reference to the drawings. In the following description, the technology disclosed in the present application is not limited by an example described later as an embodiment of the noise removal device disclosed in the present application.

  FIG. 1 is a diagram illustrating the noise removal device according to the first embodiment. As illustrated in FIG. 1, the noise removal device 1 according to the first embodiment includes a removal unit 4, a mixing detection unit 5, and a control unit 6.

  And the 1st audio | voice is mainly input into the 1st input part 2 shown in FIG. The second input unit 3 shown in FIG. 1 mainly receives the second sound.

  Further, the removal unit 4 shown in FIG. 1 uses the filter coefficient updated based on the second signal acquired from the second input unit 3, from the first signal acquired from the first input unit 2. The second audio component is removed.

  Further, the mixing detection unit 5 shown in FIG. 1 calculates a correlation value indicating the correlation between the first signal and the second signal, and whether or not the calculated correlation value exceeds a predetermined threshold value. Determine. Further, the mixing detection unit 5 compares the output level of the first signal with the output level of the second signal, and the difference between the output level of the first signal and the output level of the second signal is a predetermined threshold value. It is determined whether or not. When the contamination detection unit 5 determines that the correlation value exceeds a predetermined threshold and determines that the difference in output level exceeds the predetermined threshold, the first audio component is added to the second signal. Detects that is mixed.

  Further, the control unit 6 shown in FIG. 1 performs control so as to stop the update of the filter coefficient when the mixing detection unit 5 detects that the first signal component is included in the second signal. To do.

  That is, the noise removal apparatus according to the first embodiment stops updating the filter coefficient when the first sound component is mixed in the second signal. At this time, the noise removal apparatus according to the first embodiment determines whether or not the degree of correlation between the first signal and the second signal is high, so that the first audio component is mixed into the second signal. It is determined whether or not. Therefore, it is possible to improve the accuracy of detecting whether or not the first audio component is mixed in the second signal, and the filter in a state where the first audio component is mixed in the second signal. Coefficient update can be avoided. Therefore, as a result, the second sound component can be accurately removed from the first sound.

[Configuration of Noise Reduction Device (Example 2)]
FIG. 2 is a diagram illustrating a configuration according to the second embodiment. In the following, an embodiment in which the noise removal apparatus according to the second embodiment is applied to a service providing robot that performs an interpersonal service using voice communication in a public space such as a shopping center will be described.

  As shown in FIG. 2, the service providing robot 100 includes a sound acquisition microphone (MIC_S) 110, a noise acquisition microphone (MIC_N) 120, and a sound reproduction speaker 130. Furthermore, the service providing robot 100 includes A / D (analog / digital converters) 140a to 140c, a D / A (digital / analog converter) 150, a voice recognition unit 160, and a robot controller 170, as shown in FIG.

  The voice acquisition microphone 110 accepts input of uttered voice mainly from a user of the service providing robot 100. The noise acquisition microphone 120 accepts input of environmental sounds other than uttered voices such as announcements and BGM that are mainly flowing in the environment surrounding the service providing robot 100. The audio reproduction speaker 130 outputs the audio reproduced by the service providing robot 100 to the user.

  The A / D 140 a converts an analog audio signal input via the audio acquisition microphone 110 into a digital audio signal and outputs the digital audio signal to the noise removal apparatus 200. The A / D 140 b converts an analog noise signal input via the noise acquisition microphone 120 into a digital noise signal and outputs the digital noise signal to the noise removal apparatus 200. The A / D 140 c converts an analog reproduced audio signal input via a D / A 150 described later into a digital reproduced audio signal, and outputs the digital reproduced audio signal to the noise removal apparatus 200.

  The speech recognition unit 160 executes recognition processing of the speech signal output from the noise removal device 200 and sends the recognition result to the robot controller 170. For example, the voice recognition unit 160 extracts words or phrases included in the voice uttered by the user using a known voice recognition method. Then, the voice recognition unit 160 sends the extracted word or phrase to the robot controller 170 as a recognition result.

  The robot controller 170 generates a digital playback audio signal according to the voice recognition result sent from the voice recognition unit 160 and sends the generated playback voice signal to the D / A 150. For example, the robot controller 170 identifies a response (word or phrase) corresponding to a word or phrase sent from the speech recognition unit 160, and generates a playback audio signal that reproduces the identified response. In addition, when the robot controller 170 sends a playback sound signal to the D / A 150, the noise removal device 200 (filter coefficient estimation), which will be described later, indicates a sound playback flag indicating that the sound using the service providing robot 100 as a sound source is played back. Output to the units 222 and 232). For example, the robot controller 170 outputs “True (= sound playback)” as a sound playback flag when in the sound playback state, and “False (= no sound playback) when in the sound non-playback state. "Is output as an audio reproduction flag.

  The D / A 150 converts a digital playback audio signal sent from a robot controller 170 (to be described later) into an analog signal and sends it to the audio playback speaker 130.

  The noise removal apparatus 200 outputs to the voice recognition unit 160 a voice signal obtained by removing a noise component and a reproduced voice component from the voice signal output from the A / D 140a by a noise cancellation unit 220 and an echo cancellation unit 230 described later. The noise removal apparatus 200 includes, for example, a voice input detector 210, a noise cancellation unit 220, and an echo cancellation unit 230, as shown in FIG.

  The voice input detector 210 uses the voice signal output from the A / D 140a and the noise signal output from the A / D 140b to detect whether the voice signal is mixed in the noise signal. Then, the voice input detector 210 sends a voice input flag indicating whether or not a voice signal is mixed in the noise signal to a noise cancellation unit 220 (filter coefficient estimator 222) described later.

  FIG. 3 is a diagram illustrating the configuration of the voice input detector according to the second embodiment. For example, as shown in the figure, the audio input detector 210 includes delay taps 211a and 211b, frame division processing units 212a and 212b, a cross-correlation detector 213, a signal level comparator 214, and a flag generator 215. Have

  The delay taps 211a and 211b adjust a known delay (for example, a delay difference in A / D or a delay in a transmission path). For example, the delay tap 211a adjusts a delay difference or the like in the A / D 140a for the audio signal output from the A / D 140a, and sends it to the frame division processing unit 212a. For example, the delay tap 211b adjusts the delay difference in the A / D 140b with respect to the noise signal output from the A / D 140b, and sends it to the frame division processing unit 212b.

  The frame division processing units 212a and 212b divide the signals sent from the delay taps 211a and 211b and send them to the cross-correlation detector 213 and the signal level comparator 214, respectively. For example, the frame division processing unit 212a sequentially divides the audio signal transmitted from the delay tap 211a using several samples (for example, 512 samples) as one frame. The processes after the frame division processing units 212a and 212b are all processed in units of frames.

  Then, the frame division processing unit 212a sends the divided audio signals to the cross correlation detector 213 and the signal level comparator 214, respectively. Similarly, the frame division processing unit 212b sequentially divides the noise signal transmitted from the delay tap 211b, for example, with several samples (for example, 512 samples) as one frame. Then, the frame division processing unit 212b sends the divided audio signals to the cross-correlation detector 213 and the signal level comparator 214, respectively.

  The cross-correlation detector 213 calculates a cross-correlation value indicating a correlation between the audio signal transmitted from the frame division processing unit 212a and the noise signal transmitted from the frame division processing unit 212b.

  For example, the cross-correlation detector 213 superimposes the audio signal and the noise signal with a predetermined phase, and performs a cross-correlation operation between the audio signal and the noise signal in a range of 50 samples before and after the superimposed predetermined phase. To calculate a cross-correlation value. Then, the cross-correlation detector 213 acquires the maximum correlation position information at which the calculated cross-correlation value becomes the maximum value. Further, the cross-correlation detector 213 determines whether or not the maximum value of the cross-correlation value is larger than a predetermined threshold value. As a result of the determination, if the maximum value of the cross-correlation value is larger than a predetermined threshold value, the cross-correlation detector 213 sends “True (= correlation)” to the flag generator 215 described later as correlation presence / absence information. . On the contrary, if the maximum cross-correlation value is smaller than a predetermined threshold as a result of the determination, “False (= no correlation)” is sent to the flag generator 215 described later as correlation presence / absence information.

  The signal level comparator 214 compares the signal level (eg, power value) of the audio signal sent from the frame division processing unit 212a and the signal level (eg, power value) of the noise signal sent from the frame division processing unit 212b. Compare. For example, as shown in FIG. 3, the signal level comparator 214 includes a root mean square calculator 214a and a power comparator 214b.

  The root mean square calculator 214a squares the voltage value of the audio signal sent from the frame division processing unit 212a and the voltage value of the noise signal sent from the frame division processing unit 212b, respectively, and the power value of the audio signal and noise signal Are calculated respectively. Then, the root mean square calculator 214a sends the power values of the audio signal and the noise signal to the power comparator 214b described later.

  The power comparator 214b superimposes the audio signal and the noise signal transmitted from the root mean square calculator 214a at a phase corresponding to the correlation maximum position information acquired by the cross correlation detector 213. Then, the power comparator 214b calculates a difference between an average value of the power value of the sound signal and the power value of the noise signal when the sound signal and the noise signal are superimposed with a phase corresponding to the correlation maximum position information, It is determined whether or not the calculated difference is greater than a predetermined threshold. As a result of the determination, if the calculated difference is larger than a predetermined threshold, the power comparator 214b sends “True (= with level difference)” to the flag generator 215 described later as level comparison information. On the other hand, if the calculated difference is equal to or smaller than a predetermined threshold as a result of the determination, the power comparator 214b uses “False (= no level difference)” as a flag generator to be described later as level comparison information. 215. The average value is processed in units of frames.

  The flag generator 215 has a determination table that predefines the correspondence between the combination of correlation presence / absence information and level comparison information sent from the cross-correlation detector 213 and the signal level comparator 214 and the contents of the voice input flag to be generated. Have. Then, the flag generator 215 generates a sound input flag indicating whether or not a sound signal is mixed in the noise signal according to this determination table, and sends the generated flag to the noise canceling unit 220 described later. The audio input flag is output in units of divided frames. For example, when one frame is divided into 512 samples, the same flag is output for 512 samples.

  FIG. 4 is a diagram illustrating an example of a voice input flag determination table according to the second embodiment. As shown in the figure, the determination table defines correlation presence / absence information (maximum cross-correlation value> threshold value) sent from the cross-correlation detector 213 as an item in the leftmost column. For example, when the correlation presence / absence information transmitted from the cross-correlation detector 213 is “True”, it indicates that there is a correlation between the noise signal and the audio signal, and the correlation presence / absence information is “False”. Indicates that there is no correlation between the noise signal and the audio signal.

  Also, as shown in FIG. 4, level comparison information (root mean square difference> threshold value) sent from the signal level comparator 214 is defined in the determination table as an item in the middle column. For example, when the level comparison information sent from the signal level comparator 214 is “True”, it indicates that there is a level difference between the noise signal and the audio signal, and the level comparison information is “False”. In this case, there is no level difference between the noise signal and the audio signal.

  As shown in FIG. 4, the type of the voice input flag to be generated by the flag generator 215 is defined in the determination table as an item in the rightmost column. For example, when the audio input flag is “True”, it indicates that the audio signal is mixed in the noise signal, and when the audio input flag is “False”, the audio signal is mixed in the noise signal. Indicates not.

  As shown in FIG. 4, the determination table should be generated only when the correlation presence / absence information and the level comparison information sent from the cross-correlation detector 213 and the signal level comparator 214 are both “True”. The type of the voice input flag is defined as “True”.

  For example, when the correlation presence / absence information and the level comparison information sent from the cross-correlation detector 213 and the signal level comparator 214 are both “True”, the flag generator 215 has the highest level in the determination table shown in FIG. Follow the information defined in the row. That is, the flag generator 215 generates an audio input flag of “True” indicating that an audio signal is mixed into a noise signal.

  Further, for example, when the correlation presence / absence information and the level comparison information transmitted from the cross-correlation detector 213 and the signal level comparator 214 are both “False”, the flag generator 215 determines the determination table 4 shown in FIG. Follow the information defined in the second row. That is, the flag generator 215 generates a “False” audio input flag indicating that no audio signal is mixed into the noise signal.

  Further, for example, when either one of the correlation presence information and the level comparison information sent from the cross correlation detector 213 and the signal level comparator 214 is “False”, the flag generator 215 is shown in FIG. According to the information defined in the decision table 2 and the third row. That is, the flag generator 215 generates a “False” audio input flag indicating that no audio signal is mixed into the noise signal.

  As shown in FIG. 2, the noise cancellation unit 220 includes an FIR (Finite impulse response) filter 221 and a filter coefficient estimator 222.

  The FIR filter 221 uses the noise cancel (NC) filter coefficient sent from the filter coefficient estimator 222 to remove a noise component from the audio signal output from the A / D 140a. Note that the filter coefficient for noise cancellation (NC) is used as a coefficient of a transfer function when the noise signal is adapted so that the noise component included in the audio signal is “0”.

  The filter coefficient estimator 222 updates the filter coefficient for noise cancellation (NC) based on the noise signal output from the A / D 140 b, and sends the updated filter coefficient to the FIR filter 221. Further, the filter coefficient estimator 222 controls the update of the filter coefficient for noise cancellation (NC) based on the voice input flag sent from the voice input detector 210 and the voice reproduction flag sent from the robot controller 170. To do.

  For example, the filter coefficient estimator 222 has an execution determination table in which a sound reproduction state indicated by the sound reproduction flag and a corresponding operation corresponding to the sound reproduction state are defined in advance in association with the sound reproduction flag. Then, the filter coefficient estimator 222 updates the filter coefficient for noise cancellation according to the execution determination table.

  FIG. 5 is a diagram illustrating an example of a filter coefficient update execution determination table according to the second embodiment. As shown in the figure, in the execution determination table, the type of the audio reproduction flag is the item in the leftmost column, the state of the audio reproduction indicated by the audio reproduction flag is the item in the middle column, and the item in the rightmost column The corresponding operation of the filter coefficient estimator 222 is defined as For example, in the execution determination table, an audio reproduction state “audio reproduction” and a corresponding operation “coefficient update stop” are defined in association with the audio reproduction flag “True”. Also, for example, as shown in the figure, in the execution determination table, a sound reproduction state “no sound reproduction” and a corresponding operation “coefficient update” are defined in association with the sound reproduction flag “False”. In the execution determination table, the audio playback state (“audio playback”, “no audio playback”) is not necessarily defined.

  For example, when the filter coefficient estimator 222 has already input the sound reproduction flag “True” at the timing of executing the update of the filter coefficient, the filter coefficient estimator 222 is in the sound reproduction state based on the execution determination table shown in FIG. judge. Then, the filter coefficient estimator 222 determines to stop updating the filter coefficient for noise cancellation (NC) according to the execution determination table shown in FIG. Also, for example, if the audio reproduction flag “False” has already been input at the timing of executing the filter coefficient update, the filter coefficient estimator 222 enters the no audio reproduction state based on the execution determination table shown in FIG. Judge that there is. Then, the filter coefficient estimator 222 updates the filter coefficient for noise cancellation (NC) according to the execution determination table shown in FIG.

  In addition, the filter coefficient estimator 222 predefines the corresponding action according to the voice input state (whether the voice signal is mixed into the noise signal) and the voice input state indicated by the voice input flag in association with the voice input flag. Another execution determination table is provided. Then, the filter coefficient estimator 222 updates the filter coefficient for noise cancellation according to the execution determination table.

  FIG. 6 is a diagram illustrating an example of a filter coefficient update execution determination table according to the second embodiment. As shown in the figure, in the execution determination table, as the leftmost column item, the type of the voice input flag, the middle column item, the voice input mixed state indicating whether or not the voice signal is mixed into the noise signal, The corresponding operation of the filter coefficient estimator 222 is defined as an item in the rightmost column. For example, in the execution determination table, the voice input state “voice input mixed” and the corresponding action “coefficient update stop” are defined in association with the voice input flag “True”. Also, for example, as shown in the figure, in the execution determination table, a voice input mixed state “no voice input mixed” and a corresponding operation “coefficient update” are defined in association with the voice input flag “False”.

  For example, when the filter coefficient estimator 222 has already input the voice input flag “True” at the timing of executing the filter coefficient update, there is voice input mixing (noise) based on the execution determination table shown in FIG. It is determined that the audio signal is mixed into the signal. Then, the filter coefficient estimator 222 determines to stop updating the filter coefficient for noise cancellation (NC) according to the execution determination table shown in FIG. Also, for example, when the filter coefficient estimator 222 has already input the voice input flag “False” at the timing of executing the filter coefficient update, there is no voice input mixing based on the execution determination table shown in FIG. It is determined that the state is (no mixing of the audio signal into the noise signal). Then, the filter coefficient estimator 222 updates the filter coefficient for noise cancellation (NC) according to the execution determination table shown in FIG.

  Summarizing the update of the filter coefficient described above, for example, the filter coefficient estimator 222 inputs at least one of the audio reproduction flag “True” and the audio input flag “True” at the timing of executing the update of the filter coefficient. If it has been completed, the updating of the filter coefficient is stopped.

  Note that the filter coefficient estimator 222 sends the filter coefficient used last time to the FIR filter 221 as it is, for example, when updating of the filter coefficient for noise cancellation (NC) is stopped. Note that while the filter coefficient update is stopped by the filter coefficient estimator 222, the FIR filter 221 removes a noise component from the audio signal using the filter coefficient used last time.

  As shown in FIG. 2, the echo cancellation unit 230 includes an FIR (Finite impulse response) filter 231 and a filter coefficient estimator 232.

  The FIR filter 231 uses the echo cancel (EC) filter coefficient sent from the filter coefficient estimator 232 to remove the reproduced audio component from the audio signal output from the A / D 140a. Note that the filter coefficient for echo cancellation (EC) is used as a coefficient of a transfer function when the reproduced audio signal is adapted so that the reproduced audio component included in the audio signal is set to “0”.

  The filter coefficient estimator 232 updates the filter coefficient for echo cancellation (EC) based on the reproduced audio signal output from the A / D 140c, and sends the updated filter coefficient to the FIR filter 231. Further, the filter coefficient estimator 232 controls the update of the filter coefficient for echo cancellation (EC) based on the sound reproduction flag sent from the robot controller 170. Note that the filter coefficient estimator 232 has an execution determination table (see, for example, FIG. 5) similar to the filter coefficient estimator 222 described above, although not shown in the figure.

  For example, if the audio reproduction flag “True” has already been input at the timing of executing the update of the filter coefficient, the filter coefficient estimator 232 determines that the audio reproduction state is in effect and updates the filter coefficient. Further, if the audio reproduction flag “False” has already been input at the timing of executing the filter coefficient update, the filter coefficient estimator 232 determines that there is no audio reproduction state and stops updating the filter coefficient. .

  Note that the filter coefficient estimator 232 sends the filter coefficient used last time to the FIR filter 231 as it is, for example, when updating of the filter coefficient for echo cancellation (EC) is stopped. Note that while the update of the filter coefficient is stopped by the filter coefficient estimator 232, the FIR filter 231 removes the reproduced audio component from the audio signal using the filter coefficient used last time.

  The voice input detector 210, the noise cancellation unit 220, and the echo cancellation unit 230 are integrated circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array). Alternatively, the audio input detector 210, the noise cancellation unit 220, and the echo cancellation unit 230 may be electronic circuits such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit).

[Processing by Noise Eliminator (Example 2)]
7 to 9 are diagrams illustrating the flow of processing by the noise removal device according to the second embodiment. First, the flow of processing by the voice input detector 210 will be described with reference to FIG. As shown in the figure, when the voice input detector 210 receives the voice signal output from the A / D 140a and the noise signal output from the A / D 140b (Yes in step S1), the voice signal is mixed into the noise signal. Is detected (step S2).

  If the sound input detector 210 determines that a sound signal is mixed in the noise signal (Yes in step S2), the sound input flag “True” indicating that the sound signal is mixed in the noise signal. Is sent to the noise canceling unit 220 (step S3). On the other hand, when the voice input detector 210 determines that the voice signal is not mixed in the noise signal (No in step S2), the voice input flag “False” indicating that the voice signal is not mixed in the noise signal. Is sent to the noise canceling unit 220 (step S4).

  Next, the flow of processing by the noise cancellation unit 220 will be described with reference to FIG. As shown in the figure, the filter coefficient estimator 222 performs the update of the filter coefficient, and if at least one of the input audio input flag and the audio reproduction flag is “True” (step S1). Yes), it works as follows. That is, the filter coefficient estimator 222 stops updating the filter coefficient for noise cancellation (NC) (step S2). Then, the filter coefficient estimator 222 sends the previously used filter coefficient for noise cancellation (NC) to the FIR filter 221 (step S3). As a result, while the update of the filter coefficient is stopped by the filter coefficient estimator 222, the FIR filter 221 removes a noise component from the speech signal using the filter coefficient used last time.

  Here, the description returns to step S1. When the filter coefficient estimator 222 executes the update of the filter coefficient, and at least one of the input audio input flag and the audio reproduction flag is not “True” (when both are False) (step S <b> 3) S1 negative), the following operation is performed. That is, the filter coefficient estimator 222 updates the filter coefficient for noise cancellation (NC) (step S4). Then, the filter coefficient estimator 222 sends the updated filter coefficient for noise cancellation (NC) to the FIR filter 221 (step S5). As a result, the FIR filter 221 uses the filter coefficient updated by the filter coefficient estimator 222 to remove noise components from the audio signal.

  Next, the flow of processing by the echo cancellation unit 230 will be described with reference to FIG. As shown in the figure, the filter coefficient estimator 232 operates as follows when the input audio reproduction flag is “True” at the timing of executing the update of the filter coefficient (Yes in step S1). To do. That is, the filter coefficient estimator 232 updates the filter coefficient for echo cancellation (EC) (step S2). Then, the filter coefficient estimator 232 sends the updated filter coefficient for echo cancellation (EC) to the FIR filter 231 (step S3). As a result, the FIR filter 231 uses the filter coefficient updated by the filter coefficient estimator 232 to remove the reproduced audio component from the audio signal.

  Here, the description returns to step S1. The filter coefficient estimator 232 operates as follows when the input audio reproduction flag is not “True” at the timing of executing the update of the filter coefficient (No in Step S1). To do. That is, the filter coefficient estimator 232 stops updating the filter coefficient for echo cancellation (EC) (step S4). Then, the filter coefficient estimator 232 sends the previously used filter coefficient for echo cancellation (EC) to the FIR filter 231 (step S5). As a result, while the filter coefficient estimator 232 stops updating the filter coefficient, the FIR filter 231 removes the reproduced audio component from the audio signal using the filter coefficient used last time.

[Effects of Example 2]
As described above, according to the second embodiment, the noise removal apparatus 200 determines whether or not a sound signal is mixed in the noise signal. If it is determined that the sound signal is mixed, the sound input flag “ True ”is generated. The noise removing apparatus 200 stops updating the filter coefficient when at least one of the input audio reproduction flag and audio input flag is “True” at the timing of updating the noise cancellation filter coefficient.

  That is, the noise removal apparatus 200 stops updating the filter coefficient when the audio signal is mixed in the noise signal. At this time, the noise removal apparatus 200 determines whether or not the audio signal is mixed in the noise signal by determining whether or not the degree of correlation between the audio signal and the noise signal is high. Therefore, it is possible to improve the accuracy of detecting whether or not a sound signal is mixed in the noise signal, and to avoid updating the filter coefficient in a state where the sound signal is mixed in the noise signal. Therefore, as a result, it is possible to accurately remove noise audio components from the audio signal.

  In addition, when the audio reproduction flag that has already been input is “True” at the timing of updating the filter coefficient for echo cancellation, the noise removal apparatus 200 stops updating the filter coefficient.

  That is, the noise removal apparatus 200 can avoid updating the filter coefficient for echo cancellation in a state where audio reproduction is not performed. Therefore, as a result, the reproduced audio component can be accurately removed from the audio signal.

  Note that the noise removal apparatus 200 according to the second embodiment described above does not need to include the echo canceling unit 230 and need only include the noise canceling unit 220 unless echo cancellation is an essential process.

  Further, in the above-described second embodiment, when a person who exists in a range where sound can be input to the sound acquisition microphone 110 is detected, a sound input flag is output from the robot controller 170 to the noise canceling unit 220. It may be.

  FIG. 10 is a diagram illustrating a configuration according to the third embodiment. As shown in the figure, the service providing robot 100 includes an audio acquisition microphone (MIC_S) 110, a noise acquisition microphone (MIC_N) 120, and an audio reproduction speaker 130. Furthermore, the service providing robot 100 includes A / D (analog / digital converters) 140a to 140c, a D / A (digital / analog converter) 150, a voice recognition unit 160, and a robot controller 170, as shown in FIG. As shown in the figure, the noise removal apparatus 200 includes a noise cancellation unit 220 and an echo cancellation unit 230.

  Here, the noise removing apparatus 200 is different from the second embodiment in that the voice input detector 210 is not included. The service providing robot 100 is different from the second embodiment in that the service providing robot 100 newly includes a person detection unit 180 that detects a person existing in a range in which voice can be input to the voice acquisition microphone 110.

  The person detection unit 180 has a vision (camera) and detects a person who exists in a range where sound can be input to the sound acquisition microphone 110. For example, if the person detection unit 180 detects a person within a certain distance (for example, 100 cm) in the direction of the sound acquisition microphone 110 using a vision (camera), the person detection unit 180 should output the sound input flag “True”. Is sent to the robot controller 170. It should be noted that the human detection unit 180 performs image recognition on the image data captured by the vision using existing techniques such as edge extraction and pattern matching, and enters a range in which sound can be input to the sound acquisition microphone 110. Detect existing people.

  The human detection unit 180 is not a vision (camera), but a distance measurement sensor (distance sensor) such as an infrared sensor or an ultrasonic sensor attached at a position where an object existing in the direction of the sound acquisition microphone can be detected. Can also be included. In this case, the person detection unit 180 uses a distance sensor to measure the distance from the object for a certain period of time when the object is within a predetermined distance. Then, when there is a change in the distance from the object measured for a certain period of time, the human detection unit 180 sends to the robot controller 170 that the voice input flag “True” should be output. On the other hand, when the distance of the object measured for a certain period of time does not change, the human detection unit 180 sends to the robot controller 170 that the voice input flag “False” should be output. The human detection unit 180 is not limited to the above-described vision (camera) and distance sensor, and any device can be used as long as it detects a person in real time, and these devices are used alone or in combination. You can also

  When the robot controller 170 inputs that the voice input flag should be output from the human detection unit 180, the robot controller 170 outputs the voice input flag to the noise cancellation unit 220.

  When the voice cancel flag is input from the robot controller 170, the noise cancellation unit 220 operates in the same manner as in the second embodiment. That is, the filter coefficient estimator 222 stops updating the filter coefficient for noise cancellation when the voice input flag has already been input from the robot controller 170 at the timing of updating the filter coefficient for noise cancellation.

  Further, the noise removal apparatus 200 described in the second embodiment can be similarly applied to the hands-free telephone 300.

[Configuration of Noise Reduction Device (Example 4)]
FIG. 11 is a diagram illustrating a configuration according to the fourth embodiment. As shown in the figure, the hands-free telephone 300 includes a voice acquisition microphone (MIC_S) 310, a noise acquisition microphone (MIC_N) 320, and a voice reproduction speaker 330. Furthermore, the hands-free telephone 300 includes a noise removing device 200, A / D (analog / digital converters) 340a and 340b, and a D / A (digital / analog converter) 350, as shown in FIG. As shown in the figure, the noise removal apparatus 200 includes a voice input detector 210, a noise cancellation unit 220, an echo cancellation unit 230, and a received voice input detector 240.

  Here, as shown in FIG. 11, the noise removal apparatus 200 is different from the noise removal apparatus 200 according to the second embodiment in that it includes a received voice input detector 240.

  The received voice input detector 240 detects whether or not the received voice corresponding to the far-end speaker signal output via the voice reproduction speaker 330 is mixed in the voice signal. The received voice input detector 240 performs the same process as the process performed by the voice input detector 210 described in the second embodiment, and detects whether the received voice is mixed in the voice signal.

  FIG. 12 is a diagram illustrating the configuration of the received voice input detector 240 according to the fourth embodiment. For example, as shown in the figure, the received voice input detector 240 includes delay taps 241a and 241b, frame division processing units 242a and 242b, a cross-correlation detector 243, a signal level comparator 244, and a flag generator. H.245. The processing of the delay taps 241a and 241b, the frame division processing units 242a and 242b, the cross-correlation detector 243, the signal level comparator 244, and the flag generator 245 is the same as that of the voice input detector 210 of the second embodiment. Because there is, it is explained briefly below.

  The delay taps 241a and 241b adjust a known delay (for example, a delay difference in A / D or a delay in a transmission path). The frame division processing units 242a and 242b divide the signals sent from the delay taps 241a and 241b and send them to the cross correlation detector 243 and the signal level comparator 244, respectively.

  The cross-correlation detector 243 calculates a cross-correlation value indicating the correlation between the voice signal transmitted from the frame division processing unit 242a and the far-end speaker signal transmitted from the frame division processing unit 242b. Then, the cross-correlation detector 243 sets the correlation presence / absence information of “True (= correlation)” or “False (= no correlation)” as a flag generator according to the comparison result between the maximum value of the cross-correlation value and the threshold value. To H.245.

  For example, as shown in FIG. 12, the signal level comparator 244 includes a root mean square calculator 244a and a power comparator 244b. The signal level comparator 244 is a signal level (for example, power value) of the audio signal transmitted from the frame division processing unit 242a and a signal level (for example, power value) of the far-end speaker signal transmitted from the frame division processing unit 242b. ). Then, according to the signal level comparison result, the signal level comparator 244 sends “True (= level difference present)” or “False (= no level difference)” level comparison information to the flag generator 245.

  The flag generator 245 indicates whether or not the far-end speaker signal is mixed in the voice signal based on the correlation presence / absence information and the level comparison information sent from the cross-correlation detector 243 and the signal level comparator 244. A voice input flag is generated, and the generated flag is sent to the noise cancellation unit 220. Based on the presence / absence of correlation information and the level comparison information, the flag generator 245 indicates “True” indicating that the far-end speaker signal is mixed in the voice signal, or whether the far-end speaker signal is mixed in the voice signal. The received voice input flag of “False” indicating that there is no message is sent to the noise canceling unit 220.

[Processing by Noise Reduction Device (Example 4)]
FIGS. 13 to 15 are diagrams illustrating a flow of processing performed by the noise removal device according to the fourth embodiment. First, the flow of processing by the received voice input detector 240 will be described with reference to FIG. As shown in the figure, when the received voice input detector 240 inputs the voice signal and the far-end speaker signal output from the A / D 340a (Yes in step S1), the far-end talker signal is mixed into the voice signal. Is detected (step S2).

  When the received voice input detector 240 determines that the far-end speaker signal is mixed in the voice signal (Yes in step S2), it indicates that the far-end speaker signal is mixed in the voice signal. The received voice input flag “True” is sent to the noise canceling unit 220 (step S3). On the other hand, if the voice input detector 240 determines that the far-end speaker signal is not mixed in the voice signal (No in step S2), it indicates that the far-end speaker signal is not mixed in the voice signal. The voice input flag “False” is sent to the noise canceling unit 220 (step S4).

  Next, the flow of processing by the noise cancellation unit 220 will be described with reference to FIG. As shown in the figure, when the filter coefficient estimator 222 executes the update of the filter coefficient and at least one of the input voice input flag and the received voice input flag is “True” (step S1). Yes), it works as follows. That is, the filter coefficient estimator 222 stops updating the filter coefficient for noise cancellation (NC) (step S2). Then, the filter coefficient estimator 222 sends the previously used filter coefficient for noise cancellation (NC) to the FIR filter 221 (step S3). As a result, while the update of the filter coefficient is stopped by the filter coefficient estimator 222, the FIR filter 221 removes a noise component from the speech signal using the filter coefficient used last time.

  Here, the description returns to step S1. When the filter coefficient estimator 222 executes the update of the filter coefficient and at least one of the input voice input flag and the received voice input flag is not “True” (when both are False), In step S1, the operation is as follows. That is, the filter coefficient estimator 222 updates the filter coefficient for noise cancellation (NC) (step S4). Then, the filter coefficient estimator 222 sends the updated filter coefficient for noise cancellation (NC) to the FIR filter 221 (step S5). As a result, the FIR filter 221 uses the filter coefficient updated by the filter coefficient estimator 222 to remove noise components from the audio signal.

  Note that the noise canceling unit 220 according to the fourth embodiment, like the second embodiment described above, is configured to play a sound when an audio playback flag can be present (for example, when the hands-free phone 300 has a content playback function or the like). The update of the filter coefficient may be controlled in consideration of the reproduction flag. For example, when at least one of the input voice input flag, received voice input flag, or voice reproduction flag is “True” at the filter coefficient update timing, the update of the noise cancellation filter coefficient is stopped.

  Next, the flow of processing by the echo cancellation unit 230 will be described with reference to FIG. As shown in the figure, the filter coefficient estimator 232 performs the update of the filter coefficient, and if the input received voice input flag is “True” (Yes at Step S1), the following is performed. Operate. That is, the filter coefficient estimator 232 updates the filter coefficient for echo cancellation (EC) (step S2). Then, the filter coefficient estimator 232 sends the updated filter coefficient for echo cancellation (EC) to the FIR filter 231 (step S3). As a result, the FIR filter 231 removes the received voice component from the voice signal using the filter coefficient updated by the filter coefficient estimator 232.

  Here, the description returns to step S1. The filter coefficient estimator 232 performs the update of the filter coefficient, and when the received voice input flag that has already been input is not “True” (in the case of False) (No in step S1), the following is performed. Operate. That is, the filter coefficient estimator 232 stops updating the filter coefficient for echo cancellation (EC) (step S4). Then, the filter coefficient estimator 232 sends the previously used filter coefficient for echo cancellation (EC) to the FIR filter 231 (step S5). As a result, while the filter coefficient update is stopped by the filter coefficient estimator 232, the FIR filter 231 removes the received voice component from the voice signal using the filter coefficient used last time.

  That is, unlike the second embodiment described above, the echo cancellation unit 230 according to the fourth embodiment controls the update of the filter coefficient for echo cancellation based on the received voice input flag instead of the voice reproduction flag.

[Effects of Example 4]
As described above, according to the fourth embodiment, the noise removal apparatus 200 detects whether or not the far-end speaker signal is mixed in the voice signal, and generates a received voice input flag corresponding to the detection result. . Then, the updating of the filter coefficient is controlled based on the voice input flag and the received voice input flag. For example, when the far-end speaker signal is mixed in the voice signal, the update of the noise canceling filter coefficient is stopped.

  That is, the noise removal apparatus 200 stops updating the filter coefficient when the far-end speaker signal is mixed in the voice signal. At this time, the noise removal apparatus 200 determines whether or not the far-end speaker signal is mixed in the speech signal by determining whether or not the degree of correlation between the speech signal and the far-end speaker signal is high. . Therefore, it is possible to improve the accuracy of detecting whether or not the far-end speaker signal is mixed in the voice signal, and to avoid updating the filter coefficient in a state where the far-end speaker signal is mixed in the voice signal. . As a result, the noise component can be accurately removed from the audio signal.

  In addition, since the noise removal apparatus 200 can avoid updating the filter coefficient for echo cancellation when the far-end speaker signal is not mixed in the speech signal, the received speech component can be accurately removed from the speech signal.

  Further, when a beamform type microphone is applied instead of the voice acquisition microphone 110 of the second embodiment, the following problems can be considered. That is, the characteristic (echo characteristic) of the sound that circulates from the sound reproduction speaker 130 to the beamform microphone varies depending on the directivity direction of the beamform microphone. For this reason, for example, immediately after moving the directivity direction of the beam form microphone, a delay occurs in the follow-up of the filter coefficient update in the echo canceling unit 230, resulting in deterioration of the audio signal.

  In addition, it is conceivable that the characteristics of noise input to the beamform microphone differ depending on the directivity direction of the beamform microphone. Therefore, similar to the echo canceling unit 230 described above, immediately after the beamform microphone is moved in the directing direction, the noise canceling unit 220 also has a delay in tracking the filter coefficient update, and the audio signal may be deteriorated. is there.

  Therefore, in a fifth embodiment below, an embodiment will be described that deals with a delay that occurs in the follow-up of the filter coefficient update when a beamform microphone is applied instead of the sound acquisition microphone 110.

[Configuration of Noise Reduction Device (Embodiment 5)]
FIG. 16 is a diagram illustrating a configuration according to the fifth embodiment. As shown in the figure, the service providing robot 100 includes an audio acquisition beamform microphone (MIC_SB) 191, a noise acquisition microphone (MIC_N) 120, and an audio reproduction speaker 130. Furthermore, the service providing robot 100 includes A / Ds 140a to 140c, a D / A 150, a voice recognition unit 160, a robot controller 170, a human detection unit 180, and an array microphone control unit 192, as shown in FIG.

  The beamform type microphone 191 for voice acquisition has directivity and accepts input of uttered voices mainly emitted from users of the service providing robot 100.

  The noise acquisition microphone 120 accepts input of environmental sounds other than uttered voices such as announcements and BGM that are mainly flowing in the environment surrounding the service providing robot 100. The audio reproduction speaker 130 outputs the audio reproduced by the service providing robot 100 to the user.

  The A / D 140 a converts an analog audio signal input via the audio acquisition beamform microphone 191 into a digital audio signal and outputs the digital audio signal to the noise removal apparatus 200. The A / D 140 b converts an analog noise signal input via the noise acquisition microphone 120 into a digital noise signal and outputs the digital noise signal to the noise removal apparatus 200. The A / D 140 c converts an analog reproduced audio signal input via a D / A 150 described later into a digital reproduced audio signal, and outputs the digital reproduced audio signal to the noise removal apparatus 200.

  The speech recognition unit 160 executes recognition processing of the speech signal output from the noise removal device 200 and sends the recognition result to the robot controller 170.

  The robot controller 170 generates a digital playback audio signal according to the voice recognition result sent from the voice recognition unit 160 and sends the generated playback voice signal to the D / A 150. In addition, when the robot controller 170 sends a playback sound signal to the D / A 150, the noise removal device 200 (filter coefficient estimation), which will be described later, indicates a sound playback flag indicating that the sound using the service providing robot 100 as a sound source is played back. Output to the units 222 and 232). For example, the robot controller 170 outputs “True (= sound playback)” as a sound playback flag when in the sound playback state, and “False (= no sound playback) when in the sound non-playback state. "Is output as an audio reproduction flag.

  In addition, the robot controller 170 performs an array microphone control unit according to position information (position information of a person detected in a range where sound can be input to the sound acquisition beamform microphone 191) sent from the person detection unit 180 described later. A direction setting instruction is sent to 192.

  The D / A 150 converts a digital playback audio signal sent from a robot controller 170 (to be described later) into an analog signal and sends it to the audio playback speaker 130.

  The array microphone control unit 192 controls the directivity direction of the sound acquisition beamform microphone 191 by setting the directivity direction for receiving sound input to the sound acquisition beamform microphone 191. For example, the array microphone control unit 192 sets the directivity direction for the sound acquisition beamform microphone 191 in response to the directivity direction setting instruction sent from the robot controller 170. Then, the array microphone control unit 192 sends beamforming control information indicating the directivity direction of the sound acquisition beamform type microphone 191 to the noise canceling unit 220 and the echo canceling unit 230 described later.

  The human detection unit 180 includes a vision (camera), a distance measurement sensor (distance sensor) such as an infrared sensor or an ultrasonic sensor, and is within a range in which sound can be input to the beamform microphone 191 for sound acquisition. Detect existing people. For example, when the person detection unit 180 detects a person within a certain distance (for example, 100 cm) in the direction of the sound acquisition beamform microphone 191 using a vision (camera) or a distance sensor, the position of the detected person is detected. Information is output to the robot controller 170.

  When the beamform type microphone 191 for voice acquisition has a function of automatically following the directivity direction with respect to the voice input reception direction, beamforming control information indicating the directivity direction is transmitted to the array microphone control unit 192 described later. It can also be sent out. The array microphone control unit 192 sends the directivity directions sent from the sound acquisition beamform microphone 191 to the noise canceling unit 220 and the echo canceling unit 230, which will be described later.

  The noise removal apparatus 200 outputs to the voice recognition unit 160 a voice signal obtained by removing a noise component and a reproduced voice component from the voice signal output from the A / D 140a by a noise cancellation unit 220 and an echo cancellation unit 230 described later. The noise removal apparatus 200 includes, for example, a voice input detector 210, a noise cancellation unit 220, and an echo cancellation unit 230, as shown in FIG.

  The voice input detector 210 uses the voice signal output from the A / D 140a and the noise signal output from the A / D 140b to detect whether the voice signal is mixed in the noise signal. Then, the voice input detector 210 sends a voice input flag indicating whether or not a voice signal is mixed in the noise signal to a noise cancellation unit 220 (filter coefficient estimator 222) described later.

  For example, the voice input detector 210 detects the phase in which the voice signal and the noise signal have the highest correlation, as in the second embodiment. Then, the audio input detector 210 superimposes the audio signal and the noise signal with the phase having the highest correlation, calculates the difference between the average values of the power values of the signals, and the calculated difference exceeds a predetermined threshold value. It is determined whether or not. Then, the voice input detector 210 generates a voice input flag indicating whether or not a voice signal is mixed in the noise signal based on the determination result, and uses the generated voice input flag as the noise cancellation unit 220 (filter coefficient estimation). Device 222). For example, when the audio signal is mixed in the noise signal, the audio input detector 210 sends “True (= mixed)” as the audio input flag, and the audio signal is not mixed in the noise signal. "False" (= no mixing) is transmitted as a voice input flag.

  As shown in FIG. 16, the noise cancellation unit 220 includes an FIR filter 221, a filter coefficient estimator 222, and a filter coefficient initial value memory 223.

  The FIR filter 221 uses the noise cancel (NC) filter coefficient sent from the filter coefficient estimator 222 to remove a noise component from the audio signal output from the A / D 140a. Note that the filter coefficient for noise cancellation (NC) is used as a coefficient of a transfer function when the noise signal is adapted so that the noise component included in the audio signal is “0”.

  The filter coefficient initial value memory 223 stores an initial value of a filter coefficient for noise cancellation (NC) that can be set in advance for each directivity direction in association with the directivity direction of the beamform microphone 191 for sound acquisition. .

  The filter coefficient estimator 222 updates the filter coefficient for noise cancellation (NC) based on the noise signal output from the A / D 140 b, and sends the updated filter coefficient to the FIR filter 221. For example, the filter coefficient estimator 222 first determines the initial value of the filter coefficient for noise cancellation (NC) corresponding to the beamforming control information (information indicating the pointing direction) sent from the array microphone control unit 192 as the filter coefficient initial value. Read from the value memory 223.

  The filter coefficient estimator 222 controls the update of the filter coefficient for echo cancellation (EC) based on the sound reproduction flag sent from the robot controller 170. For example, if both the audio reproduction flag “False” and the audio input flag “False” have already been input at the timing of executing the update of the filter coefficient, the filter coefficient estimator 222 indicates that “the audio signal to the noise signal”. "No mixing" and "no sound reproduction state". Then, the filter coefficient estimator 222 uses the initial value read from the filter coefficient initial value memory 223 so that the noise component included in the audio signal is set to “0”. Update. After the filter coefficient is updated, the filter coefficient estimator 222 sends the updated noise cancellation (NC) filter coefficient to the FIR filter 221.

  Note that the filter coefficient estimator 222 overwrites and updates the updated filter coefficient in the filter coefficient initial value memory 223 in association with the beamforming control information (information indicating the directivity direction).

  In addition, for example, when the filter coefficient estimator 222 has input at least one of the audio reproduction flag “True” and the audio input flag “True” at the timing of executing the update of the filter coefficient, Stop updating coefficients. When the update of the filter coefficient is stopped, the filter coefficient estimator 222 sends the previously used filter coefficient for noise cancellation (NC) to the FIR filter 221.

  As shown in FIG. 16, the echo cancellation unit 230 includes an FIR filter 231, a filter coefficient estimator 232, and a filter coefficient initial value memory 233.

  The FIR filter 231 uses the echo cancel (EC) filter coefficient sent from the filter coefficient estimator 232 to remove the reproduced audio component from the audio signal output from the A / D 140a. Note that the filter coefficient for echo cancellation (EC) is used as a coefficient of a transfer function when the reproduced audio signal is adapted so that the reproduced audio component included in the audio signal is set to “0”.

  The filter coefficient initial value memory 233 stores an initial value of a filter coefficient for echo cancellation (EC) that can be set in advance for each directivity direction in association with the directivity direction of the beamform microphone 191 for sound acquisition. .

  The filter coefficient estimator 232 updates the filter coefficient for echo cancellation (EC) based on the reproduced audio signal output from the A / D 140c, and sends the updated filter coefficient to the FIR filter 231. For example, the filter coefficient estimator 232 first determines the initial value of the filter coefficient for echo cancellation (EC) corresponding to the beamforming control information (information indicating the pointing direction) sent from the array microphone control unit 192 as the filter coefficient initial value. Read from the value memory 233.

  The filter coefficient estimator 232 controls the update of the filter coefficient for echo cancellation (EC) based on the sound reproduction flag sent from the robot controller 170. For example, the filter coefficient estimator 232 determines “audio reproduction state” when the audio reproduction flag “True” has already been input at the timing of updating the filter coefficient. Then, the filter coefficient estimator 232 uses the initial value read from the filter coefficient initial value memory 233 to set the reproduced audio component included in the audio signal to “0”, so that the filter coefficient for echo cancellation (EC) is set. Update. After updating the filter coefficient, the filter coefficient estimator 232 sends the updated filter coefficient for echo cancellation (EC) to the FIR filter 231.

  The filter coefficient estimator 232 updates the filter coefficient initial value memory 233 by overwriting the updated filter coefficient in association with the beamforming control information (information indicating the directivity direction).

  In addition, when the audio reproduction flag “False” has already been input at the timing of executing the filter coefficient update, the filter coefficient estimator 232 determines “no sound reproduction state” and stops updating the filter coefficient. . When the update of the filter coefficient is stopped, the filter coefficient estimator 232 sends the previously used filter coefficient for echo cancellation (EC) to the FIR filter 231.

  For example, if the audio reproduction flag “True” has already been input at the timing of executing the update of the filter coefficient, the filter coefficient estimator 232 determines that the audio reproduction state is in effect and updates the filter coefficient. Further, if the audio reproduction flag “False” has already been input at the timing of executing the filter coefficient update, the filter coefficient estimator 232 determines that there is no audio reproduction state and stops updating the filter coefficient. .

[Effects of Example 5]
As described above, according to the fifth embodiment, the noise removal apparatus 200 stores the filter for noise cancellation corresponding to the directivity direction that can be set in advance in the filter coefficient initial value memory 223 with respect to the beamform microphone 191 for voice acquisition. The initial value of the coefficient is stored. In the noise removal apparatus 200, the filter coefficient initial value memory 233 stores the initial value of the filter coefficient for echo cancellation corresponding to the directivity direction that can be set in advance for the beamform microphone 191 for sound acquisition. Then, even immediately after moving the directional direction of the beamform microphone, the filter coefficient for noise cancellation and echo cancellation is updated using the initial value of the filter coefficient corresponding to the directional direction. For this reason, it is possible to avoid a delay that occurs in tracking the filter coefficient update.

  Further, in the sixth embodiment described below, an embodiment will be described in which playback audio using the service providing robot 100 as a sound source is also input by the A / D 140b that mainly inputs noise from a noise acquisition microphone. The noise removal apparatus 200 according to the sixth embodiment updates a filter coefficient for noise or echo cancellation by a single processing unit integrated physically or functionally.

  FIG. 17 is a diagram illustrating a configuration according to the sixth embodiment. As shown in the figure, the service providing robot 100 includes an audio acquisition microphone (MIC_S) 110, a noise acquisition microphone (MIC_N) 120, and an audio reproduction speaker 130. Furthermore, the service providing robot 100 includes A / Ds 140a and 140b, a D / A 150, a voice recognition unit 160, and a robot controller 170, as shown in FIG.

  The voice acquisition microphone 110 accepts input of uttered voice mainly from a user of the service providing robot 100. The noise acquisition microphone 120 accepts input of environmental sounds other than uttered voices such as announcements and BGM that are mainly flowing in the environment surrounding the service providing robot 100. The audio reproduction speaker 130 outputs the audio reproduced by the service providing robot 100 to the user.

  The A / D 140 a converts an analog audio signal input via the audio acquisition microphone 110 into a digital audio signal and outputs the digital audio signal to the noise removal apparatus 200.

  The A / D 140 b converts an analog noise signal input via the noise acquisition microphone 120 into a digital noise signal and outputs the digital noise signal to the noise removal apparatus 200. In addition, the A / D 140 b converts an analog reproduced audio signal input via a D / A 150 described later into a digital audio signal, and outputs the digital audio signal to the noise removal apparatus 200.

  The speech recognition unit 160 executes speech signal recognition processing received from the noise removal device 200 and sends the recognition result to the robot controller 170.

  The robot controller 170 generates a digital playback audio signal according to the voice recognition result sent from the voice recognition unit 160 and sends the generated playback voice signal to the D / A 150. In addition, when the robot controller 170 sends a playback sound signal to the D / A 150, the noise removal device 200 (filter coefficient estimation), which will be described later, indicates a sound playback flag indicating that the sound using the service providing robot 100 as a sound source is played back. To the device 252). For example, the robot controller 170 outputs “True (= sound playback)” as a sound playback flag when in the sound playback state, and “False (= no sound playback) when in the sound non-playback state. "Is output as an audio reproduction flag.

  The D / A 150 converts a digital playback audio signal sent from a robot controller 170 (to be described later) into an analog signal and sends it to the audio playback speaker 130.

  The noise removal apparatus 200 outputs, to the speech recognition unit 160, a speech signal obtained by removing a noise component and a reproduced speech component from the speech signal output from the A / D 140a by a noise cancellation / echo cancellation unit 250 described later. For example, as shown in FIG. 17, the noise removal apparatus 200 includes a voice input detector 210 and a noise cancellation / echo cancellation unit 250.

  The voice input detector 210 uses the voice signal output from the A / D 140a and the noise signal output from the A / D 140b to detect whether the voice signal is mixed in the noise signal. Then, the voice input detector 210 sends a voice input flag indicating whether or not a voice signal is mixed in the noise signal to a noise cancellation / echo cancellation unit 250 (filter coefficient estimator 252) described later.

  For example, the voice input detector 210 detects the phase in which the voice signal and the noise signal have the highest correlation, as in the second embodiment. Then, the audio input detector 210 superimposes the audio signal and the noise signal with the phase having the highest correlation, calculates the difference between the average values of the power values of the signals, and the calculated difference exceeds a predetermined threshold value. It is determined whether or not. Then, the voice input detector 210 generates a voice input flag indicating whether or not a voice signal is mixed in the noise signal based on the determination result, and uses the generated voice input flag as a noise cancel / echo cancel unit 250 ( To the filter coefficient estimator 252). For example, when the audio signal is mixed in the noise signal, the audio input detector 210 sends “True (= mixed)” as the audio input flag, and the audio signal is not mixed in the noise signal. "False" (= no mixing) is transmitted as a voice input flag.

  As shown in FIG. 17, the noise cancellation / echo cancellation unit 250 includes an FIR filter 251, a filter coefficient estimator 252, and a filter coefficient initial value memory 253.

  The FIR filter 251 uses a noise cancel (NC) or echo cancel (EC) filter coefficient sent from the filter coefficient estimator 252 to generate a noise component or a reproduced voice component from the voice signal output from the A / D 140a. Remove. Note that the filter coefficient for noise cancellation (NC) is used as a coefficient of a transfer function when the noise signal is adapted so that the noise component included in the audio signal is “0”. Further, the filter coefficient for echo cancellation (EC) is used as a coefficient of a transfer function when the reproduced audio signal is adapted so that the reproduced audio component included in the audio signal is set to “0”.

  The filter coefficient initial value memory 253 stores an initial value of a filter coefficient for noise cancellation (NC) and an initial value of a filter coefficient for echo cancellation (EC).

  The filter coefficient estimator 252 updates the filter coefficient for noise cancellation (NC) or echo cancellation (EC) on the basis of the noise signal or reproduced audio signal output from the A / D 140b, and uses the updated filter coefficient. Send to the FIR filter 251.

  FIG. 18 shows a correspondence relationship between the flag state and the operation of the filter coefficient estimator 252. FIG. 18 is a diagram illustrating the noise removal device according to the sixth embodiment. The items in the column in the figure indicate the types of signals input to the filter coefficient estimator 252, and the items in the row in the figure indicate the flag contents corresponding to the input signals and the initial values of the filter coefficients to be loaded. Indicates whether the type and filter coefficient are updated.

  For example, as shown in the leftmost column item in FIG. 18, the contents of the flag when only the noise signal is input to the filter coefficient estimator 252 are the audio input flag “OFF” and the audio reproduction flag “OFF”. " The voice input flag “ON” is a flag indicating that the voice signal is mixed in the noise signal, and the voice input flag “OFF” is a flag indicating that the voice signal is not mixed in the noise signal. is there. The audio reproduction flag “ON” is a flag indicating that the reproduced audio signal is mixed in the audio signal, and the audio reproduction flag “OFF” is a flag indicating that the reproduced audio signal is not mixed in the audio signal. is there.

  For example, when only the noise signal is input to the filter coefficient estimator 252, the type of the initial value of the filter coefficient loaded from the filter coefficient initial value memory 253 is “NC”. “NC” represents an initial value of a filter coefficient for noise cancellation. In addition, when only the noise signal is input to the filter coefficient estimator 252, whether or not the filter coefficient is updated is “ON”. The filter coefficient update “ON” indicates that the filter should be updated, and the filter coefficient update “OFF” indicates that the filter update should be stopped.

  Further, for example, as shown in the second column from the left in the column shown in FIG. 18, the content of the flag when only the audio signal is input to the filter coefficient estimator 252 is the audio input flag “ON”, the audio reproduction The flag becomes “OFF”. When only the audio signal is input to the filter coefficient estimator 252, the type of the initial value of the filter coefficient loaded from the filter coefficient initial value memory 253 is “NC”. Also, whether or not the filter coefficient is updated when only the audio signal is input to the filter coefficient estimator 252 is “OFF”.

  Further, for example, as shown in the third column from the left in the column shown in FIG. 18, the contents of the flag when only the reproduced audio signal is input to the filter coefficient estimator 252 are the audio input flag “OFF”, the audio The reproduction flag is “ON”. When only the reproduced audio signal is input to the filter coefficient estimator 252, the type of the initial value of the filter coefficient loaded from the filter coefficient initial value memory 253 is “EC”. “EC” indicates an initial value of a filter coefficient for echo cancellation. Further, whether or not the filter coefficient is updated when only the reproduced audio signal is input to the filter coefficient estimator 252 is “ON”.

  For example, as shown in the fourth column from the left in the column shown in FIG. 18, the content of the flag when the audio signal and the reproduced audio signal are input to the filter coefficient estimator 252 is the audio input flag “ON”. The audio reproduction flag is “ON”. When the audio signal and the reproduced audio signal are input to the filter coefficient estimator 252, the type of the initial value of the filter coefficient loaded from the filter coefficient initial value memory 253 is “EC”. Further, whether the filter coefficient is updated or not when the audio signal and the reproduced audio signal are input to the filter coefficient estimator 252 is “OFF”.

  When updating the filter coefficients, the filter coefficient estimator 252 reads the filter coefficients for noise cancellation (NC) or echo cancellation (EC) from the filter coefficient initial value memory 253 according to the correspondence shown in FIG. Read the initial value. Then, the filter coefficient estimator 252 updates the filter coefficient in accordance with the correspondence shown in the figure, using the read initial value of the filter coefficient.

  For example, when only the noise signal is input, the filter coefficient estimator 252 loads the initial value of the filter coefficient for noise cancellation according to the correspondence shown in FIG. Then, the filter coefficient estimator 252 updates the filter coefficient for noise cancellation using the initial value of the filter coefficient for noise cancellation.

  For example, when only the audio signal is input, the filter coefficient estimator 252 loads the initial value of the filter coefficient for noise cancellation according to the correspondence relationship shown in FIG. Then, the filter coefficient estimator 252 does not update the noise canceling filter coefficient using the initial value of the noise canceling filter coefficient, and sends the loaded initial value of the noise canceling filter coefficient to the FIR filter 251. To do. The FIR filter 251 removes a noise component from the audio signal using the initial value of the filter coefficient for noise cancellation.

  Further, for example, when only the reproduced audio signal is input, the filter coefficient estimator 252 loads the initial value of the filter coefficient for echo cancellation according to the correspondence shown in FIG. Then, the filter coefficient estimator 252 updates the filter coefficient for echo cancellation using the initial value of the filter coefficient for echo cancellation.

  For example, when both the audio signal and the audio reproduction signal are input, the filter coefficient estimator 252 loads the initial value of the filter coefficient for echo cancellation according to the correspondence relationship shown in FIG. Then, the filter coefficient estimator 252 does not update the filter coefficient for echo cancellation using the initial value of the filter coefficient for echo cancellation, and sends the initial value of the loaded filter coefficient for echo cancellation to the FIR filter 251. To do. The FIR filter 251 removes the reproduced audio component from the audio signal using the initial value of the filter coefficient for echo cancellation.

  The filter coefficient estimator 252 overwrites and updates the updated filter coefficient in the filter coefficient initial value memory 253.

[Effects of Example 6]
As described above, according to the sixth embodiment, the A / D 140b that mainly inputs noise from the noise acquisition microphone 120 also performs the above-described implementation even when the reproduction sound using the service providing robot 100 as a sound source is input. The filter coefficient can be updated in the same manner as in Example 2.

  Hereinafter, other embodiments of the noise removal device disclosed in the present application will be described.

(Equipment configuration etc.)
For example, each component of the noise removal apparatus 200 shown in FIG. 2 is functionally conceptual and does not necessarily need to be physically configured as illustrated. That is, the specific form of the dispersion / integration of the noise removal apparatus 200 is not limited to that shown in the figure. For example, the FIR filter 221 and the filter coefficient estimator 222 of the noise cancellation unit 220 are functionally or physically integrated. As described above, all or a part of the noise removal apparatus 200 can be configured to be functionally or physically distributed / integrated in arbitrary units according to various loads or usage conditions.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary Note 1) The first input of the first sound mainly using the filter coefficient updated based on the second signal acquired from the second input unit to which the second sound is input. A removing unit that removes the second audio component from the first signal acquired from the input unit;
A correlation value indicating a correlation between the first signal and the second signal is calculated, it is determined whether the calculated correlation value exceeds a predetermined threshold, and the first signal The output level of the second signal is compared with the output level of the second signal to determine whether the difference between the output level of the first signal and the output level of the second signal exceeds a predetermined threshold, If it is determined that the correlation value exceeds a predetermined threshold and the output level difference exceeds a predetermined threshold, the first audio component is mixed into the second signal. A contamination detection unit for detecting that there is,
A controller that controls to stop updating the filter coefficient when the mixing detection unit detects that the first audio component is mixed in the second signal. A featured noise removal device.

(Supplementary Note 2) Using the filter coefficient updated based on the third signal, which is a signal corresponding to the third sound reproduced and output from the sound output device having the noise removal device, from the first input unit A reproduced sound removing unit for removing the component of the third sound from the output first signal;
A reproduction detector for detecting reproduction of the third sound;
Further comprising
The control unit controls to stop the update of the filter coefficient that is updated based on the second signal when the reproduction detection unit detects the reproduction of the third sound. The noise removal apparatus according to appendix 1, wherein the filter coefficient updated based on the signal 3 is updated.

(Additional remark 3) The 1st audio | voice is mainly input 1st using the filter coefficient updated based on the 2nd signal acquired from the 2nd input part from which the 2nd audio | voice is mainly input. A removing unit that removes the second audio component from the first signal acquired from the input unit;
A person detection unit for detecting a person existing in a range in which the first voice can be input to the first input unit;
The noise removal apparatus according to appendix 1, further comprising: a control unit that controls to stop updating the filter coefficient when a person is detected by the human detection unit.

(Supplementary Note 4) A first input unit having directivity and mainly receiving a first sound;
A second input unit mainly receiving a second sound;
A third input unit to which a third sound mainly using a noise removing device as a sound source is input;
An initial value of the first filter coefficient updated based on the second signal acquired from the second input unit is stored for each directivity direction in association with the directivity direction of the first input unit. A first storage unit
The initial value of the second filter coefficient updated based on the third signal acquired from the third input unit is stored for each directivity direction in association with the directivity direction of the first input unit. A second storage unit to
A person detection unit for detecting a person existing in a range in which the first voice can be input to the first input unit;
A direction control unit that controls to direct the pointing direction of the first input unit in the detected direction when a person is detected by the person detection unit;
A mixing detector that detects whether or not the first audio component is mixed in the second signal;
A reproduction detector for detecting reproduction of the third sound;
The initial value of the first filter coefficient corresponding to the directivity direction controlled by the direction control unit is read from the first storage unit according to the detection result by the mixing detection unit and the detection result by the regeneration detection unit. A first updating unit that updates the first filter coefficient using the read initial value and stores the updated first filter coefficient in the first storage unit instead of the read initial value; ,
The initial value of the second filter coefficient corresponding to the directivity direction controlled by the direction control unit is read from the second storage unit according to the detection result by the mixing detection unit and the detection result by the regeneration detection unit. A second updating unit that updates the second filter coefficient using the read initial value and stores the updated second filter coefficient in the second storage unit instead of the read initial value; A noise removal apparatus comprising:

(Additional remark 5) The 1st input part into which the 1st audio | voice is mainly input,
A second input unit to which a third sound using the second sound and the noise removing device as a sound source is input;
A first storage unit that stores an initial value of a first filter coefficient that is updated based on a second signal acquired from the second input unit;
A second storage unit that stores an initial value of a second filter coefficient that is updated based on a third signal acquired from the second input unit;
A mixing detector that detects whether or not the first audio component is mixed in the second signal;
A reproduction detector for detecting reproduction of the third sound;
When the mixing detection unit detects that the first audio component is not mixed in the second signal, the initial value of the first filter coefficient is read from the first storage unit. The first filter coefficient is updated using the read initial value, the updated first filter coefficient is stored in the first storage unit, and the reproduction detection unit reproduces the third sound. If detected, the initial value of the second filter coefficient is read from the second storage unit, the second filter coefficient is updated using the read initial value, and the updated second value And an updating unit that stores the filter coefficient in the second storage unit.

DESCRIPTION OF SYMBOLS 100 Service provision robot 110 Sound acquisition microphone 120 Noise acquisition microphone 130 Sound reproduction speaker 140a-140c A / D (analog-digital converter)
150 D / A (digital / analog converter)
DESCRIPTION OF SYMBOLS 160 Speech recognition part 170 Robot controller 180 Human detection part 191 Beam form type | mold microphone 192 for voice acquisition Array microphone control part 200 Noise removal apparatus 210 Voice input detector 211a, 211b Delay tap 212a, 212b Frame division | segmentation process part 213 Cross correlation detector 214 signal level comparator 214a root mean square calculator 214b power comparator 215 flag generator 220 noise canceling unit 221 FIR (Finite impulse response) filter 222 filter coefficient estimator 223 filter coefficient initial value memory 230 echo canceling unit 231 FIR (Finite impulse) response) filter 232 filter coefficient estimator 233 filter coefficient initial value memory 240 received voice input detector 241a, 241b delay tap 242a, 242b Frame division processing unit 243 Cross correlation detector 244 Signal level comparator 244a Root mean square calculator 244b Power comparator 245 Flag generator 250 Noise cancellation / echo cancellation unit 251 FIR filter 252 Filter coefficient estimator 253 Filter coefficient initial value memory 300 Hands-free telephone 310 Microphone for sound acquisition 320 Microphone for noise acquisition 330 Sound reproduction speaker 340a, 340b A / D (analog / digital converter)
350 D / A (digital / analog converter)

Claims (2)

The first input unit is input using the first filter coefficient updated based on the second signal output corresponding to the second voice input by the second input unit. The second audio component is removed from the first signal output corresponding to the audio, and the second signal is updated based on the third signal corresponding to the third audio output from the audio output device. A removing unit that removes the third audio component from the first signal using the filter coefficient of
A correlation value indicating a correlation between the first signal and the second signal is calculated, it is determined whether the calculated correlation value exceeds a predetermined threshold, and the first signal The output level of the second signal is compared with the output level of the second signal to determine whether the difference between the output level of the first signal and the output level of the second signal exceeds a predetermined threshold, it is determined that the correlation value exceeds a predetermined threshold value, and when the difference of the output level is judged to exceed a predetermined threshold, the components of the first audio is included in the second signal And the fact that the third signal is output is detected from the audio output device, thereby detecting that the third audio component is included in the first signal. A detection unit;
When the detection unit detects that the second signal includes the first audio component , or the detection unit includes the third audio component. If the fact that is detected, controls so as to stop updating the first filter coefficients, it contains components of the third speech to the first signal by the detecting unit And a control unit that controls to stop the update of the second filter coefficient when it is not detected . The removal unit directs the first input unit according to the position of the speaker detected by the detection unit that detects the position of the speaker who speaks toward the first input unit which is a beamform microphone. The first filter coefficient and the second filter coefficient that are updated for each directivity direction of the first input unit controlled by a directivity direction control unit that controls the direction, and controlled by the directivity direction control unit Using the first filter coefficient and the second filter coefficient corresponding to the directivity direction of the first input unit, the second sound component and the third sound component from the first signal. Remove each component
The noise removal apparatus according to claim 1, wherein

Images