JP2014102318A - Noise elimination device, noise elimination method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a noise elimination device capable of effectively reducing wind noise, a noise elimination method, and a program.SOLUTION: A noise elimination device 100 comprises: a subtractor 11 for calculating a difference signal between an audio signal of one channel and an audio signal of another channel in a plurality of audio signals which are input from a plurality of audio channels; STFT processing units 13, 14, 15 for, after performing frame division on the audio signal and the difference signal in a time domain respectively, converting the signals into an audio signal and a difference signal in a frequency domain; coefficient multiplication/subtraction processing units 16, 17 for, on the basis of the difference signal in the frequency domain and the audio signal in the frequency domain, generating a subtraction signal in the frequency domain; and IFFT processing units 21, 23 for inversely converting the subtraction signal in the frequency domain into a time signal in the time domain.

Description

  The present invention relates to a noise removal device, a noise removal method, and a program.

  Patent Document 1 discloses a method using a spectral subtraction (SS) method. In general, in the SS method, a noise signal is estimated from a sound signal in a silent section, and the noise signal is removed from the sound signal. Specifically, the frequency spectrum of the noise signal is subtracted from the frequency spectrum of the voice signal including noise. The method of Patent Document 1 is to prepare a plurality of noise models in advance and subtract the frequency spectrum of the model selected from them.

  Patent Document 2 discloses another apparatus for reducing wind noise. In the apparatus of Patent Document 2, input signals of a plurality of audio channels are converted into frequency signals by an FFT unit. Then, the frequency signal of the wind noise band is taken out, and the difference between the plurality of audio channels is obtained from both the amplitude and the phase by the amplitude comparison unit and the phase comparison unit. Further, the attenuation coefficient generator converts the wind noise component into an amplitude coefficient for attenuating. The frequency selection / attenuation unit selects one coefficient of amplitude and phase and multiplies the frequency signal in the wind noise band. The band synthesizing unit synthesizes with a frequency signal other than the wind noise band, and the IFFT unit performs inverse conversion to a time signal.

JP 2006-47639 A JP 2010-28307 A

  However,

  However, in the method of Patent Document 1, it is not easy to prepare and select an appropriate model for wind noise that changes from time to time, and as a result, the effect of reducing wind noise is insufficient. . In Patent Document 2, when the wind noise band and the voice band overlap, there is a problem that the voice is simultaneously reduced.

  The present invention has been made in view of the above problems, and an object thereof is to provide a noise removal device, a noise removal method, and a program that can effectively remove wind noise.

A noise reduction apparatus according to an aspect of the present invention calculates a difference signal between a sound signal of one channel and a sound signal of another channel among a plurality of sound signals input from a plurality of sound channels. An audio signal and a difference signal in the frequency domain after the audio signal and the difference signal in the time domain are respectively divided into frames, and a difference signal in the frequency domain and an audio signal in the frequency domain. Based on this, a subtraction processing unit that generates a subtraction signal in the frequency domain and an inverse conversion unit that inversely converts the subtraction signal in the frequency domain into a time signal in the time domain are provided.
The noise removal method according to an aspect of the present invention includes a step of calculating a difference signal between a sound signal of one channel and a sound signal of another channel among a plurality of sound signals input from a plurality of sound channels. The audio signal and the difference signal in the time domain are each divided into frames, and then converted into an audio signal and a difference signal in the frequency domain, and based on the difference signal in the frequency domain and the audio signal in the frequency domain, A step of generating a subtraction signal in the frequency domain; and a step of inversely converting the subtraction signal in the frequency domain into a signal in the time domain.
A program according to an aspect of the present invention is a program for causing a computer to execute a noise removal method for removing noise, wherein the noise removal method includes a plurality of audio signals input from a plurality of audio channels. Calculating a difference signal between an audio signal of one channel and an audio signal of another channel; and dividing the audio signal and the difference signal in the time domain into frames, respectively, and then converting the audio signal and the difference signal in the frequency domain into frames. Converting, generating a subtraction signal in the frequency domain based on the difference signal in the frequency domain and the audio signal in the frequency domain, and inversely converting the subtraction signal in the frequency domain into a signal in the time domain And.

  ADVANTAGE OF THE INVENTION According to this invention, the noise removal apparatus, the noise removal method, and program which can reduce a wind noise effectively can be provided.

1 is a circuit block diagram of a configuration of a noise removal device according to a first exemplary embodiment. FIG. 3 is a diagram illustrating a coefficient multiplication / subtraction processing unit of the noise removal device according to the first exemplary embodiment. FIG. 4 is a circuit block diagram illustrating a configuration of a noise removal device according to a second exemplary embodiment. It is a graph which shows the threshold value for switching a switching part. It is a figure which shows the switching part of the noise removal apparatus concerning Embodiment 2. FIG. FIG. 6 is a circuit block diagram illustrating a configuration of a noise removal device according to a third exemplary embodiment. FIG. 6 is a circuit block diagram illustrating a configuration of a noise removal device according to a fourth exemplary embodiment. FIG. 10 is a diagram illustrating a coefficient multiplication / subtraction processing unit of the noise removal device according to the fourth exemplary embodiment. FIG. 10 is a circuit block diagram of a configuration of a noise removal device according to a fifth exemplary embodiment. FIG. 10 is a circuit block diagram illustrating a configuration of a noise removing device according to a sixth embodiment.

Embodiment 1 FIG.
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram illustrating a configuration of the noise reduction device according to the first embodiment. Stereo right and left channel (Rch, Lch) audio signals are A / D converted from two microphones arranged in the casing of the electronic device and input from the input terminal.

  The noise removal apparatus 100 includes a subtractor 11, a constant multiplier 12, STFT processing units 13 to 15, a coefficient multiplication / subtraction processing unit 16, a coefficient multiplication / subtraction processing unit 17, an IFFT processing unit 21, an IFFT processing unit 23, and a waveform synthesis. A unit 22 and a waveform synthesis unit 24 are provided. The noise removal apparatus 100 may be realized by an analog circuit, a digital circuit, or the like, or may be realized by software such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), or a combination thereof. May be.

  The audio signal Lch from the left microphone is input to the input terminal 101. The audio signal Rch from the right microphone is input to the input terminal 102. The subtractor 11 calculates a difference signal between the Lch and Rch audio signals.

  The difference signal is halved by the constant multiplier 12 and then input to the STFT processing unit 14. The Lch audio signal is input to the STFT processing unit 13, and the Rch audio signal is input to the STFT processing unit 15. The STFT processing units 13 to 15 execute STFT (Short Time Fourier Transform) processing. Specifically, the STFT processing units 13 to 15 perform frame division into frames having a predetermined length while shifting the input difference signal and audio signal every predetermined time. The STFT processing units 13 to 15 perform a process of multiplying each frame divided by a predetermined time window. The STFT processing units 13 to 15 perform FFT (Fast Fourier Transform) processing on the signal subjected to the time window, and output a phase value and an amplitude value at each frequency of each frame. The STFT processing units 13 to 15 serve as a conversion unit that converts the audio signal and the difference signal in the time domain into frames and then converts them into an audio signal and a difference signal in the frequency domain.

  For example, in the case of a 256th order FFT, the frequency signal has 128 amplitude values | Y (t, k × f0) | (K = 0 to 127) and phase value φy (k × f0) (K = 0 to 127). For difference signals, | Ys (t, k × f0) |, φys (k × f0), and for Lch audio signals, | YL (t, k × f0) |, φyL (k × f0), Rch An audio signal is represented by | YR (t, k × f0) | and φyR (k × f0). Of course, the maximum value of k is not limited to 127, and can be any natural number.

The amplitude value data in the low frequency range of the difference signal and the Lch audio signal are input to the coefficient multiplication / subtraction processing unit 16. For example, 32 low-frequency data, | Ys (t, k × f0) |, | YL (t, k × f0) | (K = 0 to 31) are input to the coefficient multiplication / subtraction processing unit 16. The
The amplitude value data in the low frequency range of the difference signal and the Rch audio signal are input to the coefficient multiplication / subtraction processing unit 17. For example, 32 low-frequency data, | Ys (t, k × f0) |, | YR (t, k × f0) | (K = 0 to 31) are input to the coefficient multiplication / subtraction processing unit 17. The

  The coefficient multiplication / subtraction processing units 16 and 17 multiply each amplitude value data of | Ys (t, k × f0) | (K = 0 to 31) by a predetermined coefficient Ck. Therefore, the multiplication value data obtained by multiplying the amplitude value data by the coefficient is Ck × | Ys (t, k × f0) |. Ck is a value that is larger as the frequency is lower and decreases as the frequency is higher. For example, Ck = C × (1− (K / 32) × (K / 32)). C is preferably a value around 1. As will be described later, the low-frequency signal of the difference signal is due to wind noise, but as the frequency becomes higher, the original stereo components other than wind noise are mixed. Therefore, the multiplication value is increased in the lower frequency range. Alternatively, it may be a constant value regardless of K. There is an advantage that the arithmetic circuit can be simplified. Further, a part or all of Ck (K = 0 to 31) may be set to 1.

The coefficient multiplication / subtraction processing unit 16 subtracts the multiplication value data of the coefficient and the amplitude value data of the difference signal from the amplitude value data of the Lch audio signal. That is, the coefficient multiplication / subtraction processing unit 16 calculates subtraction value data obtained by subtracting the multiplication value data from the amplitude value data of the Lch audio signal for each frequency. The coefficient multiplication / subtraction processing unit 16 calculates | YL (t, k * f0) | -Ck * | Ys (t, k * f0) |. | YL (t, k * f0) | -Ck * | Ys (t, k * f0) | is calculated for each k.
Similarly, the coefficient multiplication / subtraction processing unit 17 subtracts the multiplication value data of the coefficient and the amplitude value data of the difference signal from the amplitude value data of the Rch audio signal. That is, the coefficient multiplication / subtraction processing unit 17 calculates, for each frequency, subtraction value data obtained by subtracting the multiplication value data from the amplitude value data of the Rch audio signal. The coefficient multiplication / subtraction processing unit 17 calculates | YR (t, k * f0) | -Ck * | Ys (t, k * f0) |. | YR (t, k * f0) | -Ck * | Ys (t, k * f0) | is calculated for each k.

  A specific configuration of the coefficient multiplication / subtraction processing unit 16 will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration example of the coefficient multiplication / subtraction processing unit 16. The coefficient multiplication / subtraction processing unit 16 includes a pair of a constant multiplier 41 and a subtracter 42. The number of pairs of the constant multiplier 41 and the subtracter 42 corresponds to the number of data included in the low frequency range to be extracted, and is 32 (K = 0 to 31) in the above example.

  A predetermined coefficient Ck is set for each of the constant multipliers 41. Then, the constant multiplier 41 obtains the product of the amplitude value data of the difference signal and the coefficient. Then, the subtractor 42 obtains a difference between the amplitude value data of the Lch audio signal and the multiplication value data for each frequency, and sets it as a subtraction value. As a result, the coefficient multiplication / subtraction processing unit 16 generates a subtraction signal composed of the subtraction value.

If the subtraction value | YL (t, k × f0) | −Ck × | Ys (t, k × f0) | is negative, it is replaced with 0. Instead of the process of rewriting to 0 when it becomes negative, it may be replaced with the predetermined positive constant when it becomes equal to or less than a predetermined positive constant. It has the effect of making noise called musical noise inconspicuous.
The coefficient multiplication / subtraction processing unit 17 has the same configuration as the coefficient multiplication / subtraction processing unit 16 and performs the same processing. The same processing is performed when the subtraction value calculated by the coefficient multiplication / subtraction processing unit 17 becomes negative.

  A value generated from the amplitude value data of the corresponding frequency and the amplitude value data of the surrounding frequencies may be used as the difference signal data. For example, instead of | Ys (t, k × f0) |, max (| Ys (t, (k-1) × f0) |, | Ys (t, k × f0) |, | Ys (t, ( k + 1) × f0) |) The maximum value of the amplitude values of the three frequencies before and after is obtained. In this way, the maximum value of the amplitude value data of the corresponding frequency and the amplitude value data of the surrounding frequencies may be used. This is because a noise component may appear as an amplitude value of an adjacent frequency. As described above, the coefficient multiplication / subtraction processing unit 16 performs the subtraction process based on the amplitude value data of the difference signal and the amplitude value data of the Lch audio signal. That is, the coefficient multiplication / subtraction processing unit 16 generates a subtraction signal based on the difference signal and the Lch audio signal. Of course, an average value or an intermediate value of the amplitude value data may be used instead of the maximum value of the amplitude value data. By so doing, wind noise components can be more effectively reduced.

  Note that the amplitude value data in the low frequency range of the difference signal used for subtraction may be a value smoothed using the amplitude value data of the past frame for each K. For example, for amplitude value data of K = 5, the amplitude value data for the past three frames is represented by (| Ys (t−3,5 × f0) | + 2 × | Ys (t−2,5 × f0) | +3 X | Ys (t-1,5 × f0) | + 4 × | Ys (t, 5 × f0) |) / 10 The same applies to other Ks. The effect of abrupt time change of the difference signal (which is a wind noise component, as will be described later) is alleviated and noise called musical noise is made inconspicuous. The coefficient multiplication / subtraction processing unit 17 can perform the same processing as the coefficient multiplication / subtraction processing unit 16.

The subtraction signal from the coefficient multiplication / subtraction processing unit 16 is input to the IFFT processing unit 21 shown in FIG. Amplitude value data in the high frequency range of the Lch audio signal | YL (t, k × f0) | (K = 32 to 127) and the phase value of the Lch audio signal φyL (k × f0) (K = 0 to 127) Is input to the IFFT processing unit 21. The IFFT processing unit 21 performs IFFT (Inverse FFT) processing using these amplitude information and phase information. As a result, the IFFT processing unit 21 serves as an inverse transform unit that inversely transforms the subtraction signal in the frequency domain into a time signal in the time domain.
Similarly, the subtraction signal from the coefficient multiplication / subtraction processing unit 17 is input to the IFFT processing unit 23. Rch audio signal amplitude value data | YR (t, k × f0) | (K = 32 to 127) and Rch audio signal phase value φyR (k × f0) (K = 0 to 127) Is input to the IFFT processing unit 23. The IFFT processing unit 23 performs IFFT (Inverse FFT) processing using these amplitude information and phase information. As a result, the IFFT processing unit 23 becomes an inverse transform unit that inversely transforms the subtraction signal in the frequency domain into a time signal in the time domain.

The output of the IFFT processing unit 21 is input to the waveform synthesis unit 22. The waveform synthesizer 22 performs an inverse wind process and a waveform synthesize process, and outputs an audio signal ynrL (i). This audio signal is obtained by removing wind noise components from the audio signal of the left channel (Lch).
The output of the IFFT processing unit 23 is input to the waveform synthesis unit 24. The waveform synthesizer 24 performs an inverse wind process and a waveform synthesize process, and outputs an audio signal ynrR (i). This audio signal is obtained by removing wind noise components from the right channel (Rch) audio signal.

  In the case of two microphones arranged for Lch and Rch in the casing of the electronic device, the distance between the microphones is short. For this reason, there is almost no difference between the two channels in the recorded audio signal, particularly for low frequencies. As the frequency becomes higher, there is a difference in the original stereo component depending on the position of the sound source and the two microphones. On the other hand, wind noise is mainly low frequency components of 1KHz or less. Wind noise is generated without correlation between the Lch and Rch microphones.

  Therefore, the low frequency component resulting from the FFT of the difference signal between Lch and Rch is considered to be due to wind noise. The amplitude value data of the frequency corresponding to the difference signal is subtracted from the amplitude value data corresponding to the low frequency of the audio signal of Lch or Rch. By doing so, it is possible to reduce the wind noise component from the audio signal.

  In the above description, the Lch and Rch microphones of the electronic device have been described as examples. However, the present invention can be used for various electric devices. For example, the noise removal apparatus 100 can be mounted on an electronic device having a recording function or an electric device that reproduces sound with a speaker. Further, the microphone for detecting the audio signal may not be a stereo microphone having Lch and Rch, but may be a plurality of microphones arranged close to each other. The number of microphones is not limited to two as long as there are two or more microphones.

In the above description, the amplitude values | YR (t, k × f0) |, | YL (t, k × f0) |, | Ys (t, k × f0) | are used for the audio signal and the difference signal. However, the power value | YR (t, k × f0) | ² , | YL (t, k × f0) | ² , | Ys (t, k × f0) | ² A signal may be generated. For example, the coefficient multiplication / subtraction processing unit 16 subtracts a value obtained by multiplying the power value of the difference signal by the coefficient from the power value of the audio signal. The subtracted result is raised to the power of 1/2. In this case, the subtraction signal output from the coefficient multiplication / subtraction processing unit 16 is ((| YL (t, k × f0) |) ² −Ck × (| Ys (t, k × f0) |) ² ) ^0.5 It can be.
Alternatively, the power value subtraction result is divided by the square of the audio signal. The result obtained by multiplying the division value by the amplitude value data of the audio signal may be used as the subtraction signal. In this case, the subtraction signal output from the coefficient multiplication / subtraction processing unit 16 is | YL (t, k × f0) | × ((| YL (t, k × f0) |) ² −Ck × (| Ys ( t, k × f0) |) ² ) / (| YL (t, k × f0) |) ² . The coefficient multiplication / subtraction processing unit 17 performs the same processing as the coefficient multiplication / subtraction processing unit 16.

  By using the power value in this way, it is possible to obtain the same effect as when the amplitude value data of the audio signal and the difference signal is used. Thus, a subtraction signal can be generated using the amplitude spectrum or power spectrum of Fourier transform.

Embodiment 2. FIG.
The noise removal apparatus according to the present embodiment will be described with reference to FIG. FIG. 3 is a diagram illustrating a configuration of the noise removing device. As in the first embodiment, the noise removal apparatus according to the present embodiment performs noise removal processing on the audio signal from the microphone arranged in the casing of the electronic device. In other words, stereo right and left channel (Rch, Lch) audio signals are A / D converted and input from the input terminals 101 and 102 from two microphones.

  The noise removal apparatus 100 according to the present embodiment includes a composition ratio calculation unit 18, a delay unit 31, a delay unit 32, a switching unit 33, and a switching unit 34 in addition to the configuration of the first embodiment. Note that the basic configuration of the noise removal apparatus 100 is the same as that of the first embodiment, and thus description thereof will be omitted as appropriate.

  The amplitude value data | Ys (t, k × f0) | (K = 0 to 31) of the low frequency range of the difference signal is input to the synthesis ratio calculation unit 18. For example, the composition ratio calculation unit 18 obtains a weighted average value obtained by increasing the weight of the low frequency of the amplitude value data | Ys (t, k × f0) | (K = 0 to 31). The composition ratio calculation unit 18 compares the weighted average value with the threshold value 1 and the threshold value 2. As shown in FIG. 4, when the weighted average value or the like is smaller than the predetermined threshold value 1, the synthesis ratio calculation unit 18 outputs the synthesis ratio data Q = 0. When the weighted average value or the like is larger than the predetermined threshold 2, the synthesis ratio calculation unit 18 outputs Q = 1. When the weighted average value is between the threshold value 1 and the threshold value 2, the synthesis ratio calculation unit 18 outputs a value larger than 0 and smaller than 1 according to the value as the synthesis ratio data Q. Here, between the threshold value 1 and the threshold value 2, the composite ratio data Q increases linearly with respect to the weighted average value.

  Instead of the weighted average value, the sum of the amplitude value data | Ys (t, k × f0) | (K = 0 to 31) or the sum of squares may be used. That is, the sum, square sum, or weighted average value of | Ys (t, k × f0) | (K = 0 to 31) can be used as a comparison value to be compared with the threshold value 1 and the threshold value 2. This comparison value indicates the noise component in the low frequency range included in the audio signal.

  Further, the synthesis ratio data Q calculated as described above may be smoothed in the time axis direction. For example, the result of the past n−1 frames may be held, and the average value of n frames may be calculated and output together with the current result. Alternatively, a value obtained by applying LPF in the time axis direction may be output. As shown in FIG. 3, the output of the composition ratio calculation unit 18 is input to the switching units 33 and 34 for Lch and Rch.

  An Lch audio signal is input to the Lch switching unit 33 via the delay unit 31. The delay unit 31 compensates for the time required for the STFT process, IFFT process, and waveform synthesis process. That is, the delay unit 31 delays the Lch audio signal by a time corresponding to the processing of the STFT processing unit 13, the processing of the coefficient multiplication / subtraction processing unit 16, the processing of the IFFT processing unit 21, and the processing of the waveform synthesis unit 22. For example, when the STFT process, IFFT process, and waveform synthesis process take m samples in the sampling period of A / D conversion, the delay unit 31 also delays m samples and outputs it as ydL (i). The switching unit 33 also receives the output signal ynrL (i) of the waveform synthesis unit 22 and the synthesis ratio data Q that is the output of the synthesis ratio calculation unit 18. The switching unit 33 calculates (1−Q) × ydL (i) + Q × ynrL (i) from ydL (i), ynrL (i) and the composition ratio data Q and outputs the result.

  The configuration of the switching unit 33 is as shown in FIG. The switching unit 33 includes a variable number multiplier 71, a variable number multiplier 72, and an adder 73. The combination ratio data Q is input to the variable multipliers 71 and 72. The variable multiplier 71 multiplies ynrL (i) by Q. The variable number multiplier 72 multiplies ydL (i) by (1−Q). Then, the adder 73 obtains the sum of the outputs of the variable number multiplier 71 and the variable number multiplier 72. Accordingly, (1−Q) × ydL (i) + Q × ynrL (i) is calculated.

  When the weighted average value in the low frequency range is smaller than the threshold value 1, Q = 0. That is, since the noise component is small, the output signal is ydL (i), and the audio signal input to the input terminal 101 is output as it is. On the other hand, when the weighted average value in the low frequency range is larger than the threshold value 2, Q = 1. That is, since the noise component is large, the output signal is ynrL (i), and the signal synthesized by the waveform synthesizer 22 is output to the output terminal 103. As described above, the switching unit 33 switches and outputs the audio signal ydL (i) input to the input terminal 101 or the noise-reduced signal ynrL (i). When the weighted average value is greater than or equal to threshold 1 and less than or equal to threshold 2, the output signal is (1−Q) × ydL (i) + Q × ynrL (i), and noise is removed from the audio signal input to the input terminal 101. Are synthesized at a predetermined ratio.

  The same processing is performed on the Rch audio signal. That is, the Rch audio signal is input to the switching unit 34 via the delay unit 32. The delay unit 32 performs the same processing as the delay unit 31. The switching unit 34 has the same configuration as the switching unit 33 and performs the same processing as the switching unit 33. Therefore, the switching unit 34 outputs Rch (1−Q) × ydR (i) + Q × ynrR (i). The output signals from the switching units 33 and 34 are respectively output from the output terminals 103 and 104, encoded by an encoder (not shown), and recorded on a recording medium. Or after D / A conversion, it outputs to a speaker etc.

  The synthesis ratio data Q may be obtained from data other than the amplitude value data in the low frequency range of the difference signal. For example, the synthesis ratio data Q may be obtained from data of power values in the low frequency range of the difference signal. Further, it may be calculated using a wind pressure sensor described in Japanese Patent Laid-Open No. 5-328480. Alternatively, the difference signal may be passed through the BPF and the absolute value of the output may be calculated from the peak detected data. Here, the pass band of the BPF is set to 100 Hz to 1 KHz, which is a main component of wind noise. Thus, the noise component contained in the audio signal is measured using the amplitude value data of the difference signal or other means. Then, the synthesis ratio calculation unit 18 may calculate the synthesis ratio data Q according to the measurement result of the noise component. By doing so, the switching unit 33 can appropriately perform switching.

  The synthesis ratio calculation unit 18 calculates the synthesis ratio data Q using the amplitude value data for the low frequency as a result of FFT of the difference signal. The switching units 33 and 34 use the synthesis ratio data Q to synthesize the original audio signal and the audio signal with reduced wind noise. By doing this, the original audio signal is output when there is no wind noise, so that the audio signal has a stereo component. When the wind noise is large, an audio signal with reduced wind noise is output, so that an audio signal with little influence of wind noise can be output even when there is wind.

  Here, when the synthesis ratio data Q calculated by the synthesis ratio calculation unit 18 is 0 in a certain frame, a part or all of the arithmetic processing of the coefficient multiplication / subtraction processing unit, IFFT processing unit, and waveform synthesis unit is performed. You may make it not perform. For example, the IFFT processing unit 21 is configured not to perform the inverse Fourier transform according to the noise measurement result. By doing so, the processing can be simplified.

  For example, when the noise removing apparatus according to the present embodiment is used in, for example, an audio recording apparatus, the wind is not always blowing, and sometimes it is not blowing for a long period of time. When these processing units are realized by a circuit, power consumption can be reduced by not operating the circuit. When these processing units are realized by software such as a CPU or a DSP, a routine for executing these processes is skipped. As a result, when the wind noise is small, the processing can be omitted and the power consumption can be reduced.

  Specifically, when the processing in the noise removal apparatus 100 is realized by software such as a CPU or DSP, first, STFT processing is performed on the difference signal. When the magnitude values of the obtained amplitude value data | Ys (t, k × f0) | (K = 0 to 31) are all smaller than a predetermined threshold, STFT processing, coefficient multiplication / subtraction processing of the audio signal, Skip all or part of IFFT processing and waveform synthesis processing. In this way, by comparing the amplitude value data with the threshold value, all or part of the STFT processing, coefficient multiplication / subtraction processing, IFFT processing, and waveform synthesis processing of the audio signal may be omitted. In this way, power consumption can be reduced.

  Note that the noise removal apparatus 100 may use power value data instead of amplitude value data for noise measurement. The threshold value may be determined for each K or the same threshold value. Further, the noise removal apparatus 100 may compare the sum value obtained by weighting the amplitude value data or the power value data and taking the sum and the threshold value. Then, it may be determined whether the wind noise is small according to the comparison result, and the processing may be omitted according to the determination result. When at least a part of the processing is skipped, the noise removal apparatus 100 outputs the audio signal with a predetermined time delay as it is. Note that the predetermined time can be set to the same sanding period as in the case of performing STFT processing, coefficient multiplication / subtraction processing, IFFT processing, and waveform synthesis processing of an audio signal.

Embodiment 3 FIG.
A noise removal apparatus 100 according to the present embodiment will be described with reference to FIG. FIG. 6 is a diagram illustrating a configuration of the noise removal apparatus 100. The noise removal apparatus 100 according to the present embodiment performs noise removal processing using three microphones. Here, in addition to the Lch and Rch audio signals, an Mch audio signal is input to the noise removal apparatus 100. Therefore, the noise removal apparatus 100 includes an Mch input terminal 105 and an output terminal 106.

  The subtractor 11a generates a difference signal between the Lch audio signal and the Mch audio signal, and the subtractor 11b generates a difference signal between the Mch audio signal and the Rch audio signal. Note that. Since the processing for the difference signal between Lch and Mch and the difference signal between Mch and Rch are basically the same, the following description will be focused on the processing of the difference signal between Lch and Mch.

  The difference signal from the subtractor 11a is multiplied by a constant by the constant multiplier 12a and input to the STFT processing unit 14a. The difference signal from the subtractor 11b is multiplied by a constant by a constant multiplier 12b and input to the STFT processing unit 14b. The STFT processing unit 14a performs the same processing on the difference signal as the STFT processing unit 14 of the first embodiment. The Lch audio signal is input to the STFT processing unit 13a. The STFT processing unit 13a performs the same processing as the STFT processing unit 13 of the first embodiment on the Lch audio signal. The Mch audio signal is input to the STFT processing unit 15. The STFT processing unit 15 performs the same processing on the Mch audio signal as the STFT processing unit 15 of the first embodiment. The Rch audio signal is input to the STFT processing unit 13b. The STFT processing unit 13b performs the same processing on the Rch audio signal as the STFT processing unit 13 of the first embodiment.

  Also, the coefficient multiplication / subtraction processing unit 16 in the present embodiment performs the same processing as the coefficient multiplication / subtraction processing unit 16 in the first embodiment. Therefore, the coefficient multiplication / subtraction processing unit 16 calculates a subtraction signal based on the difference signal and the Lch audio signal, and outputs the subtraction signal to the IFFT processing unit 21. The coefficient multiplication / subtraction processing unit 25 in the present embodiment performs the same processing as the coefficient multiplication / subtraction processing unit 17 in the first embodiment. Therefore, the coefficient multiplication / subtraction processing unit 25 calculates a subtraction signal based on the difference signal and the Mch audio signal, and outputs the subtraction signal to the IFFT processing unit 26.

  The IFFT processing unit 21 and the waveform synthesis unit 22 of the present embodiment correspond to the IFFT processing unit 21 and the waveform synthesis unit 22 of the first embodiment. The IFFT processing unit 21 generates a time signal in the time domain based on the subtraction signal and the Lch audio signal. Based on the time signal in the time domain, the waveform synthesizer 22 outputs the Lch audio signal from which noise has been removed to the output terminal 103. In this way, an Lch audio signal from which the noise component has been removed is output from the output terminal 103.

  The IFFT processing unit 26 and the waveform synthesis unit 27 perform the same processing as the IFFT processing unit 21 and the waveform synthesis unit 22. The IFFT processing unit 26 generates a time signal in the time domain based on the subtraction signal and the Mch audio signal. Based on the time signal in the time domain, the waveform synthesizer 27 outputs the noise-removed Mch audio signal to the output terminal 106. In this way, the Mch audio signal from which the noise component has been removed is output from the output terminal 106.

  The same processing is performed for the Rch audio signal. That is, the STFT processing unit 14b corresponds to the STFT processing unit 14a, and the STFT processing unit 13b corresponds to the STFT processing unit 13a. The coefficient multiplication / subtraction processing unit 17 corresponds to the coefficient multiplication / subtraction processing unit 16, and the IFFT processing unit 23 corresponds to the IFFT processing unit 21. The waveform synthesis unit 24 corresponds to the waveform synthesis unit 22.

  The difference signal between Rch and Mch is input to the STFT processing unit 14b, and the Rch audio signal is input to the STFT processing unit 13b. The STFT processing unit 14b performs the same processing as the STFT processing unit 14a on the difference signal. The STFT processing unit 13b performs the same processing as the STFT processing unit 13a on the Rch signal. The coefficient multiplication / subtraction processing unit 17 calculates a subtraction signal based on the difference signal and the Rch audio signal, and outputs the subtraction signal to the IFFT processing unit 23. Further, the STFT processing unit 13 b outputs the amplitude component and phase component in the high frequency region of the Rch audio signal to the IFFT processing unit 23. The IFFT processing unit 23 generates a time signal in the time domain based on the amplitude information and the phase information. The waveform synthesis unit 24 outputs the Rch audio signal from which noise has been removed to the output terminal 104.

  Since Mch is processed based on the Lch and Mch audio signals, the Mch noise removal process is performed without using the Rch signal. The three microphones Lch, Mch, and Rch are arranged close to each other. Considering the case where a microphone is attached to an electronic device or the like, Lch and Rch microphones may be arranged on the left and right sides of the housing, respectively, and Mch may be arranged at a position near the middle of Lch and Rch. This is preferable because the distance between the Lch and Mch microphones and the distance between the Rch and Mch microphones are close.

  By doing so, it becomes possible to process the audio signals of the three microphones. Of course, the same processing may be performed for four or more cases. Also in the configuration of the third embodiment, the switching unit 33, the switching unit 34, and the like described in the second embodiment can be added.

Embodiment 4 FIG.
The noise removal apparatus according to this embodiment will be described with reference to FIG. FIG. 7 is a diagram illustrating an overall configuration of the noise removal apparatus 100. In the present embodiment, a time domain speech signal is converted into a frequency domain signal using discrete cosine transform instead of discrete Fourier transform. Therefore, the STFT processing units 13 to 15 are replaced with window function processing + DCT processing units 63 to 65. Further, the IFFT processing units 21 and 23 are replaced with inverse DCT processing units 81 and 83. Note that the description of the same processing as in the first embodiment will be omitted as appropriate.

  As in the first embodiment, the subtractor 11 calculates a difference signal between Lch and Rch. These difference signals are multiplied by ½ by the constant multiplier 12 and then input to the window function processing + DCT processing unit 64. The Lch audio signal is input to the window function processing + DCT processing unit 63, and the Rch audio signal is input to the window function processing + DCT processing unit 65. Window function processing + DCT processing units 63 to 65 execute window function processing and DCT processing.

  In the window function process, a process of dividing the input signal into frames having a predetermined length while shifting the input signal at predetermined time intervals is performed. Then, a process of multiplying each frame divided by a predetermined time window is performed. This frame has an overlap of a predetermined number of samples. Further, a discrete cosine transform DCT (Discrete Cosine Transform) process is performed on the signal subjected to the time window to convert it into a frequency domain signal. The window function processing + DCT processing units 63 to 65 serve as conversion units that convert the audio signal and the difference signal in the time domain into frames and then convert them into an audio signal and a difference signal in the frequency domain.

  For a frame at time t, if it is a 256th order DCT, the 256 frequency domain data are F (t, k) (K = 0 to 255). The difference signal is expressed as Fs (t, k), the Lch audio signal is expressed as FL (t, k), and the Rch audio signal is expressed as FR (t, k). Here, data having a small k is data in a low frequency range.

  The frequency domain data in the low frequency range of the difference signal and the Lch audio signal is input to the coefficient multiplication / subtraction processing unit 16. For example, 32 low frequency data, Fs (t, k), FL (t, k) (K = 0 to 31). The configuration of the coefficient multiplication / subtraction processing unit 16 is shown in FIG. The coefficient multiplication / subtraction processing unit 16 includes an ABS unit 43, a constant multiplier 41, a subtractor 42, and an SGN unit 44.

  The ABS unit 43 of the coefficient multiplication / subtraction processing unit 16 converts each of Fs (t, k) and FL (t, k) into an absolute value to obtain | Fs (t, k) |, | FL (t, k) | And Here, the coefficient multiplication / subtraction processing unit 16 stores the sign of each value of FL (t, k).

  The constant multiplier 41 multiplies each data of | Fs (t, k) | (K = 0 to 31) by a predetermined coefficient Ck. The coefficient Ck can be the same value as in the first embodiment. Thereby, the effect similar to Embodiment 1 can be acquired.

  Next, the subtractor 42 subtracts the product of the absolute value and the coefficient from the absolute value of the frequency domain data of the audio signal. If the subtraction result, that is, | FL (t, k) | −Ck × | Fs (t, k) | Instead of the process of rewriting to 0 when it becomes negative, it may be replaced with the predetermined positive constant when it becomes equal to or less than a predetermined positive constant. It has the effect of making noise called musical noise inconspicuous.

  As in the first embodiment, the absolute value of the corresponding frequency data and the maximum value of the absolute values of the peripheral frequency data may be used. In this case, instead of | Fs (t, k) |, max (| Fs (t, k-1) |, | Fs (t, k) |, | Fs (t, k + 1) |) is used. It will be. Alternatively, an average value or an intermediate value of these values may be used. As in the first embodiment, the frequency domain data in the low frequency domain of the difference signal used for subtraction may be a value smoothed using data of past frames for each K. In this case, for example, for the data of K = 5, the data for the past three frames is (| Fs (t−3,5) | + 2 × | Fs (t−2,5) | + 3 × | Fs (t -3,5) | + 4 × | Fs (t, 5) |) / 10. Thereby, the effect similar to Embodiment 1 can be acquired.

  The SGN unit 44 attaches a positive or negative sign of each value stored when making the absolute value to the subtraction result. That is, the sign of the frequency domain data of the audio signal does not change, and the absolute value thereof decreases by an amount related to the absolute value of the frequency domain data of the difference signal. As a result, if such a value is obtained, it is not always necessary to make an absolute value at the position of the ABS unit 43 or to add a sign at the position of the SGN unit 44. You may make it calculate.

Further, instead of converting the frequency domain data into absolute values, the ABS unit 43 may convert them into square values. In this case, the output of the subtracter 42 is (| FL (t, k) |) ² −Ck × (| Fs (t, k) |) ² . Then, the SGN unit 44 divides the output data of the subtractor 42 by a power of 2 and adds a sign. The output of the SGN unit 44 is a value with a sign (((| FL (t, k) |) ² −Ck × (| Fs (t, k) |) ² ) ^0.5 ). Alternatively, a division value obtained by dividing the output data of the subtractor 42 by the square of the audio signal may be obtained, and the division value may be multiplied by the frequency domain data of the audio signal. In this case, the subtraction signal output from the coefficient multiplication / subtraction processing unit 16 is FL (t, k) × ((| FL (t, k) |) ² −Ck × (| Fs (t, k × f0) | ) ² ) / (| FL (t, k) |) ² Note that the numerator of the division part does not become negative as described above. Accordingly, since the sign of the frequency domain data of the audio signal does not change, it is not necessary to add a sign at the position of the SGN unit 44. In this way, the subtraction signal may be calculated by subtracting the square value of the frequency domain data.

  The frequency domain data with positive and negative signs added to the subtraction result and the high frequency domain frequency domain data FL (t, k) (K = 32 to 127) of the audio signal are input to the inverse DCT processing unit 81. The inverse DCT processing unit 81 performs inverse DCT processing using these frequency domain data. As a result, the frequency domain signal is inversely transformed into a time domain signal. The output of the inverse DCT processing unit 81 is input to the waveform synthesis unit 22. The inverse DCT processing unit 81 is an inverse transform unit that inversely transforms the subtraction signal in the frequency domain into a time signal in the time domain. Similarly to the first embodiment, the waveform synthesis unit 22 performs waveform synthesis processing and outputs an audio signal ynrL (i). This audio signal is obtained by removing wind noise components from the audio signal of the left channel (Lch).

  The Rch audio signal is also processed in the same manner as the Lch. As a result, as in the first embodiment, it is possible to obtain Lch and Rch audio signals from which noise has been removed. Instead of the DCT transform, modified discrete cosine transform (MDCT), Hartley transform, or discrete sine transform may be used. In particular, by using MDCT and an appropriate window function, the number of frequency domain samples to be calculated can be reduced due to the overlap of frames by setting the frame overlap to ½ of the total frame length. Also. Although it is preferable to convert from the time domain to the frequency domain signal using these orthogonal transforms, the orthogonal transform may not be used. Even in the non-orthogonal transformation, it is only necessary to perform time domain and frequency domain transformation similar to the orthogonal transformation.

Embodiment 5 FIG.
The configuration of the noise removal apparatus according to the present embodiment will be described with reference to FIG. FIG. 9 is a diagram illustrating a configuration of the noise removing device. Note that the basic configuration of the noise removal apparatus 100 is the same as that of the first embodiment, the second embodiment, or the fourth embodiment, and thus the description thereof is omitted. Specifically, in the present embodiment, the second embodiment and the fourth embodiment are combined. That is, in the second embodiment, the STFT processing units 13 to 15 and the IFFT processing units 21 and 23 are replaced with window function processing + DCT processing units 63 to 65 and inverse DCT processing units 81 and 83 to perform DCT conversion and inverse DCT. Using conversion. In other words, in the fourth embodiment, the composition ratio calculation unit 18, the delay units 31 and 32, and the switching units 33 and 34 are added.

  In such a configuration, the synthesis ratio can be calculated according to the noise component included in the audio signal. Therefore, in addition to the effects of the first and fourth embodiments, the effects shown in the second embodiment can be obtained. By performing such control, the noise component can be deleted more effectively. Furthermore, when the noise component is low and the audio signal is output as it is to the output terminals 103 and 104, a part of the processing can be omitted. Therefore, processing can be simplified and power consumption can be reduced.

Embodiment 6 FIG.
The configuration of the noise removal apparatus according to this embodiment will be described with reference to FIG. FIG. 10 is a diagram illustrating a configuration of the noise removal device. Note that the basic configuration of the noise removal apparatus 100 is the same as that of the first embodiment, the third embodiment, or the fourth embodiment, and thus the description thereof is omitted. Specifically, the present embodiment has a configuration in which the third embodiment and the fourth embodiment are combined. That is, in the third embodiment, the STFT processing units 13a, 13b, 14a, 14b, and 15 and the IFFT processing units 21, 23, and 26 are changed to window function processing + DCT processing units 63a, 63b, 64a, 64b, 65, and inverse DCT. Instead of the processing units 81, 83, 86, DCT transformation and inverse DCT transformation are used. In other words, the fourth embodiment corresponds to the case where the microphones are set to three channels of Lch, Rch, and Mch.

  With such a configuration, the same effect as in the first, third, and fourth embodiments can be obtained. That is, it is possible to effectively remove noise for each of the three or more microphones. Of course, the present embodiment and the second embodiment may be combined. That is, in a configuration using three or more microphones, a delay unit and a switching unit may be added. By doing in this way, the effect similar to Embodiment 2 can be acquired.

Other embodiments.
In the first to sixth embodiments, the number of samples in the frame is described as being fixed. However, the number of samples in the frame may be changed every moment. By changing according to the state of the input audio signal, the sound quality after removing wind noise can be improved. Alternatively, the number of samples may be changed depending on the detection state of wind noise (the distribution state of the low frequency amplitude of the difference signal). Increase the number of samples when the wind noise is relatively small and the frequency distribution stays in the low range, and increase the number of samples when the wind noise is relatively large and the frequency distribution extends to the high range. Make it smaller. By performing such control, wind noise can be removed more effectively.

  In the above embodiment, the number of data in the low frequency range to be subjected to coefficient multiplication / subtraction processing is fixed (in the embodiment, 32), and the degree of decrease as Ck becomes higher is fixed. However, the frequency distribution of wind noise is composed of only a relatively low frequency component when the wind is gentle. However, as the wind becomes stronger, the frequency distribution becomes higher. On the other hand, depending on the distance between the two microphones, the original stereo component exists above a certain frequency. Therefore, when the air volume is relatively small, the number of data in the low frequency range for coefficient multiplication / subtraction processing is reduced. Alternatively, the attenuation degree of Ck is made steep. On the other hand, when the air volume is relatively large, the number of data in the low frequency range to be multiplied / subtracted by the coefficient is increased. Alternatively, the degree of attenuation of Ck is moderated. In this manner, the low frequency component range in which the coefficient multiplication / subtraction processing unit 16 performs the subtraction process can be adjusted according to the air volume. In this way, the number of data in the low frequency range is changed according to the air volume. In other words, the range of the low frequency component in which the subtraction process is performed is adjusted. By doing so, wind noise can be effectively reduced.

  The detection of the air volume is the weighted average of the sum of the amplitude value data | Ys (t, k × f0) | (K = 0 to 20) or the sum of squares, or the weight of the low frequency data. Perform by value. Alternatively, a wind pressure sensor may be used as in the first embodiment. Similarly to the first embodiment, the difference signal may be passed through the BPF and the absolute value of the output may be calculated from the data obtained by peak detection. Here, the pass band of the BPF is set to 100 Hz to 1 KHz, which is a main component of wind noise.

  In the coefficient multiplication / subtraction process, each data of | Ys (t, k × f0) | (K = 0 to 31) is multiplied by a predetermined coefficient Ck, and the multiplication result is obtained as one ch signal. The amplitude value data (for example, Lch data | YL (t, k × f0) |) is subtracted from the amplitude value data. For example, when the value of | Ys (t, k × f0) | is small, it may be small and when large, it may be converted to a large value, and the amplitude value data of one ch signal may be divided by that value.

  Note that the Lch and Rch microphones do not have to be provided, and there may be a plurality of channel microphones. For example, it may be two or more microphones arranged close to each other. Further, noise may be removed from only one of Rch and Lch. Further, noise may be removed from a sound signal from a microphone array in which microphones are arranged in an array.

  The processing for removing noise described above may be executed by a computer program. The above-described program can be stored and supplied to a computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)) are included. The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

  In addition to the case where the function of the above-described embodiment is realized by the computer executing the program that realizes the function of the above-described embodiment, this program is not limited to the OS ( The case where the functions of the above-described embodiment are realized in cooperation with the Operating System) or application software is also included in the embodiment of the present invention.

DESCRIPTION OF SYMBOLS 100 Noise removal apparatus 11 Subtractor 13-15 STFT process part 16, 17 Coefficient multiplication / subtraction process part 18 Composition ratio calculation part 63-65 Window function process + DCT process part 21, 23 IFFT process part 22, 24 Waveform composition part 31, 32 Delay part 33, 34 Switching part 41 Constant multiplier 42 Subtractor

Claims (12)

A signal calculation unit that calculates a difference signal between an audio signal of one channel among a plurality of audio signals input from a plurality of audio channels and an audio signal of another one channel;
A conversion unit for dividing the audio signal and the difference signal in the time domain into frames, respectively, and then converting the audio signal and the difference signal in the frequency domain;
A subtraction processing unit that generates a subtraction signal in the frequency domain based on the difference signal in the frequency domain and the audio signal in the frequency domain;
A noise removing device comprising: an inverse transform unit that inversely transforms the subtraction signal in the frequency domain into a time signal in the time domain.   The subtraction processing unit subtracts a value obtained by multiplying a value corresponding to the magnitude of the data value of the difference signal in the frequency domain by a coefficient from a value corresponding to the magnitude of the data value of the audio signal in the frequency domain. The noise removal apparatus according to claim 1, wherein the subtraction signal is calculated.   The noise removal apparatus according to claim 1, wherein the subtraction processing unit generates the subtraction signal by performing a subtraction process only on a low frequency component of the audio signal and the difference signal.   The noise removal device according to claim 3, wherein a range of the low frequency component in which the subtraction process is performed is changed according to an air volume.   5. The switching unit according to claim 1, further comprising: a switching unit that switches and outputs the time signal inverted by the inverse conversion unit and the audio signal input from the audio channel. Noise removal device.   The noise removal device according to claim 5, wherein when the switching unit outputs the audio signal, the inverse transformation process by the inverse transformation unit is omitted.   Depending on the magnitude of the data value of the difference signal in the low frequency range, the conversion unit converts the audio signal into a frequency signal, the subtraction unit performs subtraction, and the inverse conversion unit performs inverse processing. The noise removal device according to claim 1, wherein at least one of the conversion processes is omitted.   The noise removal according to any one of claims 1 to 7, wherein a value of the subtraction signal at an arbitrary frequency is calculated based on a value of the difference signal having a frequency including before and after the arbitrary frequency. apparatus. The transform unit performs Fourier transform on the audio signal and the difference signal in the time domain, thereby converting the audio signal and the difference signal in the frequency domain,
The noise removal apparatus according to claim 1, wherein the subtraction signal is calculated using an amplitude spectrum or a power spectrum of the Fourier transform. The conversion unit may calculate the audio signal and the difference signal in the frequency domain with a positive / negative sign,
The subtraction processing unit uses the absolute value of the audio signal and the difference signal, or the square value of the audio signal and the difference signal, and attaches the sign of the audio signal in the frequency domain to the resulting signal, The noise removal device according to claim 1, wherein the subtraction signal is calculated. Calculating a difference signal between an audio signal of one channel and an audio signal of another channel among a plurality of audio signals input from a plurality of audio channels;
The audio signal and the difference signal in the time domain are each divided into frames, and then converted into an audio signal and a difference signal in the frequency domain;
Generating a subtraction signal in the frequency domain based on the difference signal in the frequency domain and an audio signal in the frequency domain;
A step of inversely transforming the subtracted signal in the frequency domain into a signal in the time domain. A program for causing a computer to execute a noise removal method for removing noise,
The noise removal method comprises:
Calculating a difference signal between an audio signal of one channel and an audio signal of another channel among a plurality of audio signals input from a plurality of audio channels;
The audio signal and the difference signal in the time domain are each divided into frames, and then converted into an audio signal and a difference signal in the frequency domain;
Generating a subtraction signal in the frequency domain based on the difference signal in the frequency domain and an audio signal in the frequency domain;
Back-converting the subtraction signal in the frequency domain into a signal in the time domain.

Images