JP6725014B1 - Sound collecting device, sound collecting program, and sound collecting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress sound quality deterioration during area sound collection processing. The present invention relates to a sound collecting device. Then, the sound collecting device of the present invention extracts a non-target area sound by a means for acquiring a target direction signal based on the beamformer outputs of a plurality of microphone arrays and a spectrum subtraction process for the acquired target direction signal. A means for extracting the target area sound by spectrally subtracting the non-target area sound from the direction signal, and a peak frequency from the non-target area sound component dominant signal in which the sound component whose sound source is the non-target area based on the captured signal is dominant. When a suppression filter that detects and suppresses the suppression frequency component based on the peak frequency of the non-target area sound component dominant signal is formed, the suppression filter is applied to the mixing signal to mix with the target area sound and the mixed signal is obtained. And a means for outputting the mixed signal as an area sound collection result of the target area. [Selection diagram]

Description

The present invention relates to a sound collecting device, a sound collecting program, and a sound collecting method, and can be applied to, for example, area sound collecting processing that emphasizes sound in a specific area and suppresses sound in other areas.

BACKGROUND ART Conventionally, there is a beam former (Beam Former; hereinafter referred to as “BF”) using a microphone array as a technique for separating and collecting only a sound in a specific direction in an environment where a plurality of sound sources exist. BF is a technique for forming directivity by utilizing the time difference between signals that reach each microphone (see Non-Patent Document 1). BFs are roughly classified into two types: addition type and subtraction type. In particular, the subtraction type BF has an advantage that directivity can be formed with a smaller number of microphones than the addition type BF.

FIG. 6 is a block diagram showing a configuration of the subtraction type BF 300 when the number of microphones is two.

The subtraction type BF 300 shown in FIG. 6 has a delay device 310 and a subtractor 320.

The subtraction type BF 300 first calculates the time difference between the signals of the sounds existing in the target direction (hereinafter referred to as the “target sound”) that arrive at each microphone by the delay device 310, and adds the delay to obtain the phase of the target sound. Match. The time difference is calculated by the following equation (1). Here, “d” is the distance between the microphones, “c” is the speed of sound, and “τ _L ”is the delay amount. Further, here, “θ _L ”is an angle from the vertical direction to the straight line connecting the microphones (M1, M2) to the target direction.
τ _L =(dsin θ _L )/c (1)

Here, when the blind spot exists in the direction of the microphone M1 with respect to the centers of the microphones M1 and M2, the delay device 310 delays the input signal x ₁ (t) of the microphone M1. After that, in the subtraction type BF 300, the subtractor 320 performs the subtraction process according to the equation (2).
_{m (t) = x 2 (} t) -x 1 (t-τ L) ... (2)

The subtractor 320 can also perform the subtraction processing in the frequency domain as well, and in that case, the expression (2) is changed to the following expression (3).

FIG. 7 is a diagram showing a directional characteristic formed by the subtraction type BF300 using the two microphones M1 and M2.

Here, when θ _L =±π/2, the directivity formed by the subtractor 320 is a cardioid unidirectionality as shown in FIG. 7A, and when θ _L =0,π Has a figure-8 bidirectional pattern as shown in FIG. Here, a filter that forms unidirectionality from an input signal is called a "unidirectional filter", and a filter that forms bidirectionality is called a "bidirectional filter".

Further, in the subtractor 320, a strong directivity can be formed in the bi-directional blind spot by using a spectral subtraction method (Spectral Subtraction; hereinafter also simply referred to as “SS”). The directivity due to the SS is formed at all frequencies or a designated frequency band according to the equation (4). In the formula (4), the input signal X ₁ of the microphone M1 is used, but the same effect can be obtained even with the input signal X ₂ of the microphone M2. Here, β is a coefficient for adjusting the strength of SS.

In the subtractor 320, when the value becomes negative during the subtraction process, a process (flooring process) of replacing 0 or the original value is performed. With this method, the subtractor 320 extracts sounds existing in directions other than the target direction (hereinafter, referred to as “non-target sound”) by the bidirectional filter, and extracts the amplitude spectrum of the extracted non-target sound as the amplitude spectrum of the input signal. The target sound can be emphasized by subtracting from.
Y(n)=X ₁ (n)-βM(n) (4)

By the way, when only the sound existing in a specific area (hereinafter, referred to as “target area sound”) is desired to be picked up, the sound source (hereinafter, referred to as “non There is also a possibility that the target area sound will be picked up). Therefore, in the technique described in Patent Document 1, a method of collecting a target area sound by using a plurality of microphone arrays, directing the directivity from different directions to the target area, and intersecting the directivity in the target area (hereinafter, Called "area pickup").

In the conventional area sound collection, first, the ratio of the amplitude spectrum of the target area sound included in the BF output of each microphone array is estimated and used as the correction coefficient. For example, when two microphone arrays are used, the correction coefficient of the target area sound amplitude spectrum is calculated by the equations (5), (6) or (7), (8).

Here, “Y _1k (n)” and “Y _2k (n)” are the amplitude spectra of the BF outputs of the first and second microphone arrays, respectively. Also, "N" is the total number of frequency bins and "k" is the frequency. Furthermore, “α ₁ (n)” and “α ₂ (n)” are amplitude spectrum correction coefficients for the BF outputs of the first and second microphone arrays, respectively. Furthermore, “mode” represents the mode value, and “median” represents the median value.

In the conventional area sound collection process, each BF output is then corrected by the correction coefficient and SS is performed to extract the non-target area sound existing in the target area direction. Furthermore, the target area sound can be extracted by performing SS of the extracted non-target area sound from the output of each BF.

In this case, in the conventional area sound collection processing, in order to extract the non-target area sound N ₁ (n) existing in the direction of the target area viewed from the first microphone array, as shown in Expression (9), to SS BF output _Y 1 of the microphone array from (n) to the BF output _Y 2 of the second microphone array (n) a multiplied by the amplitude spectrum correction coefficient alpha _2. Similarly, according to the equation (10), the non-target area sound N ₂ (n) existing in the direction of the target area viewed from the second microphone array is extracted.
N ₁ (n)=Y ₁ (n)−α ₂ (n)Y ₂ (n) (9)
N ₂ (n)=Y ₂ (n)−α ₁ (n)Y ₁ (n) (10)

After that, in the conventional area sound collection processing, the target area sound is extracted by SS of the non-target area sound from each BF output according to the expressions (11) and (12). Expression (11) shows a process of extracting the target area sound with the first microphone array as a reference, and Expression (12) shows a process of extracting the target area sound with the second microphone array as a reference.
Z ₁ (n)=Y ₁ (n)−γ ₁ (n)N ₁ (n) (11)
Z ₂ (n)=Y ₂ (n)−γ ₂ (n)N ₂ (n) (12)

Here, γ ₁ (n) and γ ₂ (n) are coefficients for changing the strength during SS.

When the volume level of the background noise or the non-target area sound is high, the target area sound may be distorted by the SS performed when the target area sound is extracted, or an unpleasant noise such as musical noise may occur.

Therefore, in the area sound collection method described in Patent Document 2, the volume level of the input signal of the microphone and the volume level of the estimated noise are adjusted according to the levels of the background noise and the non-target area sound, and the extracted target area sound is obtained. Mixed. The musical noise generated by the process of extracting the target area sound becomes stronger as the volume level of the background noise and the non-target area sound increases, so the volume level of the sum of the input signal and the estimated noise to be mixed is also the background noise and the non-target area. Increase in proportion to the volume level of the area sound. Therefore, in the method described in Patent Document 2, the volume level of the background noise is calculated from the estimated noise obtained in the process of suppressing the background noise. Further, in the method described in Patent Document 2, the volume level of the non-target area sound exists in the non-target area sound existing in the target area direction extracted in the process of emphasizing the target area sound and in the direction other than the target area direction. Calculated from the sum of non-target area sounds.

By the way, here, it is assumed that in the conventional area sound collection processing, the ratio of the input signal to be mixed with the estimated noise is determined from the volume levels of the estimated noise and the non-target area sound. Then, when there is a non-target area sound near the target area, if the volume level of the input signal to be mixed is too high, the target area sound is mixed with the non-target area sound, and it becomes unclear which is the target area sound. ..

Therefore, in the method described in Patent Document 2, when the non-target area sound is large, the volume level of the input signal to be mixed is lowered, and the volume level of the estimated noise is increased to mix. That is, in the method described in Patent Document 2, the ratio of the input signal is increased when the non-target area sound does not exist or the volume level is low, and conversely, the ratio of the estimated noise when the volume level of the non-target area sound is high. Mix and mix a lot.

As described above, by using the method of Patent Document 2, it is possible to mask musical noise by mixing the target area sound with the input signal and the estimated noise, and to make the sound as if it were normal background noise. Further, if the method of Patent Document 2 is used, the distortion of the target area sound can be corrected by the component of the target area sound included in the microphone input signal, and the sound quality can be improved.

JP, 2014-072708, A JP, 2017-183902, A

Futoshi Asano, “Acoustic Technology Series 16 Sound Array Signal Processing-Sound Source Localization/Tracking and Separation-”, Acoustical Society of Japan, Corona Publishing, February 25, 2011

However, in the method of Patent Document 2, when the surrounding non-target area sound level is high, addition to the target area sound reduces the input signal and increases estimated noise, so that musical noise can be masked. , The effect of sound quality improvement is weakened.

Therefore, a sound collecting device, a sound collecting program, and a sound collecting method that suppress sound quality deterioration during the area sound collecting process are desired.

The first aspect of the present invention is: (1) Forming directivity in a target area sound direction by a beamformer with respect to each of capture signals output from a plurality of microphone arrays or signals based on the capture signals, And (2) extracting the non-target area sound existing in the target area direction by spectrally subtracting each of the target direction signals, and extracting the extracted non-target area sound. Target area sound extraction means for extracting a target area sound by spectrally subtracting from any one of the target direction signals; and (3) a non-target area in which a sound component whose sound source is a non-target area based on the captured signal is dominant. Mixing filter forming means for detecting a peak frequency from a sound component dominant signal and forming a suppression filter for suppressing a suppression frequency component based on the peak frequency of the non-target area sound component dominant signal; and (4) the mixing filter forming means. The suppression filter formed in 1. is applied to a mixing signal composed of the capture signal or a signal based on the capture signal output from any one of the microphone arrays to obtain a filtered mixing signal, and the filtered signal is further filtered. A signal mixing means for mixing a mixing signal with the target area sound extracted by the target area sound extracting means to obtain a mixed signal; and (5) the mixed signal acquired by the signal mixing means in the target area. It is characterized by having an output means for outputting as an area sound collection result.

A sound collecting program according to a second aspect of the present invention causes a computer to (1) form a directivity in a sound direction of a target area by a beam former for each of a capture signal output by a plurality of microphone arrays or a signal based on the capture signal. Then, directivity forming means for acquiring a target direction signal for each microphone array, and (2) spectral subtraction of each target direction signal to extract non-target area sounds existing in the target area direction, and extract Target area sound extraction means for extracting the target area sound by spectrally subtracting the non-target area sound from any of the target direction signals; and (3) a sound sourced from the non-target area based on the captured signal. A mixing filter forming means for detecting a peak frequency from a non-target area sound component dominant signal having a dominant component, and forming a suppression filter for suppressing a suppression frequency component based on the peak frequency of the non-target area sound component dominant signal; 4) The suppression filter formed by the mixing filter forming means is applied to a mixing signal composed of the captured signal output from any one of the microphone arrays or a signal based on the captured signal to obtain a filtered mixed signal. And (5) the signal mixing means that acquires and further mixes the filtered mixed signal with the target area sound extracted by the target area sound extraction means to acquire a mixed signal. It is characterized in that it functions as an output means for outputting the mixed signal as an area sound collection result of the target area.

A third aspect of the present invention is a sound collecting method performed by a sound collecting device, including (1) directivity forming means, target area sound extracting means, mixing filter forming means, signal mixing means and output means, and (2) the above The directivity forming means forms a directivity in the target area sound direction by the beam former for each of the capture signals output by the plurality of microphone arrays or the signals based on the capture signals, and the target direction signal for each microphone array. (3) The target area sound extraction means extracts the non-target area sound existing in the target area direction by spectrally subtracting each of the target direction signals, and the extracted non-target area sound is A target area sound is extracted by spectrally subtracting from the target direction signal, and (4) the mixing filter forming means is a non-target area in which a sound component whose sound source is a non-target area based on the captured signal is dominant. A peak frequency is detected from the sound component dominant signal, and a suppression filter for suppressing a suppression frequency component based on the peak frequency of the non-target area sound component dominant signal is formed; (5) the signal mixing means forms the mixing filter. The suppression filter formed by means is applied to a mixing signal composed of the captured signal or a signal based on the captured signal output from any one of the microphone arrays to obtain a filtered mixed signal, and further the filter The mixed signal is mixed with the target area sound extracted by the target area sound extraction means to obtain a mixed signal, and (6) the output means targets the mixed signal acquired by the signal mixing means. It is characterized in that it is output as the area sound collection result of the area.

According to the present invention, it is possible to suppress sound quality deterioration during area sound collection processing.

It is a block diagram which shows the functional structure of the sound collection device which concerns on 1st Embodiment. It is the block diagram shown about the example of the hardware constitutions of the sound collection device concerning a 1st and 2nd embodiment. 3 is a graph showing an example of a mixing signal and a notch filter processed by the sound collection device according to the first embodiment. 6 is a graph showing an example of signal mixing processing performed by the sound collection device according to the first embodiment. It is a block diagram which shows the functional structure of the sound collection device which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the conventional subtraction type BF. It is explanatory drawing shown about the example of the directional filter formed of the conventional subtraction type BF.

(A) First Embodiment Hereinafter, a first embodiment of a sound collecting device, a sound collecting program, and a sound collecting method according to the present invention will be described in detail with reference to the drawings.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing a functional configuration of a sound collecting device 100 according to the first embodiment.

The sound collection device 100 uses two microphone arrays MA (MA1 and MA2) to perform target area sound collection processing for collecting a target area sound from a sound source in the target area.

The microphone arrays MA1 and MA2 are arranged at any place in the space where the target area exists. The positions of the microphone arrays MA1 and MA2 with respect to the target area may be anywhere as long as the directivities overlap only in the target area, for example, they may be arranged opposite to each other across the target area. Each microphone array MA is composed of two or more microphones M, and each microphone M picks up an acoustic signal. In this embodiment, two microphones M1 and M2 that pick up an acoustic signal are arranged in each microphone array MA. That is, in this embodiment, each microphone array MA constitutes a 2ch microphone array. The distance between the two microphones M1 and M2 is not limited, but in the example of this embodiment, the distance between the two microphones M1 and M2 is 3 cm. Note that the number of microphone arrays MA is not limited to two, and when there are a plurality of target areas, it is necessary to arrange the microphone arrays MA in a number that can cover all areas.

Next, the internal configuration of the sound collection device 100 will be described with reference to FIG.

As shown in FIG. 1, the sound collection device 100 includes a signal input unit 1, a noise suppression unit 2, a directivity formation unit 3, a delay correction unit 4, spatial coordinate data 5, a correction coefficient calculation unit 6, and a target area sound extraction unit 7. , A mixing filter forming unit 8, a signal mixing unit 9, and a signal output unit 10.

The sound collection device 100 may be configured entirely by hardware (for example, a dedicated chip or the like), or may be configured as software (program) for a part or all. The sound collection device 100 may be configured, for example, by installing a program (including the signal processing program of the embodiment) in a computer having a processor and a memory.

Next, the hardware configuration of the sound collection device 100 will be described with reference to FIG.

The sound collection device 100 may be configured entirely by hardware (for example, a dedicated chip or the like), or may be configured as software (program) for a part or all. The sound collection device 100 may be configured, for example, by installing a program (including the sound collection program of the embodiment) in a computer having a processor and a memory.

FIG. 2 is a block diagram showing an example of the hardware configuration of the sound collection device 100.

FIG. 2 shows an example of a hardware configuration when the sound collection device 100 is configured using software (computer).

The sound collection device 100 illustrated in FIG. 2 includes a computer 200 in which a program (including the sound collection program of the embodiment) is installed as a hardware component. If the computer 200 does not include a conversion unit that converts an analog signal (a signal supplied from the super-directional microphones M1 and M2) into a digital signal, the sound collection device 100 includes a conversion unit (not shown). You may do it. Further, the computer 200 may be a computer dedicated to a sound collection program, or may be configured to be shared with a program having another function.

The computer 200 illustrated in FIG. 2 includes a processor 201, a primary storage unit 202, and a secondary storage unit 203. The primary storage unit 202 is a storage unit that functions as a work memory (work memory) of the processor 201. For example, a high-speed operating memory such as a DRAM (Dynamic Random Access Memory) can be applied. The secondary storage unit 203 is a storage unit that records various data such as an OS (Operating System) and program data (including data of the sound collection program according to the embodiment), and is a nonvolatile memory such as a FLASH memory or an HDD. Sex memory can be applied. In the computer 200 of this embodiment, when the processor 201 is activated, the OS and programs recorded in the secondary storage unit 203 (including the sound collection program according to the embodiment) are read and expanded on the primary storage unit 202. Execute.

The specific configuration of the computer 200 is not limited to the configuration of FIG. 2, and various configurations can be applied. For example, if the primary storage unit 202 is a non-volatile memory (for example, a FLASH memory or the like), the secondary storage unit 203 may be excluded.

(A-2) Operation of the first embodiment The signal input unit 1 receives an acoustic signal picked up by each microphone array MA (MA1, MA2) and converts the acoustic signal from an analog signal to a digital signal. .. Then, the signal input unit 1 transforms the acoustic signal (digital signal) from the time domain to the frequency domain using a predetermined method (for example, fast Fourier transform). Hereinafter, in each microphone array MA, the input signals in the frequency domain of the microphones M1 and M2 will be described as X ₁ and X ₂ , respectively.

The noise suppression unit 2 estimates and suppresses the background noise component included in the input signal acquired by the signal input unit 1. The noise suppression method applied to the noise suppression unit 2 is not limited, but for example, SS or Wiener filtering can be used.

The directivity forming unit 3 forms a directivity in the target area direction by the BF according to the expression (4) for the signal in which the background noise is suppressed by the noise suppressing unit 2 for each microphone array MA. Hereinafter, the amplitude spectra of the BF outputs of the microphone arrays MA1 and MA2 will be described as Y _1k (n) and Y _2k (n), respectively.

The delay correction unit 4 calculates and corrects the delay caused by the difference in the distance between the target area and each microphone array MA. The delay correction unit 4 first acquires the position of the target area and the position of the microphone array from the spatial coordinate data 5, and calculates the difference in the arrival time of the target area sound to each microphone array. Next, with reference to the microphone array arranged farthest from the target area, a delay is added so that sounds of the target area reach all the microphone arrays at the same time.

The spatial coordinate data 5 holds the positional information of all the target areas, the microphone arrays MA, and the microphones M forming the microphone arrays MA.

The correction coefficient calculation unit 6 calculates a correction coefficient for making the amplitude spectra of the target area sound components included in each BF output the same. Hereinafter, the amplitude spectrum correction coefficients for the BF outputs of the microphone arrays MA1 and MA2 will be described as α ₁ (n) and α ₂ (n). The correction coefficient calculation unit 6 calculates the correction coefficients α ₁ (n) and α ₂ (n) according to, for example, Expression (5), Expression (6), Expression (7), or Expression (8).

The target area sound extraction unit 7 extracts the non-target area sound existing in the target area direction from each BF output data corrected by the correction coefficient calculated by the correction coefficient calculation unit 6. The target area sound extraction unit 7 SS-processes each BF output data corrected by the correction coefficient calculated by the correction coefficient calculation unit 6 according to, for example, Expression (9) or Expression (10), and the non-target area existing in the direction of the target area. The sound (N ₁ (n) or N ₂ (n)) is extracted.

Further, the target area sound extraction unit 7 performs SS by extracting the extracted non-target area sound (N ₁ (n) or N ₂ (n)) from the output of each BF according to Expression (11) or Expression (12). Area sounds (Z ₁ (n) or Z ₂ (n)) are extracted.

The mixing filter formation unit 8 has a frequency at which the power reaches a peak from the non-target area sound (N ₁ (n) or N ₂ (n)) extracted by the target area sound extraction unit 7 by the expression (9) or the expression (10). (Hereinafter, referred to as “peak frequency”) and suppresses only the component (power) of the peak frequency (hereinafter referred to as “notch filter”, “band stop filter”, or “band elimination filter”) Form).

When detecting the peak frequency, the mixing filter forming unit 8 outputs not only the extracted non-target area sound (N ₁ (n) or N ₂ (n)) but also another input signal (of either microphone array MA). Based on the (input signal), a signal in which the component of the “sound whose sound source is the non-target area” is dominant (hereinafter, referred to as “non-target area sound component dominant signal”) may be used. For example, in the mixing filter forming unit 8, when detecting the peak frequency, the sound (non-target sound) extracted by the directivity forming unit 3 by the formula (2) or the formula (3) may be used. In the mixing filter forming unit 8, the peak frequency may be detected by, for example, smoothing the amplitude spectrum and obtaining the maximum amplitude spectrum. In addition, the mixed filter forming unit 8 may return the extracted non-target area sound to the time domain, perform linear predictive coding (LPC) analysis, and detect the peak of the LPC spectrum envelope as a peak frequency.

The mixed filter forming unit 8 may use an existing notch filter forming method or an original filter forming method when forming the notch filter after detecting the peak frequency. When the notch filter is formed by the unique method in the mixed filter forming unit 8, for example, the frequency band to be suppressed (hereinafter referred to as “suppression frequency”) may be only the peak frequency, or the frequency band including the peak frequency ( For example, it may be a frequency band having a predetermined bandwidth centered on the peak frequency).

Further, the mixing filter forming unit 8 may have a plurality of suppression frequencies. For example, the mixing filter forming unit 8 sets not only the peak frequency (first peak) of the non-target area sound but also the second and subsequent peaks (second, third,..., Nth) as suppression frequencies. You may do so.

The method of determining the suppression amount (attenuation amount) applied to each suppression frequency in the mixing filter forming unit 8 is not limited. In the mixing filter forming unit 8, for example, the amount of suppression for each suppression frequency may be a fixed value, or for each suppression frequency dynamically according to the amplitude spectrum of the peak frequency of the non-target area sound or the average amplitude spectrum of the entire band. The amount of suppression may be set. For example, in the mixing filter forming unit 8, the suppression amount applied to the suppression frequency component may be set to be larger as the amplitude spectrum of the peak frequency of the non-target area sound or the average amplitude spectrum of the entire band is larger. Moreover, the mixing filter forming unit 8 may change the suppression amount based on the sound output from the signal output unit 10.

When the suppression frequency has a bandwidth, the mixing filter forming unit 8 may uniformly apply the same suppression amount in the bandwidth, or may become weaker as the frequency deviates from the peak frequency (so that the suppression amount decreases). You may set it.

When the same non-target area sound constantly exists (for example, when there is a steady loud siren sound, engine sound, etc.), the mixed filter forming unit 8 does not dynamically form a filter. Instead, the peak frequency (the non-target area sound) may be detected in advance to form the notch filter, or a plurality of notch filters may be prepared and switched in advance. In addition, in the mixing filter forming unit 8, the notch filter can be applied in both the frequency domain and the time domain.

The signal mixing unit 9 forms a signal obtained by applying the notch filter formed by the mixing filter forming unit 8 to the mixing signal (hereinafter, referred to as “filtered mixing signal X _MIX (n)”). Then, the signal mixing unit 9 mixes the filtered mixed signal X _MIX (n) with the target area sound (Z ₁ (n) or Z ₂ (n)) extracted by the target area sound extracting unit 7. A signal (hereinafter, referred to as “mixed signal W(n)”) is formed.

The mixing signal may be an input signal acquired by the signal input unit 1 (an input signal of any microphone in any microphone array MA), or a signal in which background noise is suppressed by the noise suppression unit 2 (any microphone). A signal with background noise suppressed for an input signal of any microphone in the array MA) may be used.

Here, as an example, it is assumed that the target area sound extraction unit 7 collects an area sound with the microphone array MA1 as a reference according to the expression (11) and acquires the target area sound Z ₁ (n). In this case, the signal mixing unit 9 may mix the target area sound (Z ₁ (n)) and the filtered mixed signal X _MIX (n) according to (13) below, for example. Here, “μ” is a parameter (coefficient) for adjusting the magnitude of the filtered mixed signal X _MIX (n) to be mixed. Further, here, “W ₁ (n)” is a mixed signal calculated based on the target area sound Z ₁ (n) extracted with the microphone array MA1 as a reference. In the target area sound extraction unit 7, according to the expression (12), the area sound is picked up with the microphone array MA2 as a reference, and the target area sound Z ₂ (n) is filtered by the mixed signal X _MIX (n). When the signals are mixed to obtain the mixed signal W ₂ (n), the expression is given by the expression (14).

In the target area sound extraction unit 7, μ may be a predetermined constant or may be changed adaptively (dynamically) according to the level of the non-target area sound. In the target area sound extraction unit 7, the value of μ may be changed according to the signal output from the signal output unit 10 (for example, the power of W ₁ (n) or W ₂ (n)). Good.

Further, in the target area sound extraction unit 7, for example, as shown in the following Expression (15) or Expression (16), when calculating the mixed signal, the target area sound (Z ₁ (n) or Z ₂ ( n)) may be added as a parameter (coefficient) for adjusting the size. In this case, the target area sound extraction unit 7 can output only the mixing signal by setting ρ to 0.
W ₁ (n)=Z ₁ (n)+μX _MIX (n) (13)
W ₂ (n)=Z ₂ (n)+μX _MIX (n) (14)
W ₁ (n)=ρZ ₁ (n)+μX _MIX (n) (15)
W ₂ (n)=ρZ ₂ (n)+μX _MIX (n) (16)

The signal output unit 10 outputs the mixed signal (W ₁ (n) or W ₂ (n)) calculated by the signal mixing unit 9 or a signal based on the mixed signal as the area sound collection result. The output format and the output means (output medium) when the signal output unit 10 outputs the mixed signal are not limited. For example, the signal output unit 10 may output the mixed signal in the frequency domain or the time domain.

Next, a specific example of the area sound collecting process in the sound collecting device 100 will be described with reference to FIGS. 3 and 4.

First, here, the frequency of the first peak in the non-target area sound is "fPNT1", the frequency of the second peak in the non-target area sound is "fPNT2",..., The frequency of the N-th peak in the non-target area sound. Shall be referred to as “fPNTN”.

FIG. 3A shows the signal input unit 1 and the noise suppression unit 2 when the sound (noise) having the sound source of the non-target area included in the input signal X ₁ (n) as the mixing signal is a siren sound. , The spectrum of the siren sound extracted by the directivity forming unit 3 or the target area sound extracting unit 7. Further, FIG. 3B is a diagram showing an example of a notch filter designed to suppress the first peak (maximum peak) and the second peak of the sound (noise) whose sound source is the non-target area. , The suppression amount for each frequency is shown. In addition, FIG. 4 shows the spectrum of the input signal X ₁ (n) in the case where the mixing area signal includes the target area sound and the sound (noise) generated by the sound source in the non-target area (the siren sound shown in FIG. 3A). Is shown.

In the example of FIG. 3 and FIG. 4, the mixing filter forming unit 8 has a band of 200 Hz before and after the frequency fPNT1 of the first peak (maximum peak) of the sound (noise) whose sound source is the non-target area, and A band of 200 Hz before and after the frequency fPNT2 of the second peak of the sound (noise) whose sound source is the target area is set as the suppression frequency. Hereinafter, the suppression frequency corresponding to the frequency fPNT1 will be referred to as a first suppression frequency, and the suppression frequency corresponding to the frequency fPNT2 will be referred to as a second suppression frequency.

In the examples of FIGS. 3 and 4, at the first suppression frequency and the second suppression frequency of the mixing signal (input signal X ₁ (n)), a siren sound that is a sound (noise) that uses the non-target area as a sound source. Will be described as being dominant over the target area sound (for example, the voice of the speaker of the telephone device).

In the notch filter shown in FIG. 3B, a first suppression frequency band (a band of 200 Hz before and after centering on the frequency fPNT1) and a second suppression frequency band (a band of 200 Hz before and after centering on the frequency fPNT2). ), the suppression amount is set to a value of 1 or less. In the notch filter, the suppression amount of each frequency can be represented by a numerical value between 0 and 1, and the smaller the numerical value, the larger the suppression amount. In the notch filter shown in FIG. 3B, the suppression amount of the peak frequencies (fPNT1, fPNT2) is maximum in both the first and second suppression frequency bands, and the peak frequencies (fPNT1, fPNT2) It is set so that the amount of suppression decreases as the distance increases.

As a result, in the sound (noise) (siren sound) having the non-target area as the sound source shown in FIG. 3A, the power is reduced in the bands of the first and second suppression frequencies in the notch filter shown in FIG. 3B. It is suppressed.

FIG. 4 shows the case where the input signal X ₁ (n) in the case where the target area sound and the sound (noise) generated by the sound source in the non-target area (the siren sound shown in FIG. 3A) are included are mixed signals X _MIX (n ) Is indicated by a solid line, and the mixed signal (input signal X ₁ (n)) is filtered by a notch filter (notch filter shown in FIG. 3B) and then mixed. The spectrum of the signal X _MIX (n) is shown by the dotted line. In the example of FIG. 4, by applying the filtered mixed signal X _MIX (n), in the mixed signal W ₁ (n), the component of the siren sound that is the sound (noise) whose sound source is the non-target area It can be seen that the SN ratio is improved in the band (1st and 2nd suppression frequencies) in which is dominant.

(A-3) Effects of First Embodiment According to the first embodiment, the following effects can be achieved.

The sound collecting device 100 of the first embodiment detects the peak frequency of the non-target area sound extracted at the time of extracting the target area sound, and forms a notch filter that suppresses the suppression frequency component based on the peak frequency. Then, the sound collection device 100 can obtain the filtered mixed signal X _MIX (n) in which the main component of the non-target area sound included in the mixed signal is suppressed by applying the notch filter to the mixed signal. it can. Then, the sound collection device 100 generates a mixed signal (W ₁ (n) or W ₂ (n)) in which the filtered mixed signal X _MIX (n) is mixed with the target area sound, and the mixed signal is generated. The signal Z(n) based on is output as the area sound collection result. As a result, the sound collection device 100 can improve the sound quality of the area sound collection result while suppressing the mixing of the non-target area sound.

(B) Second Embodiment Hereinafter, a second embodiment of the sound collecting device, the sound collecting program, and the sound collecting method according to the present invention will be described in detail with reference to the drawings.

(B-1) Configuration of Second Embodiment FIG. 5 is a block diagram showing a functional configuration according to the sound collecting device 100A of the second embodiment, and regarding the same or corresponding portions as those in FIG. 1 described above. Are given the same or corresponding symbols.

The differences between the second embodiment and the first embodiment will be described below.

The hardware configuration of the sound collection device 100A of the second embodiment can also be shown using FIG. 2 described above.

The sound collecting device 100A of the second embodiment differs from that of the first embodiment in that a mixing filter adjusting unit 11 is added.

(B-2) Operation of Second Embodiment Next, the operation (sound collecting method according to the embodiment) of the sound collecting apparatus 100A of the second embodiment having the above-described configuration will be described as the first embodiment. The difference will be mainly explained.

In the second embodiment, as described above, only the mixing filter adjusting unit 11 is different from the first embodiment, and hence the operation of the mixing filter adjusting unit 11 will be mainly described below.

The mixing filter adjusting unit 11 first acquires the target area sound from the target area sound extracting unit 7 and detects the peak frequency.

Next, the mixing filter adjusting unit 11 compares the peak frequency of the target area sound with the suppression frequency (peak of the non-target area sound) of the notch filter formed in the mixing filter forming unit 8, and determines the distance (frequency) of those frequency bands. For example, the suppression amount by the notch filter is adjusted by the difference between the peak frequency of the target area sound and the peak frequency of the non-target area sound; hereinafter referred to as “frequency difference”).

For example, the mixed filter adjusting unit 11 may reduce the suppression amount of the notch filter formed by the mixed filter forming unit 8 to 1/2 when the frequency difference is within 100 Hz.

In the mixing filter adjustment unit 11, the adjustment of the suppression amount may be uniformly changed as long as it is equal to or less than a certain difference, or may be set to gradually increase as the difference increases.

In addition, in the mixing filter forming unit 8, not only the peak frequency (first peak) of the non-target area sound but also the second and subsequent peaks (second, third,..., Nth) are set as suppression frequencies. In this case, the mixing filter adjusting unit 11 detects the 1st to Nth peaks of the target area sound as well, and detects each peak of the non-target area sound (1st to Nth peak) and each peak of the target area sound. (1st to Nth peak) may be compared to adjust the suppression amount of each suppression frequency corresponding to each peak (1st to Nth peak) of the non-target area sound.

Here, the frequency of the first peak in the target area sound is "PT1", the frequency of the second peak in the target area sound is "PT2",..., The frequency of the Nth peak in the target area sound is "PTN". I shall call it. Further, here, the frequency of the first peak in the non-target area sound is "PNT1", the frequency of the second peak in the non-target area sound is "PNT2",..., The frequency of the N-th peak in the non-target area sound. Shall be referred to as "PNTN". In this case, the mixed filter adjustment unit 11 adjusts the suppression amount of the suppression frequency based on the frequency PT1 according to the frequency difference between the frequency PT1 and the frequency PNT1, and according to the frequency difference between the frequency PT2 and the frequency PNT2. Then, the suppression amount of the suppression frequency based on the frequency PT2 is adjusted, and the suppression amount of the suppression frequency based on the frequency PTN is adjusted according to the frequency difference between the frequency PTN and the frequency PNTN. In this case, the method of adjusting each suppression amount adjusted by the mixing filter adjusting unit 11 may be the same method as the above example.

(B-3) Effects of Second Embodiment According to the second embodiment, the following effects can be achieved.

In the sound collecting device 100A of the second embodiment, the mixing filter adjusting unit 11 compares the peak frequency of the target area sound with the suppression frequency of the notch filter (peak frequency of the non-target area sound) to adjust the suppression amount. Thus, the sound quality can be stably improved regardless of the type of the non-target area sound.

(C) Other Embodiments The present invention is not limited to each of the above-described embodiments, and modified embodiments as exemplified below can be cited.

(C-1) In each of the above embodiments, the noise suppression unit 2 is not essential and may be omitted. For example, in a quiet environment where there is almost no background noise, the process of the noise suppressing unit 2 may be omitted.

(C-2) In each of the above-described embodiments, the delay correction unit 4 may be omitted because it is not essential. For example, the processing of the delay correction unit 4 may be excluded if the delay does not occur from the beginning or is negligible depending on the arrangement of each microphone array MA and the target area sound.

(C-3) In each of the above embodiments, the correction coefficient calculation unit 6 is not essential and may be omitted. For example, when it is clear that the difference in the amplitude spectrum of the target area sound captured by each microphone M (each microphone M forming each microphone array MA) is small due to the arrangement of each microphone array MA and the target area sound, The processing of the correction coefficient calculation unit 6 may be excluded.

100, 100A... Sound collection device, 1... Signal input unit, 2... Noise suppression unit, 3... Directivity formation unit, 4... Delay correction unit, 5... Spatial coordinate data, 6... Correction coefficient calculation unit, 7... Target area Sound extraction unit, 8... Mixing filter forming unit, 9... Signal mixing unit, 10... Signal output unit, 11... Mixing filter adjusting unit, M, M1, M2... Microphone, MA, MA1, MA2... Microphone array.

Claims (7)

Directivity formation for obtaining a target direction signal for each microphone array by forming a directivity in the target area sound direction by the beamformer with respect to each of the capture signals output by the plurality of microphone arrays or the signals based on the capture signals. Means and
A non-target area sound existing in the target area direction is extracted by spectrally subtracting each of the target direction signals, and the target area sound is extracted by spectrally subtracting the extracted non-target area sound from any of the target direction signals. Target area sound extraction means for extracting
The peak frequency is detected from the non-target area sound component dominant signal in which the sound component whose sound source is the non-target area based on the captured signal is dominant, and the suppression frequency component based on the peak frequency of the non-target area sound component dominant signal is detected. Mixing filter forming means for forming a suppressing filter for suppressing,
The suppression filter formed by the mixing filter forming means is applied to a mixing signal composed of the captured signal or a signal based on the captured signal output from any one of the microphone arrays to obtain a filtered mixed signal. A signal mixing means for further mixing the filtered mixing signal with the target area sound extracted by the target area sound extracting means to obtain a mixed signal,
Output means for outputting the mixed signal acquired by the signal mixing means as an area sound collection result of the target area. The sound collecting device according to claim 1, wherein the mixing filter forming unit sets a frequency in a predetermined range including a peak frequency of the non-target area sound as the suppression frequency of the suppression filter. The mixing filter forming means calculates and acquires a frequency difference between the suppression frequency and a peak frequency of the target area sound, and suppresses the suppression frequency component for the suppression filter according to the acquired frequency difference. The sound pickup device according to claim 1 or 2, wherein a suppression amount is set. 4. The sound collecting device according to claim 1, wherein the non-target area sound extracted by the target area sound extracting means is applied as the non-target area sound component dominant signal. The non-target sound whose directivity is directed in a direction other than the target area is applied as the non-target area sound component dominant signal based on the captured signal of any one of the microphone arrays. The sound collecting device according to any one of 1. Computer,
Directivity formation for obtaining a target direction signal for each microphone array by forming a directivity in the target area sound direction by the beamformer with respect to each of the capture signals output by the plurality of microphone arrays or the signals based on the capture signals. Means and
A non-target area sound existing in the target area direction is extracted by spectrally subtracting each of the target direction signals, and the target area sound is extracted by spectrally subtracting the extracted non-target area sound from any of the target direction signals. Target area sound extraction means for extracting
The peak frequency is detected from the non-target area sound component dominant signal in which the sound component whose sound source is the non-target area based on the captured signal is dominant, and the suppression frequency component based on the peak frequency of the non-target area sound component dominant signal is detected. Mixing filter forming means for forming a suppressing filter for suppressing,
The suppression filter formed by the mixing filter forming means is applied to a mixing signal composed of the captured signal or a signal based on the captured signal output from any one of the microphone arrays to obtain a filtered mixed signal. A signal mixing means for further mixing the filtered mixing signal with the target area sound extracted by the target area sound extracting means to obtain a mixed signal,
A sound collecting program, which is caused to function as output means for outputting the mixed signal acquired by the signal mixing means as an area sound collection result of a target area. In the sound collecting method performed by the sound collecting device,
A directivity forming unit, a target area sound extracting unit, a mixing filter forming unit, a signal mixing unit and an output unit,
The directivity forming means forms a directivity in a target area sound direction by a beam former with respect to each of a capture signal output by a plurality of microphone arrays or a signal based on the capture signal, and the directivity is generated for each microphone array. Get the signal,
The target area sound extraction means extracts a non-target area sound existing in the target area direction by spectrally subtracting each of the target direction signals, and extracts the extracted non-target area sound from any of the target direction signals. Extract the target area sound by subtracting the spectrum,
The mixing filter forming means detects a peak frequency from a non-target area sound component dominant signal in which a sound component whose sound source is a non-target area based on the captured signal is dominant, and a peak frequency of the non-target area sound component dominant signal. Form a suppression filter that suppresses the suppression frequency component based on
The signal mixing means applies the suppression filter formed by the mixing filter forming means to the mixing signal output from any one of the microphone arrays, or the mixing signal composed of the signal based on the acquisition signal and filtered. A signal for mixing is acquired, and the filtered signal for mixing is further mixed with the target area sound extracted by the target area sound extracting means to acquire a signal after mixing,
The sound collecting method, wherein the output means outputs the mixed signal acquired by the signal mixing means as an area sound collection result of a target area.

Images