JP5482854B2 - Sound collecting device and program - Google Patents

Description

  The present invention relates to a sound collection device and a program, and can be applied to, for example, a sound collection device and a program that emphasizes sounds in a specific area and suppresses sounds in other areas.

  A beamformer using a microphone array (hereinafter referred to as a technique for emphasizing sound existing in a specific direction (speech and sound; hereinafter referred to as sound collectively)) and suppressing other sounds. BF). Here, the beamformer is a technique for forming directivity using a time difference between signals reaching each microphone (see Non-Patent Document 1).

  BF is roughly divided into two types, an addition type and a subtraction type. In particular, the subtraction type BF has an advantage that directivity can be formed with a smaller number of microphones than the addition type BF. As a typical method of the subtraction type BF, there is BF using a spectral subtraction method (hereinafter referred to as SS) (see Non-Patent Document 2).

FIG. 15 is a block diagram showing a configuration related to SS when the number of microphones is two. 15, the distance between the two microphones 11 and the microphone 12 is d, the angle to the target sound source T and theta _L from the front of the microphone 11 and the microphone 12. In SS, first delay unit 13 calculates the time difference tau _L of signals arriving from the target direction theta _L microphone 11 and the microphone 12, adjust the phase of the target sound source direction of the sound signal by adding delay. Note that the sound from other directions is not emphasized because the phases are not aligned even through the delay device 13. The time difference τ _L is calculated by the following equation (1). In equation (1), c is the speed of sound and _Di is the amount of delay.

  Here, when the target sound source T exists in the direction of the microphone 11 with respect to the center of the microphone 11 and the microphone 12, a delay process is performed on the input of the microphone 11. Thereafter, the adder 14 performs addition processing according to the equation (2), and the subtractor 15 performs subtraction processing according to the equation (3). Thereby, the sound in the target sound source direction is emphasized by the addition process, and the sound other than the target sound source direction is extracted by the subtraction process. Note that addition processing and subtraction processing can be performed in the same manner in the frequency domain, and in this case, Equation (2) and Equation (3) are changed to Equation (4) and Equation (5), respectively. FIG. 15 illustrates a case where addition processing and subtraction processing are performed according to equations (4) and (5).

  The spectrum subtractor 16 performs processing according to the expression (6) using the data subjected to the addition processing and the subtraction processing, whereby the sound in the target sound source direction can be emphasized and the other sounds can be suppressed. β is a coefficient for changing the strength of SS.

  In an actual environment, when it is desired to collect only a sound in a specific area (hereinafter referred to as a target area sound), there may be a situation in which a large number of noises (hereinafter referred to as non-target area sounds) exist around the area. Ordinarily, BF can form directivity only in the vertical and horizontal directions. Therefore, as shown in FIG. 16, when a non-target area sound source exists in the same direction as the target area, there is a problem that not only the target area sound but also the non-target area sound is emphasized.

  In order to solve this problem, the technique described in Patent Document 1 uses two microphone arrays, and directs the directivity of each microphone array from different positions in a target area direction and a direction other than the target area by BF. We have proposed a method for estimating and emphasizing the target area sound from the level difference of the sound coming from each direction.

JP 2007-235358 A

Tadashi Asano, “Sound Array Signal Processing-Localization, Tracking and Separation of Sound Sources”, The Acoustical Society of Japan, Corona, published on February 25, 2011 Takashi Yagami, Makoto Morito, Satoshi Yamada, Tetsuji Ogawa, "Sound source separation technology using a square microphone array (<Special issue> Efforts for practical application of speech recognition technology)", Information Processing Society of Japan, Information Processing 51 (11), pp. 1410-1416, 2010

  However, in the technique described in Patent Document 1, there is a limitation that the microphone array must be arranged at an equal distance from the target area. That is, for example, when two microphone arrays 1 and 2 are arranged, it is necessary to make the distance from the microphone array 1 to the target area equal to the distance from the microphone array 2 to the target area. For this reason, when the target area is changed, there is a problem that the microphone array has to be rearranged for each change. Further, since the technique described in Patent Document 1 is based on the addition type BF, it is necessary to provide a large number of microphones for constituting one microphone array.

  For this reason, a microphone array can be configured with a small number of microphones, and only the target area sound can be emphasized without adjusting the position of the microphone array, even in a situation where the target area is surrounded by a non-target area sound source. There is a need for sound devices and programs.

  In order to solve such a problem, the first aspect of the present invention provides (1) a plurality of microphone arrays, and (2) directivity formation that forms a directivity in the direction of a target area by a beamformer with respect to the output of each microphone array. And (3) a microphone that corrects a delay caused by a difference in distance between the target area and each microphone array so that the target area sound simultaneously arrives at all microphone arrays in the output after the beam former of each microphone array. (4) In order to make the power of the target area sound included in the beamformer output of each microphone array all the same, the maximum of the ratio of the amplitude spectrum between the beamformer outputs of each microphone array. A target area sound power correction coefficient calculation unit that calculates a frequency value or a median value as a correction coefficient; (5) Using the correction coefficient calculated by the target area sound power correction coefficient calculation unit, the beamformer output of each microphone array is corrected, and the non-target area sound existing in the target area direction is extracted by spectrum subtraction. And a target area sound extraction unit that extracts a target area sound by performing spectral subtraction of the non-target area sound extracted thereafter from the beamformer output of each microphone array.

  The second aspect of the present invention includes: (1) a plurality of microphone arrays; (2) a directivity forming unit that forms directivity in the direction of a target area with a beamformer with respect to the output of each microphone array; and (3) each microphone. An inter-microphone array delay correction unit that corrects a delay caused by a difference in the distance between the target area and each microphone array so that the target area sound arrives at all the microphone arrays at the same time after the beamformer of the array; 4) In order to make the power of the target area sound included in the beamformer output of each microphone array all the same, calculate a coefficient that minimizes the square of the difference in power of the beamformer output of each microphone array; A target area sound power correction coefficient calculation unit as a correction coefficient; and (5) target area sound power correction coefficient calculation. The beamformer output of each microphone array is corrected using the correction coefficient calculated in the section, and the non-target area sound existing in the target area direction is extracted by subtracting the spectrum, and then the extracted non-target area sound is A sound collection device comprising: a target area sound extraction unit that extracts a target area sound by performing spectral subtraction from a beamformer output of a microphone array.

  According to a third aspect of the present invention, there is provided a computer to which signals from a plurality of microphone arrays are provided. (1) A directivity forming unit that forms directivity in the direction of a target area by a beam former with respect to the output of each microphone array; ) Inter-microphone array delay correction unit that corrects the delay caused by the difference in distance between the target area and each microphone array so that the target area sound arrives at all the microphone arrays simultaneously at the output after the beamformer of each microphone array (3) In order to make the power of the target area sound included in the beamformer output of each microphone array all the same, the mode value or median value of the ratio of the amplitude spectrum between the beamformer outputs of each microphone array is set. A target area sound power correction coefficient calculation unit that calculates and sets a correction coefficient, (4 Using the correction coefficient calculated by the target area sound power correction coefficient calculator, correct the beamformer output of each microphone array, extract the non-target area sound that exists in the direction of the target area by subtracting each spectrum, and then extract The sound collection program is configured to function as a target area sound extraction unit that extracts a target area sound by subtracting the spectrum of the non-target area sound from the beamformer output of each microphone array.

  According to a fourth aspect of the present invention, there is provided a computer to which signals from a plurality of microphone arrays are provided. (1) A directivity forming unit that forms directivity in the direction of a target area by a beam former with respect to the output of each microphone array; ) Inter-microphone array delay correction unit that corrects the delay caused by the difference in distance between the target area and each microphone array so that the target area sound arrives at all the microphone arrays simultaneously at the output after the beamformer of each microphone array (3) In order to make the power of the target area sound included in the beamformer output of each microphone array all the same, the coefficient that minimizes the square of the power difference of the beamformer output of each microphone array is calculated. And a target area sound power correction coefficient calculation unit as a correction coefficient, (4) target error Using the correction coefficient calculated by the sound power correction coefficient calculation unit, the beamformer output of each microphone array is corrected, and the non-target area sound existing in the direction of the target area is extracted by subtracting each spectrum, and then extracted. A sound collection program that functions as a target area sound extraction unit that extracts a target area sound by subtracting the spectrum of the non-target area sound from the beamformer output of each microphone array.

  According to the present invention, a microphone array can be configured with a small number of microphones, and only the target area sound is emphasized even in a situation where the target area is surrounded by a non-target area sound source without adjusting the position of the microphone array. be able to.

It is a block diagram which shows the structure of the sound collection device which concerns on 1st Embodiment. It is a block diagram which shows the structure of the target area sound extraction part. It is a flowchart which shows the process of the sound collection device which concerns on 1st Embodiment. It is the figure which showed arrangement | positioning of the microphone array and each sound source in the performance evaluation experiment which concerns on 1st Embodiment. It is the figure which showed the suppression amount of the non-target area sound in each arrangement pattern of 1st Embodiment and the existing method. It is a block diagram which shows the structure of the sound collection device which concerns on 2nd Embodiment. It is a flowchart which shows the process of the sound collection device which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the sound collection device which concerns on 3rd Embodiment. It is a flowchart which shows the process of the sound collection device which concerns on 3rd Embodiment. It is a block diagram which shows the structure of the target area sound power correction coefficient calculation part. It is the figure which showed arrangement | positioning of the microphone array and each sound source in the performance evaluation experiment which concerns on 3rd Embodiment. It is the figure which showed the suppression amount of the non-target area sound in each arrangement pattern of 3rd Embodiment and the existing method. It is a block diagram which shows the structure of the sound collection device which concerns on 4th Embodiment. It is a flowchart which shows the process of the sound collection device which concerns on 4th Embodiment. It is a block diagram which shows the structure which concerns on a spectrum subtraction method. It is explanatory drawing which shows the state which orient | assigned the directional beam to the target area direction from one microphone array. It is explanatory drawing which shows the state which used two microphone arrays and directed the directional beam from the different places to the direction of the target area. It is explanatory drawing which shows the difference in the spectrum of BF output signal of each microphone array, a target area sound component, and a non-target area sound component. It is explanatory drawing which showed the ratio of the amplitude spectrum between BF output signals of each microphone array with the histogram.

(A) Technical idea common to the first to fourth embodiments In the first to fourth embodiments, first, a plurality of microphone arrays are arbitrarily arranged in a space including the target area, and the target area is directed by BF. Form a directional beam. As an example, FIG. 17 shows an image when the directional beams of two microphone arrays are directed to the target area. In this state, the BF directivity of each of the microphone arrays 1 and 2 includes a non-target area sound component in the target area sound direction. However, the target area is included in the directional beams of all microphone arrays 1 and 2. Therefore, as shown in FIGS. 18A and 18C, the target area sound component is included in the output signals of all BFs in the same ratio and distribution. In contrast, the components of the non-target area sounds 1 and 2 are different for the BF output signals of the microphone arrays 1 and 2 as shown in FIGS. The first to fourth embodiments utilize such features.

  That is, when the BF output signal of the microphone array 2 is SS from the BF output signal of one of the microphone arrays 1, the target area sound component overlapping in FIG. 18 (e) is deleted. At this time, since the components of the non-target area sound 1 and the non-target area sound 2 do not overlap, only the non-target area sound 1 can be extracted. By further SS processing the extracted component of the non-target area sound 1 from the BF output signal of the microphone array 1, the target area sound can be finally extracted.

  In order to extract the target area sound by this method, it is assumed that the power of the target area sound component is included in each BF output signal with the same magnitude. However, normally, the power of the target area sound component of each BF output signal varies depending on the difference in distance between the target area and each of the microphone arrays 1 and 2 and the difference in gain between the microphone arrays 1 and 2.

  Therefore, in the first and second embodiments, first, the ratio of the amplitude spectrum is obtained between the BF output signals, and the mode value of the ratio is calculated. As described above, since the target area sound component is included in all BF output signals in the same ratio and distribution, all the ratios are the same at the frequency of the target area sound component. On the contrary, the non-target area sound component differs in each BF output signal, so that the ratio varies. From this characteristic, if the mode value is obtained for the ratios for all frequencies, the value is used as it is as a coefficient for correcting the power of the target area sound component of each BF output signal to be equal. FIG. 19 is an explanatory diagram showing the ratio of the amplitude spectrum between the BF output signals of the microphone arrays 1 and 2 as a histogram. FIG. 19A shows a case where the microphone arrays 1 and 2 are arranged equidistant from the target area. Since the distance from the target area is the same, the powers of the input target area sound components are substantially equal, and the mode of the ratio is a value close to 1. FIG. 19B shows a case where the microphone array 2 is closer to the target area than the microphone array 1. It can be seen that the mode value of the ratio is smaller than 1 because the power of the target area sound component is larger in the microphone array 2 closer to the target area. The power correction coefficient can also be obtained by calculating the median value as an approximation of the mode value. As can be seen from FIGS. 19A and 19B, since the distribution of the ratio is unimodal, the median value is close to the mode value. As described above, in the first and second embodiments, the mode value or median value of the ratio of the amplitude spectrum between the BF output signals is calculated as the power correction coefficient. After correcting the power of all target area sound components included in each BF output signal using the calculated power correction coefficient, the target area sound is extracted by the above method.

  In the third and fourth embodiments, first, a value that minimizes the square of the power difference of each BF output signal is calculated, and this minimum value is used as the power correction coefficient of the target area sound component. Since the distribution of the target area sound component of each BF output signal becomes the same when normalized, it is considered that the power of the target area sound component matches when the difference in power after each BF is minimized. . After correcting the power of all target area sound components included in each BF output signal using the calculated power correction coefficient, the target area sound is extracted by the above method.

(B) First Embodiment Hereinafter, a first embodiment of a sound collection device and a program according to the present invention will be described in detail with reference to the drawings.

(B-1) Configuration of First Embodiment FIG. 1 is a block diagram illustrating a configuration of a sound collection device according to the first embodiment. The processing configuration after being converted into a digital signal in the sound collection device 10A can be realized by a CPU and a program executed by the CPU, but can be functionally represented in FIG.

  In FIG. 1, a sound collection device 10A according to the first embodiment includes a microphone array 1, a microphone array 2, a data input unit 3, a directivity forming unit 4, an inter-microphone array delay correction unit 5, a target area sound power correction coefficient. A calculation unit 6 and a target area sound extraction unit 7 are provided.

  The microphone array 1 is arranged at a location where the target area can be directed in the space where the target area exists. The microphone array 1 is composed of two or more microphones, collects sound by each microphone, and inputs an acoustic signal to the data input unit 3 of the sound collecting device 10A.

  The microphone array 2 has the same configuration as the microphone array 1 and is arranged at a different location from the microphone array 1.

  The arrangement of the plurality of microphones constituting the microphone arrays 1 and 2 may be any arrangement that can execute the BF, and may be, for example, one horizontal row, one vertical row, a cross shape, or a lattice shape. The number of microphone arrays may be two or more.

  The data input unit 3 converts an acoustic signal collected by the microphone arrays 1 and 2 from an analog signal to a digital signal (data).

  The directivity forming unit 4 forms a directional beam directed to a target area by BF based on output signals from all the microphone arrays 1 and 2. Various methods such as an addition type delay sum method and a subtraction type SS can be applied to the BF. Further, the directivity forming unit 4 can change the intensity of directivity according to the range of the target area as a target.

  The inter-microphone array delay correction unit 5 is generated by the difference in distance between the target area and each microphone array so that the target area sound reaches all the microphone arrays simultaneously in the output after BF of each microphone array 1 and 2. This is to correct the delay.

  The target area sound power correction coefficient calculation unit 6 calculates a correction coefficient for making the powers of the target area sound components included in the data after each BF all the same.

  The target area sound extraction unit 7 extracts each BF output data corrected by the correction coefficient calculated by the target area sound power correction coefficient calculation unit 6 and extracts a non-target area sound existing in the target area direction. Furthermore, the target area sound extraction unit 7 extracts and outputs the target area sound by performing SS on the extracted non-target area sound from each BF output data.

FIG. 2 is a block diagram showing the configuration of the target area sound extraction unit 7. Here, the output data after BF of the microphone arrays 1 and 2 are X ₁ (n) and X ₂ (n), and the power correction coefficients for each BF output data are α ₁ (n) and α ₂ (n). . Further, a non-target area sound component existing in the target area direction viewed from the microphone array 1 is N ₁ (n), and a non-target area sound component existing in the target area direction viewed from the microphone array 2 is N ₂ (n). .

In this case, the target area sound extraction unit 7 performs SS by multiplying the BF output data X ₂ of the microphone array 2 by the power correction coefficient α ₁ (n), and is included in the BF output data X ₁ (n) of the microphone array 1. The non-target area sound component N ₁ (n) in the target area direction is extracted. Furthermore, the target area sound extraction unit 7 SSs N ₁ (n) for the BF output data X ₁ (n) of the microphone array 1 and extracts the target area sound component Y ₁ (n).

Similarly for the target area sound component Y ₂ (n), the target area sound extraction unit 7 performs SS by multiplying the BF output data X ₁ of the microphone array 1 by the power correction coefficient α ₂ (n), and performs microphone array 2. extracting the BF output data _X 2 (n) non-target area sound object area direction included in the component _n 2 (n). Further, the target area sound extraction unit 7 SS SSs N ₂ (n) for the BF output data X ₂ (n) of the microphone array 2 and extracts the target area sound component Y ₂ (n).

(B-2) Operation of the First Embodiment Next, processing of the sound collection device 10A according to the first embodiment will be described. FIG. 3 is a flowchart showing the processing of the sound collection device 10A according to the first embodiment.

  Sounds from various sound sources existing in a space with a target area are picked up by the microphones constituting the microphone arrays 1 and 2, and the acoustic signals acquired by the microphone arrays 1 and 2 are input to the data input unit 3, The signal is converted into a digital signal (S1).

  The directivity forming unit 4 forms directivity in the direction of the target area by BF with respect to the outputs of all the microphone arrays 1 and 2 (S2).

  The inter-microphone array delay correcting unit 5 corrects a delay caused by a difference in distance between the target area and each microphone array (S3).

  The inter-microphone array delay correcting unit 5 first calculates the arrival time of the target area sound to each microphone array from the position of the target area and the position of the microphone array. Then, with reference to the microphone array arranged farthest from the target area, a delay is added so that the target area sound reaches all the microphone arrays simultaneously. This operation by the inter-microphone array delay correction unit 5 makes it possible to simultaneously handle output data of the microphone arrays 1 and 2 that are arbitrarily arranged.

  The target area sound power correction coefficient calculation unit 6 calculates a target area sound power correction coefficient for making all the powers of the target area sound components included in the output data after BF from the microphone arrays 1 and 2 equal ( S4).

In order to obtain the power correction coefficient, the target area sound power correction coefficient calculation unit 6 first obtains the ratio of the amplitude spectrum between the BF output data X ₁ and X ₂ . At this time, if the directivity forming unit 4 performs BF in the time domain, each BF output data is converted to the frequency domain. Then, the target area sound power correction coefficient calculation unit 6 calculates the mode value from the obtained ratio, and sets the value as the power correction coefficient (Equations (7) and (8)). Alternatively, the target area sound power correction coefficient calculation unit 6 can also calculate the median of the ratios and use it as the power correction coefficient (Equations (9) and (10)).

Here, X _1k (n), X _2k (n) are output data after BF of the microphone arrays 1 and 2, N is the total number of frequency bins, k is the frequency, α ₁ (n), α ₂ (n) are It is a power correction coefficient for each BF output. The target area sound power correction coefficient calculation unit 6 does not have to obtain all the power correction coefficients, and when one is obtained, the other may be the reciprocal thereof. That is, when the target area sound power correction coefficient calculation unit 6 calculates α ₁ (n), α ₂ (n) = 1 / α ₁ (n) can be set for the other α ₂ (n). .

The target area sound extraction unit 7 SS each BF output data corrected by the power correction coefficient calculated by the target area sound power correction coefficient calculation unit 6, and extracts the non-target area sound existing in the target area direction (S5). . Furthermore, the target area sound extraction unit 7 extracts the target area sound by performing SS on the extracted non-target area sound from the output of each BF (S6). In order to extract the non-target area sound N ₁ (n) existing in the direction of the target area viewed from the microphone array 1, the microphone array 2 is obtained from the BF output X ₁ (n) of the microphone array 1 as shown in the equation (11). SS is obtained by multiplying the BF output X ₂ (n) by the power correction coefficient α ₂ . Similarly, the non-target area sound N ₂ (n) existing in the target area direction viewed from the microphone array 2 is extracted according to the equation (12).

Thereafter, the target area sound extraction unit 7 extracts the target area sound by SS of the non-target area sound from each BF output data according to the equations (13) and (14). In equations (13) and (14), γ ₁ (n) and γ ₂ (n) are coefficients for changing the strength at the time of SS.

  The following experiment was conducted to show the effect of the first embodiment.

  FIG. 4 shows the arrangement of the microphone arrays 1 and 2 and the sound source. The area to be collected was a square with a side of 2 m, and the area to be collected was divided into four sections. One target area sound source and two non-target area sound sources were arranged in three areas. The sound sources were all human voices, and these were reproduced simultaneously at approximately the same volume and recorded with a microphone array. Two microphone arrays are used and arranged so that the target area sound source and the non-target area sound source overlap each other in the front direction.

  In the arrangement pattern 1 of FIG. 4A, the target area sound source is arranged in front of the non-target area sound source for each of the microphone arrays 1 and 2. 4B, the target area sound source is placed behind the non-target area sound source. Each of the microphone arrays 1 and 2 is composed of the same number of microphones, and the number of microphones used in one microphone array is two. All the microphone intervals were 3 cm. Using recorded data, the amount of suppression of non-target area sound of BF by the method of the present invention and the microphone array alone was compared by computer simulation. The existing subtractive BF (see Non-Patent Document 2) was used as the BF method.

  The extent to which non-target area sounds can be suppressed was evaluated using Noise Reduction Rate (NRR).

  FIG. 5 shows the suppression amount of the non-target area sound in each arrangement pattern. In the arrangement pattern 1 of FIG. 5A, the method of the present invention has a large suppression amount of about 3 dB non-target area sound compared to the BF of the microphone array alone. Also in the arrangement pattern 2 of FIG. 5 (B), the method of the present invention can suppress approximately 3.6 dB larger than the BF of the microphone array alone. Thus, according to the present embodiment, it is possible to suppress the non-target area sound existing in the target area direction.

(B-3) Effect of First Embodiment According to the first embodiment, each microphone is extracted in order to extract a target area sound by correcting the magnitude of the target area sound component included in the output of each BF. Without adjusting the position of the array, it is possible to emphasize only the past target area even in a situation where the target area is surrounded by non-target area sound sources. That is, only the target area sound can be emphasized by arranging a plurality of microphone arrays once in different directions.

  Further, according to the first embodiment, the directivity formed by the directivity forming unit can be changed, so that it is possible to easily change the target area without changing the positions of a plurality of microphone arrays. can do.

  Furthermore, according to the first embodiment, since the subtractive BF can be used, one microphone array can be configured with a small number of microphones.

(C) Second Embodiment In the first embodiment, data from which target area sounds are extracted is output by the number of microphone arrays. When using the area sound pickup device, a situation is assumed in which one data is finally selected and output from these data.

  Accordingly, in the second embodiment, the distance between the target area and each microphone array and the SN ratio of the target area sound and the non-target area sound are used as feature amounts to select data in which the target area sound is most emphasized. An output data selection unit is provided.

  Hereinafter, a second embodiment of a sound collecting device and a program according to the present invention will be described with reference to the drawings.

(C-1) Configuration of Second Embodiment FIG. 6 is a block diagram showing a configuration of a sound collection device according to the second embodiment. In FIG. 6, the sound collection device 10B according to the second embodiment includes a microphone array 1, a microphone array 2, a data input unit 3, a directivity forming unit 4, an inter-microphone array delay correction unit 5, and a target area sound power correction coefficient. A calculation unit 6, a target area sound extraction unit 7, and an output data selection unit 8 are provided.

  The sound collection device 10B according to the second embodiment includes an output data selection unit 8 subsequent to the target area sound extraction unit 7 in addition to the components described in the first embodiment.

  The output data selection unit 8 uses the distance or S / N ratio between the target area and each of the microphone arrays 1 and 2 from the output of the target area sound extraction unit 7 as an index of the target area sound enhancement, and the target area sound is most emphasized. The selected data is selected.

(C-2) Operation of the Second Embodiment Next, processing of the sound collection device 10B according to the second embodiment will be described. FIG. 7 is a flowchart showing the processing of the sound collection device 10B according to the second embodiment. In FIG. 7, the process of S1-S6 is the same as the process of S1-S6 of FIG.

  The output data selection unit 8 selects data in which the target area sound is most emphasized from the plurality of data extracted by the target area sound extraction unit 7 (S7).

The output data selection unit 8 selects the target area sound enhancement index as the distance between the target area and the microphone arrays 1 and 2 and outputs the closest distance as the output data. Alternatively, the SN ratio (in this case, Y _i (n) / N _i (n)) is used as an index for the target area sound enhancement, and the output data selection unit 8 selects the one with the best SN ratio. Furthermore, the output data selection unit 8 can also select a combination of these indices.

(C-3) Effects of the Second Embodiment According to the second embodiment, in addition to the effects of the first embodiment, the target area is the most out of the data in which a plurality of target area sounds are extracted. Data with enhanced sound can be selected and output.

(D) Modified Embodiments of First and Second Embodiments In the first and second embodiments described above, the case where there are two microphone arrays is shown, but there may be three or more microphone arrays. In this case, the equations (7) to (14) can be expanded as follows. Here, M is the total number of microphone arrays.

(E) Third Embodiment Hereinafter, a third embodiment of the sound collection device and the program according to the present invention will be described in detail with reference to the drawings.

(E-1) Configuration of Third Embodiment FIG. 8 is a block diagram showing a configuration of a sound collection device according to the third embodiment. The processing configuration after being converted into a digital signal in the sound collecting device 10C can be realized by a CPU and a program executed by the CPU, but can be functionally represented by FIG.

  The sound collection device 10C according to the third embodiment includes a microphone array 1, a microphone array 2, a data input unit 3, a directivity forming unit 4, an inter-microphone array delay correction unit 5, a target area sound power correction coefficient calculation unit 9, A target area sound extraction unit 7 is provided.

  The microphone array 1 is arranged at a location where the target area can be directed in the space where the target area exists. The microphone array 1 is composed of two or more microphones, collects sound by each microphone, and inputs an acoustic signal to the data input unit 3 of the sound collecting device 10A.

  The microphone array 2 has the same configuration as the microphone array 1 and is arranged at a different location from the microphone array 1.

  The arrangement of the plurality of microphones constituting the microphone arrays 1 and 2 may be any arrangement that can execute the BF, and may be, for example, one horizontal row, one vertical row, a cross shape, or a lattice shape. The number of microphone arrays may be two or more.

  The data input unit 3 converts an acoustic signal collected by the microphone arrays 1 and 2 from an analog signal to a digital signal (data).

  The directivity forming unit 4 forms a directional beam directed to a target area by BF based on output signals from all the microphone arrays 1 and 2. Various methods such as an addition type delay sum method and a subtraction type SS can be applied to the BF. Further, the directivity forming unit 4 can change the intensity of directivity according to the range of the target area as a target.

  The inter-microphone array delay correction unit 5 is generated by the difference in distance between the target area and each microphone array so that the target area sound reaches all the microphone arrays simultaneously in the output after BF of each microphone array 1 and 2. This is to correct the delay.

  The target area sound power correction coefficient calculation unit 9 calculates a power correction coefficient for making all the powers of the target area sound components included in the data after each BF the same. That is, the target area sound power correction coefficient calculation unit 9 calculates a coefficient that minimizes the square of the power difference between the BF outputs of the microphone arrays 1 and 2 and sets this as the power correction coefficient.

  The target area sound extraction unit 7 extracts each BF output data corrected by the power correction coefficient calculated by the target area sound power correction coefficient calculation unit 9 and extracts a non-target area sound existing in the direction of the target area. Furthermore, the target area sound extraction unit 7 extracts and outputs the target area sound by performing SS on the extracted non-target area sound from each BF output data.

FIG. 2 is a block diagram showing the configuration of the target area sound extraction unit 7. Here, the output data after BF of the microphone arrays 1 and 2 are X ₁ (n) and X ₂ (n), and the power correction coefficients for each BF output data are α ₁ (n) and α ₂ (n). . Further, a non-target area sound component existing in the target area direction viewed from the microphone array 1 is N ₁ (n), and a non-target area sound component existing in the target area direction viewed from the microphone array 2 is N ₂ (n). .

In this case, the target area sound extraction unit 7 performs SS by multiplying the BF output data X ₂ of the microphone array 2 by the power correction coefficient α ₁ (n), and is included in the BF output data X ₁ (n) of the microphone array 1. The non-target area sound component N ₁ (n) in the target area direction is extracted. Furthermore, the target area sound extraction unit 7 SSs N ₁ (n) for the BF output data X ₁ (n) of the microphone array 1 and extracts the target area sound component Y ₁ (n).

Similarly for the target area sound component Y ₂ (n), the target area sound extraction unit 7 performs SS by multiplying the BF output data X ₁ of the microphone array 1 by the power correction coefficient α ₂ (n), and performs microphone array 2. extracting the BF output data _X 2 (n) non-target area sound object area direction included in the component _n 2 (n). Further, the target area sound extraction unit 7 SS SSs N ₂ (n) for the BF output data X ₂ (n) of the microphone array 2 and extracts the target area sound component Y ₂ (n).

(E-2) Operation of the Third Embodiment Next, the operation of the sound collection device according to the embodiment will be described. FIG. 9 is a flowchart showing processing of the sound collecting device 10C according to the third embodiment.

  Sounds from various sound sources existing in a space with a target area are picked up by the microphones constituting the microphone arrays 1 and 2, and the acoustic signals acquired by the microphone arrays 1 and 2 are input to the data input unit 3, The signal is converted into a digital signal (S1).

  The directivity forming unit 4 forms directivity in the direction of the target area by BF with respect to the outputs of all the microphone arrays 1 and 2 (S2).

  The inter-microphone array delay correcting unit 5 corrects a delay caused by a difference in distance between the target area and each microphone array (S3).

  The inter-microphone array delay correcting unit 5 first calculates the arrival time of the target area sound to each microphone array from the position of the target area and the position of the microphone array. Then, with reference to the microphone array disposed farthest from the target area, a delay is added so that the target area sound reaches all the microphone arrays simultaneously. This operation by the inter-microphone array delay correction unit 5 makes it possible to simultaneously handle output data of the microphone arrays 1 and 2 that are arbitrarily arranged.

  The target area sound power correction coefficient calculation unit 9 calculates a power correction coefficient for making all the powers of the target area sound components included in the BF output data from the microphone arrays 1 and 2 the same. At this time, the target area sound power correction coefficient calculation unit 9 updates the target area sound power correction coefficient so that the difference between the outputs after BF of the microphone arrays 1 and 2 becomes the smallest (S14).

FIG. 10 is a block diagram showing the configuration of the target area sound power correction coefficient calculation unit 9. The target area sound power correction coefficient calculation unit 9 squares the power difference between the outputs after BF of the two microphone arrays 1 and 2 according to the expressions (19) and (20) in order to obtain the power correction coefficient. The value of the evaluation function is calculated. At this time, if the directivity forming unit 4 performs BF in the time domain, the target area sound power correction coefficient calculation unit 9 converts the post-BF output data into the frequency domain.

Here, X _1k (n) and X _2k (n) are output data after BF of the microphone arrays 1 and 2, N is the total number of frequency bins, k is the frequency, and α ₁ (n) and α ₂ (n) are each This is a power correction coefficient for the BF output.

Destination area sound power correction coefficient calculator 9, the evaluation function _J 1 _(n), such that the value of _J 2 (n) is the smallest, (21), in accordance with (22), the power correction coefficient alpha ₁ ( n), α ₂ (n) is updated. ρ is a learning coefficient. In order to reduce the amount of calculation, the target area sound power correction coefficient calculation unit 9 may obtain one power correction coefficient first, and the other power correction coefficient may be the reciprocal of one power correction coefficient.

  The target area sound extraction unit 7 SS SS each BF output data corrected by the correction coefficient calculated by the target area sound power correction coefficient calculation unit 6 and extracts a non-target area sound existing in the target area direction (S5). Furthermore, the target area sound extraction unit 7 extracts the target area sound by performing SS on the extracted non-target area sound from the output data of each BF (S6).

In order to extract the non-target area sound N ₁ (n) existing in the direction of the target area viewed from the microphone array 1, the microphone array 2 is obtained from the BF output X ₁ (n) of the microphone array 1 as shown in the equation (11). SS is obtained by multiplying the BF output X ₂ (n) by the power correction coefficient α ₂ . Similarly, the non-target area sound N ₂ (n) existing in the target area direction viewed from the microphone array 2 is extracted according to the equation (12).

Thereafter, the target area sound extraction unit 7 extracts the target area sound by SS of the non-target area sound from each BF output data according to the equations (13) and (14). In equations (13) and (14), γ ₁ (n) and γ ₂ (n) are coefficients for changing the strength at the time of SS.

  The following experiment was conducted to show the effect of the third embodiment.

  FIG. 11 shows the arrangement of the microphone arrays 1 and 2 and the sound source. The area to be collected was a square with a side of 2 m, and the area to be collected was divided into four sections. One target area sound source and two non-target area sound sources were arranged in three areas. The sound sources were all human voices, and these were reproduced simultaneously at approximately the same volume and recorded with a microphone array. Two microphone arrays are used and arranged so that the target area sound source and the non-target area sound source overlap each other in the front direction.

  In the arrangement pattern 1 of FIG. 11A, the target area sound source is arranged in front of the non-target area sound source for each of the microphone arrays 1 and 2. Further, in the arrangement pattern 2 of FIG. 11B, the target area sound source is arranged behind the non-target area sound source. Each of the microphone arrays 1 and 2 is composed of the same number of microphones, and the number of microphones used in one microphone array is two. All the microphone intervals were 3 cm. Using recorded data, the amount of suppression of non-target area sound of BF by the method of the present invention and the microphone array alone was compared by computer simulation. The existing subtractive BF (see Non-Patent Document 2) was used as the BF method.

  The extent to which non-target area sounds can be suppressed was evaluated using Noise Reduction Rate (NRR).

  FIG. 12 shows the suppression amount of the non-target area sound in each arrangement pattern. In the arrangement pattern 1 of FIG. 12A, the method of the present invention has a large suppression amount of about 4 dB non-target area sound compared to the BF of the microphone array alone. Also in the arrangement pattern 2 of FIG. 12B, the method of the present invention can suppress approximately 5.5 dB larger than the BF of the microphone array alone. Thus, according to the third embodiment, it is possible to suppress the non-target area sound existing in the target area direction.

(E-3) Effects of the Third Embodiment According to the third embodiment, each microphone is used to extract the target area sound by correcting the magnitude of the target area sound component included in the output of each BF. Without adjusting the position of the array, it is possible to emphasize only the past target area even in a situation where the target area is surrounded by non-target area sound sources. That is, only the target area sound can be emphasized by arranging a plurality of microphone arrays once in different directions.

  Further, according to the third embodiment, since the directivity formed by the directivity forming unit can be changed, it is possible to easily change the target area without changing the positions of a plurality of microphone arrays. can do.

  Furthermore, according to the third embodiment, since the subtractive BF can be used, one microphone array can be configured with a small number of microphones.

(F) Fourth Embodiment In the third embodiment, data from which target area sounds are extracted is output by the number of microphone arrays. When using the area sound pickup device, a situation is assumed in which one data is finally selected and output from these data. Accordingly, in the fourth embodiment, the distance between the target area and each microphone array and the SN ratio of the target area sound and the non-target area sound are used as feature amounts to select data in which the target area sound is most emphasized. An output data selection unit is provided.

  Hereinafter, a fourth embodiment of a sound collecting apparatus and a program according to the present invention will be described with reference to the drawings.

(F-1) Configuration of Fourth Embodiment FIG. 13 is a block diagram illustrating a configuration of a sound collection device according to the fourth embodiment. In FIG. 13, a sound collection device 10D according to the fourth embodiment includes a microphone array 1, a microphone array 2, a data input unit 3, a directivity forming unit 4, an inter-microphone array delay correction unit 5, a target area sound power correction coefficient. A calculation unit 9, a target area sound extraction unit 7, and an output data selection unit 8 are provided.

  The sound collection device 10D according to the fourth embodiment includes an output data selection unit 8 at the subsequent stage of the target area sound extraction unit 7 in addition to the components described in the third embodiment.

  The output data selection unit 8 uses the distance or S / N ratio between the target area and each of the microphone arrays 1 and 2 from the output of the target area sound extraction unit 7 as an index of the target area sound enhancement, and the target area sound is most emphasized. The selected data is selected.

(F-2) Operation of Fourth Embodiment Next, processing of the sound collection device 10D according to the fourth embodiment will be described. FIG. 14 is a flowchart showing the processing of the sound collection device 10D according to the fourth embodiment. In FIG. 14, the processes of S1, S2, S3, S14, S5, and S6 are the same as the processes of S1, S2, S3, S14, S5, and S6 of FIG.

  The output data selection unit 8 selects data in which the target area sound is most emphasized from the plurality of data extracted by the target area sound extraction unit 7 (S7).

The output data selection unit 8 selects the target area sound enhancement index as the distance between the target area and the microphone arrays 1 and 2 and outputs the closest distance as the output data. Alternatively, the SN ratio (in this case, Y _i (n) / N _i (n)) is used as an index for the target area sound enhancement, and the output data selection unit 8 selects the one with the best SN ratio. Furthermore, the output data selection unit 8 can also select a combination of these indices.

(F-3) Effect of the fourth embodiment According to the fourth embodiment, in addition to the effect of the third embodiment, the target area is the most out of the data from which a plurality of target area sounds are extracted. Data with enhanced sound can be selected and output.

(G) Modified Embodiment of Third and Fourth Embodiments In the third and fourth embodiments, two microphone arrays are shown, but there may be three or more microphone arrays. In this case, the equations (19) to (22) can be expanded as follows. Here, M is the total number of microphone arrays.

(H) Other Embodiments In each of the above embodiments, the acoustic signal obtained by the microphone array is shown in real time. However, the acoustic signal obtained by the microphone array is stored in a storage medium. Thereafter, the enhancement signal of the target area sound may be obtained by reading out from the storage medium and processing. When the storage medium is used in this way, the place where the microphone array is set and the place where the enhancement processing is performed may be separated from each other. Similarly, when processing in real time, the place where the microphone array is set and the place where the emphasis processing is performed may be separated from each other, and the signal may be supplied to a remote place by communication. The case where the above storage medium or communication is used is also included in the concept of the “sound collecting device” of the present invention.

10A, 10B, 10C, 10D ... sound collecting device,
1 ... microphone array, 2 ... macrophone array, 3 ... data input unit,
4 ... directivity forming part, 5 ... delay correction part between microphone arrays,
6 and 9: a target area sound power correction coefficient calculation unit, 7: a target area sound extraction unit.

Claims (5)

Multiple microphone arrays,
A directivity forming unit that forms directivity in the direction of the target area by a beamformer with respect to the output of each microphone array,
Inter-microphone array delay correction that corrects the delay caused by the difference in the distance between the target area and each microphone array so that the target area sound arrives at all microphone arrays simultaneously at the output after the beam former of each microphone array. And
In order to make the power of the target area sound included in the beamformer output of each microphone array all the same, the mode value or median value of the ratio of the amplitude spectrum between the beamformer outputs of each microphone array is calculated. A target area sound power correction coefficient calculation unit as a correction coefficient;
Using the correction coefficient calculated by the target area sound power correction coefficient calculating unit, correcting the beamformer output of each microphone array, extracting each non-target area sound existing in the target area direction by subtracting the spectrum, A sound collection apparatus comprising: a target area sound extraction unit that extracts a target area sound by performing spectral subtraction on the extracted non-target area sound from the beamformer output of each microphone array. Multiple microphone arrays,
A directivity forming unit that forms directivity in the direction of the target area by a beamformer with respect to the output of each microphone array,
Inter-microphone array delay correction that corrects the delay caused by the difference in the distance between the target area and each microphone array so that the target area sound arrives at all microphone arrays simultaneously at the output after the beam former of each microphone array. And
In order to make all the power of the target area sound included in the beamformer output of each microphone array the same magnitude, a coefficient that minimizes the square of the difference in power of the beamformer output of each microphone array is calculated, A target area sound power correction coefficient calculation unit as a correction coefficient;
Using the correction coefficient calculated by the target area sound power correction coefficient calculating unit, correcting the beamformer output of each microphone array, extracting each non-target area sound existing in the target area direction by subtracting the spectrum, A sound collection apparatus comprising: a target area sound extraction unit that extracts a target area sound by performing spectral subtraction on the extracted non-target area sound from the beamformer output of each microphone array.   An output data selection unit for selecting the data in which the target area sound is most emphasized from the outputs of the target area sound extraction unit, using the distance or SN ratio between the target area and each of the microphone arrays as an index of the target area sound enhancement. The sound collecting device according to claim 1, further comprising: A computer that receives signals from multiple microphone arrays
A directivity forming unit that forms directivity in the direction of the target area by a beamformer with respect to the output of each microphone array,
Inter-microphone array delay correction that corrects the delay caused by the difference in the distance between the target area and each microphone array so that the target area sound arrives at all microphone arrays simultaneously at the output after the beam former of each microphone array. Part,
In order to make the power of the target area sound included in the beamformer output of each microphone array all the same, the mode value or median value of the ratio of the amplitude spectrum between the beamformer outputs of each microphone array is calculated. , A target area sound power correction coefficient calculation unit as a correction coefficient,
Using the correction coefficient calculated by the target area sound power correction coefficient calculating unit, correcting the beamformer output of each microphone array, extracting each non-target area sound existing in the target area direction by subtracting the spectrum, A sound collection program that functions as a target area sound extraction unit that extracts a target area sound by performing spectral subtraction on the extracted non-target area sound from the beamformer output of each microphone array. A computer that receives signals from multiple microphone arrays
A directivity forming unit that forms directivity in the direction of the target area by a beamformer with respect to the output of each microphone array,
Inter-microphone array delay correction that corrects the delay caused by the difference in the distance between the target area and each microphone array so that the target area sound arrives at all microphone arrays simultaneously at the output after the beam former of each microphone array. Part,
In order to make all the power of the target area sound included in the beamformer output of each microphone array the same magnitude, a coefficient that minimizes the square of the difference in power of the beamformer output of each microphone array is calculated, A target area sound power correction coefficient calculation unit as a correction coefficient,
Using the correction coefficient calculated by the target area sound power correction coefficient calculating unit, correcting the beamformer output of each microphone array, extracting each non-target area sound existing in the target area direction by subtracting the spectrum, A sound collection program that functions as a target area sound extraction unit that extracts a target area sound by performing spectral subtraction on the extracted non-target area sound from the beamformer output of each microphone array.

Images