JP6885483B1 - Sound collecting device, sound collecting program and sound collecting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound collecting device, a sound collecting program and a sound collecting method for collecting only a target area sound regardless of the arrival direction of a non-target area sound. SOLUTION: A sound collecting device acquires a target direction signal by a beamformer for an input signal of a plurality of microphone arrays, sets three or more microphone array sets formed by a combination of two microphone arrays, and sets each microphone. Each microphone array set is used based on the result of comparing the target area sound extracted by the spectrum subtraction process based on the target direction signal of the array set and the target area sound extracted by each microphone array set. One output signal is generated and output from the target area sound extracted by the above. [Selection diagram] Fig. 1

Description

The present invention relates to a sound collecting device, a sound collecting program, and a sound collecting method, and can be applied to, for example, a system that emphasizes sound in a specific area and suppresses sound in other areas.

There is a beam former (hereinafter also referred to as "BF") using a microphone array as a technique for separating and collecting only sound in a specific direction in an environment where a plurality of sound sources exist. BF is a technique for forming directivity by utilizing the time difference between signals arriving at each microphone (see Non-Patent Document 1).

Conventionally, BF is roughly classified into two types, an addition type and a subtraction type. In particular, the subtraction type BF has an advantage that directivity can be formed with a smaller number of microphones than the addition type immediate.

FIG. 11 is a block diagram showing a configuration related to the subtraction type BF200 when the number of microphones M is two.

FIG. 12 is an explanatory diagram showing an example of a directional filter formed by a subtraction type BF200 using two microphones M1 and M2.

The subtraction type BF200 first calculates the time difference between the signals that the sound existing in the target direction (hereinafter referred to as "target sound") arrives at the microphones M1 and M2 by the delay device 210, and adds a delay to the object. Match the phase of the sound. The above time difference can be calculated by the following equation (1).

Here, d is the distance between the microphones M1 and M2, c is the speed of sound, and τ _i is the delay amount. Further, θ _L is an angle from the vertical direction to the target direction with respect to the straight line connecting the microphones M (M1, M2).

Further, when the blind spot exists in the direction of the microphone M1 with respect to the center of the microphones M1 and M2, the delay device 210 performs delay processing on _{the input signal x 1 (t) of the microphone M1.} After that, in the subtraction type BF200, processing (subtraction processing) is performed according to the following equation (2).

The processing of the subtraction type BF200 can be performed in the same manner in the frequency domain, in which case the equation (2) is modified as follows (3).

Here, when θ _L = ± π / 2, the directivity formed by the subtraction type BF200 is a cardioid type unidirectionality as shown in FIG. 12A. Further, in the case of "θ _L = 0, π", the directivity formed by the subtraction type BF200 is a figure eight bidirectionality as shown in FIG. 12B.

Hereinafter, a filter that forms unidirectionality from an input signal is referred to as a "unidirectional filter", and a filter that forms bidirectionality is referred to as a bidirectional filter.

Further, in the subtractor 220, a strong directivity can be formed in a bidirectional blind spot by using a spectral subtraction method (hereinafter, also simply referred to as “SS”). The directivity by SS is formed in all frequencies or a designated frequency band according to the following equation (4).

In the following equation (4), and using the input signal X ₁ microphone M1, but it is possible to obtain the same effect input signal X ₂ microphones M2. Here, β is a coefficient for adjusting the intensity of SS. Further, in the subtractor 220, when the value becomes negative at the time of subtraction, a flooring process is performed in which 0 or the original value is replaced with a smaller value. In the subtraction type BF200 processing method as described above, sounds existing in directions other than the target direction (hereinafter referred to as "non-purpose sounds") are extracted due to the bidirectional characteristics, and the amplitude spectrum of the extracted non-purpose sounds is input. The target sound can be emphasized by subtracting it from the amplitude spectrum of the signal.

If you want to collect only the sound that exists in a specific area (hereinafter referred to as "target area sound"), simply using the subtraction type BF will result in the sound of the sound source that exists around that area (hereinafter, "non-" There is a possibility that the sound will be picked up (called the target area sound). Therefore, in Patent Document 1, a method of collecting sound in a target area by using a plurality of microphone arrays, directing directivity from different directions to the target area, and intersecting the directivity in the target area (hereinafter, "area"). It is called "sound collection"). In the area sound collection, first, the ratio of the amplitude spectrum of the target area sound included in the BF output of each microphone array is estimated, and this is used as the correction coefficient.

For example, when two microphone arrays are used, the correction coefficient of the target area sound amplitude spectrum is determined by the combination of the following equations (5) and (6) or the combination of the following equations (7) and (8). Can be calculated. Here, Y _1k (n) is the amplitude spectrum of the BF output of the first microphone array, Y _2k (n) is the amplitude spectrum of the BF output of the second microphone array, and N is the total number of frequency bins. Yes, k is the frequency. Further, here, α ₁ (n) and α ₂ (n) are amplitude spectrum correction coefficients for each BF output. Further, here, mode represents the mode and median represents the median.

Through the above processing, the subtractor 220 obtains the amplitude spectrum correction coefficients α ₁ (n) and α ₂ (n), corrects each BF output with the obtained correction coefficients, and SSs the amplitude spectrum correction coefficients to exist in the target area direction. Extract non-purpose area sounds. Further, the subtractor 220 can extract the target area sound by SSing the extracted non-purpose area sound from the output of each BF.

_{The subtraction type BF200 extracts the non-purpose area sound N 1} (n) existing in the direction of the target area as viewed from the first microphone array, for example, as shown in Eq. (9), the BF output of the first microphone array. The SS is obtained by multiplying _{the BF output Y 2} (n) of the second microphone array from _{Y 1} (n) by the amplitude spectrum correction coefficient α _{2. Similarly, the subtraction type BF200 extracts the non-purpose area sound N 2} (n) existing in the direction of the target area as viewed from the second microphone array according to the following equation (10).

After that, the subtraction type BF200 extracts the target area sound by SSing the non-target area sound from each BF output according to the following equation (11) or (12). The following equation (11) shows the process when the target area sound is extracted with reference to the first microphone array. Further, the following equation (12) shows the process when the target area sound is extracted with reference to the second microphone array. Here, γ ₁ (n) and γ ₂ (n) are coefficients for changing the intensity at the time of SS.

In the conventional area sound collection processing, in order to extract the target area sound, SS, which is a non-linear processing in the equations (4), (11) and (12), is performed, so that the musical noise is generated in a high noise environment. An unpleasant noise called may occur.

Therefore, in the technique described in Patent Document 2, the section in which the target area sound exists and the section in which the target area sound does not exist are determined in the input signal, and the area pickled sound is output in the section in which the target area sound does not exist. By not doing so, abnormal sounds such as musical noise are suppressed.

In the technique described in Patent Document 2, in order to determine whether or not the target area sound exists, first, the input signal and the target area sound are extracted according to the equation (13), and the output (hereinafter referred to as "area sound output"). Calculate the amplitude spectrum ratio R (= area sound output / input signal) between them.

Further, when the sound source exists in the _{target area, the target area sound is commonly included in the input signal X 1} and the area sound output Z ₁ , so that the amplitude spectrum ratio of the target area sound component is close to 1. On the contrary, since the non-purpose area sound component is suppressed in the area sound output, the amplitude spectrum ratio becomes a small value. Since the area sound collection process also performs SS a plurality of times for other background noise components, they are suppressed to some extent without performing a dedicated noise suppression process in advance, and the amplitude spectrum ratio becomes a small value. On the contrary, when the target area sound does not exist, the area sound output contains only weak noise that remains unerased as compared with the input signal, so that the amplitude spectrum ratio becomes a small value in the entire range.

In the technique described in Patent Document 2, if the average value U of the amplitude spectral ratios obtained at each frequency is taken according to the equation (14) due to this feature, a large difference is generated between the presence and absence of the target area sound. It will be. Here, m and n are the upper limit and the lower limit of the processing band (frequency band), respectively, and are set to, for example, 100 Hz to 6 kHz, which sufficiently include voice information.

Then, in the technique described in Patent Document 2, the average power spectral ratio is determined by a preset threshold value, and when it is determined that the target area sound does not exist, there is no sound or an input signal without outputting the area sound output data. Outputs a sound with a reduced gain.

Japanese Unexamined Patent Publication No. 2014-072708 Japanese Unexamined Patent Publication No. 2016-127457

Tadashi Asano, "Acoustic Technology Series 16 Sound Array Signal Processing-Localization, Tracking and Separation of Sound Sources-", edited by Acoustical Society of Japan, Corona Publishing Co., Ltd., February 25, 2011

In the sound collection method described in Patent Document 1, two microphone arrays are installed on the left and right in front of the area where the sound is to be collected, and the sound existing in an arbitrary area can be obtained by adjusting the distance and angle between the microphone arrays. Sound can be picked up.

However, when sound is picked up using the technique described in Patent Document 1, the sound picking area has a wide directivity in the vertical direction. Therefore, if there is a noise source in the vertical direction of the target area sound, the noise is also present. The sound will be picked up. The case where there is a noise source in the vertical direction of the target area sound is, for example, a case where there is a sound source such as a speaker directly above the target area. Further, in the sound collection method described in Patent Document 1, the directivity in the vertical direction has a property of gradually expanding as the distance from the height at which the microphone array is installed increases, so that the sound can be collected even if the speaker is not directly above. It may make a noise.

On the other hand, as in the technique described in Patent Document 2, when determining whether or not the target area sound exists based on the amplitude spectrum ratio of the input and the output, if the volume reproduced from the speaker above the target area is low, the sound is picked up. Not done. However, even if the technique described in Patent Document 2 is applied, there is a risk that the sound of the announcement will be picked up in an environment where a loud announcement (non-purpose area sound) is flowing, such as in a station yard.

In view of the above problems, a sound collecting device, a sound collecting program, and a sound collecting method that collect only the target area sound regardless of the arrival direction of the non-target area sound are desired.

The first sound collecting device of the present invention (1) forms directivity for each of the input signals supplied from the plurality of microphone arrays in the direction of the target area where the target area exists by the beam former, and the microphone array. For each, three or more microphone array sets formed by a combination of the direction forming means for acquiring the target direction signal from the target area direction and (2) two microphone arrays are set, and each microphone array set is set. By subtracting the spectrum of the target direction signal of each of the microphone arrays, the non-purpose area sound existing in the target area direction is extracted, and the extracted non-purpose area sound is spectrumd from any of the target direction signals. Based on the result of comparing the target area sound extraction means for extracting the target area sound by subtraction and the target area sound extracted using each of the microphone array sets, each microphone array set is obtained. It is characterized by having an output means for generating and outputting one output signal from the target area sound extracted by the use.

The second sound pick-up program of the present invention makes the computer (1) direct the input signals supplied from the plurality of microphone arrays toward the target area where the target area exists by the beam former. For each of the microphone arrays, three or more microphone array sets formed by (2) a combination of the two microphone arrays and the directional forming means for acquiring the target direction signal from the destination area direction are set, and the respective microphone arrays are set. For the microphone array set, the non-purpose area sound existing in the target area direction is extracted by subtracting the spectrum of the target direction signal of each of the microphone arrays, and the extracted non-purpose area sound is used in any of the target directions. Based on the result of comparing the target area sound extraction means for extracting the target area sound by subtracting the spectrum from the signal and the target area sound extracted using each of the microphone array sets, each microphone It is characterized in that it functions as an output means for generating and outputting one output signal from the target area sound extracted by using the array set.

According to a third aspect of the present invention, in the sound collecting method performed by the sound collecting device, (1) the sound collecting device has a directivity forming means, a target area sound extracting means, and an output means, and (2) the directivity. The forming means forms a directivity toward the target area where the target area exists by the beam former for each of the input signals supplied from the plurality of microphone arrays, and the target direction from the target area direction for each microphone array. The signal is acquired, and (3) the target area sound extraction means sets three or more microphone array sets formed by a combination of the two microphone arrays, and for each of the microphone array sets, the respective microphone array is set. The non-purpose area sound existing in the target area direction is extracted by subtracting the spectrum of the target direction signal, and the extracted non-purpose area sound is spectrally subtracted from any of the target direction signals to extract the target area sound. (4) The output means uses the respective microphone array sets to extract the target area sound based on the result of comparing the target area sounds extracted using the respective microphone array sets. It is characterized in that one output signal is generated from sound and output.

According to the present invention, only the target area sound can be picked up regardless of the direction of arrival of the non-target area sound.

It is a block diagram which shows the functional structure of the sound collecting apparatus which concerns on 1st Embodiment. It is a perspective view of the area around the target area (including the microphone array device) which concerns on 1st Embodiment. It is a figure which showed the structure of the microphone array apparatus (microphone array) which concerns on 1st Embodiment. It is the figure which looked at the periphery of the target area (including the microphone array apparatus) which concerns on 1st Embodiment from the upper direction. It is the figure which looked at the periphery of the target area (including the microphone array apparatus) which concerns on 1st Embodiment from the left. It is a block diagram which showed the example of the hardware composition of the sound collecting apparatus which concerns on 1st Embodiment. It is a block diagram which shows the structure of the sound collecting apparatus which concerns on 2nd Embodiment. It is a figure which showed the structure of the microphone array apparatus (microphone array) which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the sound collecting apparatus which concerns on 3rd Embodiment. It is a figure which showed the configuration example of the microphone array apparatus which concerns on the modification of 2nd Embodiment. It is a block diagram which shows the structure of the conventional subtraction type BF. It is explanatory drawing which showed the example of the directivity filter formed by the conventional subtraction type BF.

(A) First Embodiment Hereinafter, the first embodiment of the sound collecting device, the sound collecting program, and the sound collecting method according to the present invention will be described with reference to the drawings.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing a functional configuration of the sound collecting device 10 according to the first embodiment.

The sound picking device 10 uses a microphone array device ME including a plurality of microphone arrays to perform a target area sound picking process for picking up a target area sound from a sound source in the target area. In this embodiment, it is assumed that the microphone array MA (MAL, MAR, MAU, MAB) is provided.

Next, the configuration of the microphone array device ME will be described with reference to FIGS. 2 and 3.

FIG. 2 is a diagram showing an arrangement configuration of each microphone array constituting the microphone array device ME and a setting (design) of a target area.

FIG. 2 is a view (perspective view) of the periphery of the target area in which the microphone array constituting the microphone array device ME is arranged, viewed from an obliquely upward direction.

As shown in FIG. 2, the microphone arrays MAL, MAR, MAU, and MAB are arranged at arbitrary positions in the space where the target area exists. The position of the microphone array with respect to the target area may be anywhere as long as the directivity of each microphone array overlaps only in the target area.

In this embodiment, as shown in FIG. 2, it is assumed that the target area (sound collection area) to be sound-collected by the sound-collecting device 10 is set on a flat surface DH in an arbitrary space. In FIG. 2, the target area set on the plane DH is designated by TA, and the center position of the target area is designated by PC.

In FIG. 2, a coordinate system based on the origin P0 set on the plane DH is set for the space around the target area. In FIG. 2, when viewed from the direction of FIG. 2, the right direction is the + X direction, the left direction is the -X direction, the front side direction is the -Y direction, the back side direction is the + Y direction, the upward direction is the + Z direction, and the downward direction is the downward direction. It is in the −Z direction. In the following, it is assumed that the space around the target area is shown in the form of (X, Y, Z).

Here, as shown in FIG. 2, it is assumed that the center position PC of the target area is arranged in the + X direction (on the right side when viewed from the direction of FIG. 2) from the origin P0. Further, here, as shown in FIG. 2, the microphone array MAL is in the −Y direction from the origin P0 (front side when viewed from the direction of FIG. 2), and the microphone array MAR is in the + Y direction from the origin P0 (viewed from the direction of FIG. 2). The microphone array MAU is located in the + Z direction from the origin P0 (upper side when viewed from FIG. 2), and the microphone array MAD is located in the −Z direction (lower side when viewed from FIG. 2) from the origin P0. It is assumed that

FIG. 3 is a diagram showing an arrangement configuration of each microphone array when the direction from the center position of the target area TA to the origin P0 (−X direction; hereinafter, also referred to as “front direction”) is viewed.

As shown in FIGS. 2 and 3, the microphone arrays MAL, MAR, MAU, and MAB are arranged so that the directivity overlaps only in the target area TA. Further, as shown in FIGS. 2 and 3, each microphone array MA is composed of two or more microphones M, and each microphone M collects an acoustic signal.

In this embodiment, as shown in FIG. 3, the sound collecting device 10 includes a set of microphone arrays MAL and MAR arranged in the left-right direction (horizontal direction) when viewed from the target area TA (center position PC) and the target area. It is assumed that the sound collection processing for the target area TA is performed separately for the set of the microphone arrays MAU and MAB arranged in the vertical direction (vertical direction) when viewed from the TA (center position PC). In the following, for the sake of simplicity, the two microphone arrays treated as a set in the sound collection process will be referred to as a "microphone array set". In this embodiment, the set of the microphone array MAL and MAR is referred to as the microphone array set MSLR, and the set of the microphone array MAU and MAB is referred to as the microphone array set MSUB.

In this embodiment, it is assumed that two microphones M1 and M2 for collecting acoustic signals are arranged in each microphone array. That is, in this embodiment, it is assumed that each microphone array MA constitutes a 2ch microphone array. The distance between the two microphones M1 and M2 is not limited, but in the example of this embodiment, the distance between the two microphones M1 and M2 is 3 cm.

In this embodiment, the microphone arrays MAL and MAR may be arranged anywhere on the plane DH so that the directivity overlaps only in the target area TA, and may be arranged opposite to each other with the target area TA in between, for example. ..

FIG. 4 is a view when the periphery of the target area TA (center position PC) is viewed from above (+ Z direction).

FIG. 5 is a view when the periphery of the target area TA (center position PC) is viewed from the left direction (−Y direction) when viewed from the center position PC. In FIGS. 4 and 5, a straight line (dotted arrow) is attached to the front direction (direction orthogonal to the line connecting the microphones M1 and M2; directivity direction) from each microphone array.

Here, as shown in FIGS. 4 and 5, it is assumed that the straight lines (dotted arrows) extending in the front direction from each microphone array are adjusted so as to intersect at the center position PC of the target area TA.

Next, the internal configuration of the sound collecting device 10 will be described with reference to FIG.

As shown in FIG. 1, the sound collecting device 10 includes a signal input unit 11, an input level correction unit 12, a directivity forming unit 13, a spatial coordinate data storage unit 14, a delay correction unit 15, a correction coefficient calculation unit 16, and a target area. It has a sound extraction unit 17 and a target area sound selection unit 18.

Then, the signal input unit 11, the directivity forming unit 13, the delay correction unit 15, the correction coefficient calculation unit 16, and the target area sound extraction unit 17 form components that perform processing corresponding to the two microphone array sets MSLR and MSUB, respectively. Have. Specifically, the signal input unit 11, the directivity forming unit 13, the delay correction unit 15, the correction coefficient calculation unit 16, and the target area sound extraction unit 17 are the signal input processing unit 111 (111A, 111B) and the directivity formation, respectively. It has a processing unit 131 (131A, 131B), a delay correction processing unit 151 (151A, 151B), a correction coefficient calculation processing unit 161 (161A, 161B), and a target area sound extraction processing unit 171 (171A, 171B). ..

Here, the signal input processing unit 111A, the directivity forming processing unit 131A, the delay correction processing unit 151A, the correction coefficient calculation processing unit 161A, and the target area sound extraction processing unit 171A each perform processing corresponding to the microphone array set MSLR. It shall be an element. Further, here, the signal input processing unit 111B, the directivity formation processing unit 131B, the delay correction processing unit 151B, the correction coefficient calculation processing unit 161B, and the target area sound extraction processing unit 171B each perform processing corresponding to the microphone array set MSUB. It shall be a component to be performed.

Next, the hardware configuration of the sound collecting device 10 will be described with reference to FIG.

FIG. 2 is a block diagram showing an example of a hardware configuration of the sound collecting device 10.

The sound collecting device 10 may be configured entirely by hardware (for example, a dedicated chip or the like), or may be partially or entirely configured as software (program). The sound collecting device 10 may be configured by installing a program (including the sound collecting program of the embodiment) on a computer having a processor and a memory, for example.

FIG. 2 shows an example of a hardware configuration when the sound collecting device 10 is configured by using software (computer).

The sound collecting device 10 shown in FIG. 2 has a computer 200 in which a program (including the sound collecting program of the embodiment) is installed as a hardware component. Further, the computer 200 may be a computer dedicated to a sound collecting program, or may be configured to be shared with a program having another function.

The computer 200 shown in FIG. 2 has a processor 201, a primary storage unit 202, and a secondary storage unit 203. The primary storage unit 202 is a storage means that functions as a working memory (work memory) of the processor 201, and for example, a memory that operates at high speed such as a DRAM (Dynamic Random Access Memory) can be applied. The secondary storage unit 203 is a storage means for recording various data such as an OS (Operating System) and program data (including data of a sound collecting program according to an embodiment), and is, for example, a non-volatile memory such as a FLASH memory or an HDD. Sexual memory can be applied. In the computer 200 of this embodiment, when the processor 201 is started, the OS and programs (including the sound collecting program according to the embodiment) recorded in the secondary storage unit 203 are read and expanded on the primary storage unit 202. Execute.

The specific configuration of the computer 200 is not limited to the configuration shown in FIG. 2, and various configurations can be applied. For example, if the primary storage unit 202 is a non-volatile memory (for example, FLASH memory or the like), the secondary storage unit 203 may be excluded.

(A-2) Operation of the First Embodiment Next, the operation of the sound collecting device 10 of the first embodiment having the above configuration will be described.

The signal input unit 11 performs a process of converting an acoustic signal picked up by each microphone array from an analog signal to a digital signal and inputting the sound signal. The signal input unit 11 then converts the input signal (digital signal) from the time domain to the frequency domain by using, for example, a fast Fourier transform.

As described above, the signal input processing unit 111A performs signal processing of the microphone array set MSLR (microphone array MAL, MAR), and the signal input processing unit 111B performs signal processing of the microphone array set MSUB (microphone array MAU, MAB). Hereinafter, in each microphone array input signals in the frequency domain of the microphones M1, M2, described as _X 1, _{X 2,} respectively.

The input level correction unit 12 corrects the levels of the input signals supplied from the microphone arrays MAL, MAR, MAU, and MAB so that they all have the same magnitude. For example, the input level correction unit 12 acquires the distance between the center position PC of the target area and each microphone array from the space coordinate data storage unit 14, and calculates the distance attenuation of each microphone array from the difference in the distance of each microphone array. The microphone array that is calculated and is closest to the center position TC of the target area may be set with reference to the target area sound, and the input signals of other microphone arrays may be corrected so as to match the reference microphone array. Alternatively, a speaker (not shown) is installed in advance in the center position PC of the target area TA, white noise is reproduced from the speaker (not shown) and recorded in each microphone array, and the level difference of the white noise recorded in each microphone array is calculated. (Compare the level of white noise recorded in each microphone array), and the level of the input signal of other microphone arrays may be corrected based on the microphone array in which the largest level of white noise is recorded. .. If all the microphone arrays are equidistant from the center position PC of the target area TA, the processing of the input level correction unit 12 may be omitted.

The directivity forming unit 13 forms directivity in the direction of the target area by BF according to the equation (4) with respect to the input signal for each microphone array. As described above, the directivity forming processing unit 131A performs signal processing of the microphone array set MSLR (microphone array MAL, MAR), and the directivity forming processing unit 131B performs signal processing of the microphone array set MSUB (microphone array MAU, MAB). Do.

Hereinafter, in each microphone array set, it is assumed that the BF output of any one microphone array is Y _1k (n) and the BF output of the other microphone array is Y _2k (n). .. For example, the amplitude spectra of the BF outputs of the microphone arrays MAL and MAR _{are applied to each equation as Y 1k} (n) and Y _2k (n), respectively, and the amplitude spectra of the BF outputs of the microphone arrays MAU and MAB are Y _{1k, respectively.} It may be applied to each equation as (n) and Y _{2k (n).}

The spatial coordinate data storage unit 14 holds all the target areas and the position information of the microphones constituting each microphone array.

The delay correction unit 15 calculates and corrects the delay generated due to the difference in the distance between the target area TA (center position PC) and each microphone array. The delay correction unit 15 first acquires the position of the target area TA (center position PC) and the position of each microphone array from the spatial coordinate data storage unit 14, and calculates the difference in the arrival time of the target area sound to each microphone array. .. Next, the delay correction unit 15 adds a delay so that the target area sound reaches all the microphone arrays at the same time with reference to the microphone array arranged at the position farthest from the target area. As described above, the delay correction processing unit 151A performs signal processing of the microphone array set MSLR (microphone array MAL, MAR), and the delay correction processing unit 151B performs signal processing of the microphone array set MSUB (microphone array MAU, MAB). If all the microphone arrays are equidistant from the center position PC of the target area TA, the processing of the delay correction unit 15 may be omitted.

The correction coefficient calculation unit 16 calculates, for each microphone array set, a correction coefficient for making the amplitude spectrum of the target area sound component included in each BF output the same. The correction coefficient calculation unit 16 calculates the correction coefficient according to "Equation (5), (6)" or "Equation (7), Eq. (8)". Here, the correction coefficients for the BF output of the microphone arrays MAL and MAR will be described as _{α 1} (n) and α _{2 (n).} Further, here, the correction coefficients for the BF output of the microphone arrays MAU and MAB will be described as α ₁ (n) and α ₂ (n). For example, in the microphone array set MSLR, when the main microphone array (the microphone array that serves as a reference in the extraction processing of the target area sound) is set to the microphone array MAL, the equations (6) and (8) are used. The amplitude spectrum correction coefficient α _{1 (n) is calculated, and α 2} (n) is calculated by the “formulas (5) and (7)” as needed.

Further, here, it is assumed that one of the microphone arrays is set as the main microphone array for each of the microphone array sets MSLR and MSUB. In the sound collecting device 10, the specific method of the process of selecting the main microphone array from the microphone array set is not limited. Further, as described above, in the correction coefficient calculation unit 16, the correction coefficient calculation processing unit 161A performs the processing related to the microphone array set MSLR (microphone array MAL, MAR), and the correction coefficient calculation processing unit 161B performs the processing related to the microphone array set MSUB (microphone). Performs processing related to array MAU, MAB).

The target area sound extraction unit 17 extracts the target area sound for each of the microphone array sets MSLR and MSUB with reference to the main microphone array (the microphone array that serves as a reference during the extraction process of the target area sound).

For example, in the microphone array set MSLR, when the main microphone array is set to the microphone array MAL, the target area sound extraction unit 17 outputs each BF output by _{the correction coefficient α 2 (n) calculated by the correction coefficient calculation unit 16.} 9) SS is performed according to the equation, and the non-purpose area sound existing in the direction of the target area is extracted. Further, the target area sound extraction unit 17 extracts the target area sound by SSing the extracted non-purpose area sound from the output of each BF according to the equation (11). Further, for example, in the microphone array set MSLR, when the main microphone array is set to the microphone array MAR, the target area sound extraction unit 17 outputs each BF output according to the equation (10) according to _{the correction coefficient α 1 (n) to the target area.} Extract the non-purpose area sound that exists in the direction. Further, the target area sound extraction unit 17 extracts the extracted non-purpose area sound from the output of each BF according to the equation (12).

As described above, in the target area sound extraction unit 17, the target area sound extraction processing unit 171A performs signal processing of the microphone array set MSLR (microphone array MAL, MAR), and the target area sound extraction processing unit 171B performs the microphone array set. Signal processing of MSUB (microphone array MAU, MAB) is performed.

The target area sound selection unit 18 compares the target area sound extracted by the target area sound extraction unit 17A with the target area sound extracted by the target area sound extraction unit 17B, and selects one of the target area sounds. It is an output means that outputs as a final target area sound (output signal). The method of selecting one of the two target area sounds by the target area sound selection unit 18 is not limited, but may be selected based on, for example, the average amplitude spectrum. For example, the target area sound selection unit 18 may calculate average amplitude spectra for each of the two target area sounds, compare the values, and finally select and output the smaller value.

(A-3) Effect of First Embodiment According to the first embodiment, the following effects can be obtained.

In the sound collecting device 10 of the first embodiment, the microphone arrays MAL and MAR (microphone array set MSLR) installed in the left-right direction in front of the target area and the microphone arrays MAU and MUB (microphone array set MSUB) installed in the vertical direction are used. The target area sound is extracted for each of the above, and the one with the lower volume level (the one with the smaller average amplitude spectrum) is selected and output from the extracted target area sounds. As a result, the sound collecting device 10 of the first embodiment does not collect sound even if non-purpose area sound exists in the vertical and horizontal directions of the target area, and stably collects the sound in the target area floating in the air. Can make a sound.

Further, as a result, the sound collecting device 10 of the first embodiment can be expected to reduce discomfort due to noise mixing and improve the voice recognition rate. Selecting the target area sound extracted by the combination of the upper and lower microphone arrays and the left and right microphone arrays means that the sound is picked up at the overlapping portion of the two sound collecting areas by the upper and lower microphone arrays and the left and right microphone arrays. That is, in the sound collecting device 10 of the first embodiment, it is possible to collect sound in an arbitrary floating area by changing the installation position and angle of the microphone array.

For example, when a speaker that reproduces a loud sound such as an announcement is placed on the ceiling above the target area, the speaker is added to the target area sound extracted by the left and right microphone arrays MAL and MAR (microphone array set MSLR). Sound is included, but is not included in the target area sound extracted by the upper and lower microphone arrays MAU and MUB (microphone array set MSUB). In this case, the volume level of the target area sound extracted by the left and right microphone arrays is higher than that of the target area sound extracted by the upper and lower microphone arrays because the speaker sound is included. That is, in the sound collecting device 10 of the first embodiment, if the target area sound extracted by the upper and lower microphone arrays having a low volume level is selected, the speaker sound is not included in the target area sound. Further, when a person is speaking outside the target area, the sound collecting device 10 of the first embodiment selects the target area sound extracted by the left and right microphone arrays, and the voice of the person outside the target area is not picked up.

(B) Second Embodiment Hereinafter, a second embodiment of the sound collecting device, the sound collecting program, and the sound collecting method according to the present invention will be described with reference to the drawings.

(B-1) Configuration of Second Embodiment FIG. 7 is a block diagram showing an overall configuration of the sound collecting device 10A of the second embodiment. In FIG. 7, the same or corresponding reference numerals are given to the same or corresponding parts as those in FIG. 1 described above. Hereinafter, the second embodiment will be described focusing on the differences from the first embodiment.

In the sound collecting device 10A of the second embodiment, the signal input unit 11, the input level correction unit 12, the directivity forming unit 13, the delay correction unit 15, the correction coefficient calculation unit 16, the target area sound extraction unit 17, and the target area The sound selection unit 18 is assigned to the signal input unit 11A, the input level correction unit 12A, the directivity forming unit 13A, the delay correction unit 15A, the correction coefficient calculation unit 16A, the target area sound extraction unit 17A, and the target area sound selection unit 18A, respectively. It has been replaced. Further, in the sound collecting device 10A of the second embodiment, the combination selection unit 19 is added.

FIG. 8 is a diagram showing an arrangement configuration of each microphone array when the direction from the center position of the target area TA to the origin P0 is viewed in the second embodiment.

In the second embodiment, the physical configuration (number, arrangement position, etc.) of each microphone array constituting the microphone array device ME is the same as that in the first embodiment, but the microphone array processed by the sound collecting device 10A. The set structure (logical structure) is different.

In the first embodiment, a microphone array set MSLR that is a combination of microphone arrays MAL and MAR arranged in the left-right direction (horizontal direction) and a microphone that is a combination of microphone arrays MAU and MAB arranged in the vertical direction (vertical direction) are used. The array set MSUB was logically configured, but in the second embodiment, a microphone array set that is a combination of diagonal (diagonal) microphone arrays is added. Specifically, in the second embodiment, as shown in FIG. 8, as the microphone array set of the combination of the diagonal microphone arrays, the microphone array set MSUL, the microphone array MAL, and the MAB of the combination of the microphone array MAU and MAL. The combination of the microphone array set MSLB, the microphone array MAB, the combination of the microphone array set MSBR, and the combination of the microphone array MAR and the MAU, the microphone array set MSRU are added.

That is, in the sound collecting device 10A of the second embodiment, it is assumed that six microphone array sets MSLR, MSUB, MSUL, MSLB, MSBR, and MSRU can be set.

By the way, when the non-purpose area sound exists in both the vertical direction and the horizontal direction of the target area TA (center position PC) and the volume is high, the target area sound is extracted using either the vertical or horizontal microphone array. However, there is a risk that non-purpose area sounds will be mixed in both. Therefore, in the sound collecting device 10A of the second embodiment, not only the combination of the upper / lower / left / right microphone arrays but also the combination of the diagonal microphone arrays (the above-mentioned six microphone array sets) is used to produce the target area sound. Are extracted and compared, and the one with the lowest volume level of the extracted target area sound is selected as the final output.

In the first embodiment, the components to be processed for each microphone array set are shown separately, but in the sound collecting device 10A of the second embodiment, the description is omitted. In the sound collecting devices 10 and 10A, the components to be processed for each microphone array set may be separated (software or hardware separated) or integrated. Good.

(B-2) Operation of the Second Embodiment Next, the operation of the sound collecting device 10A of the second embodiment having the above configuration will be described.

Hereinafter, the operation of the sound collecting device 10A of the second embodiment will be described focusing on the difference from the first embodiment.

The signal input unit 11A performs a process of converting an acoustic signal picked up by each microphone array from an analog signal to a digital signal and outputting it as an input signal.

The combination selection unit 19 has M pieces (2 or more, maximum number of microphone array sets) from the above-mentioned six microphone array sets MSLR, MSUB, MSUL, MSLB, MSBR, and MSRU (combination of microphone arrays) based on a predetermined rule. The following integers; here, integers of 2 or more and 6 or less) are selected. The combination of the M microphone array sets selected by the combination selection unit 19 may have preset contents, or the contents may be changed according to the time series.

Specifically, for example, in consideration of the installation environment of the microphone array in advance, M combinations (microphone array set) in which non-purpose area sounds such as speakers are unlikely to be mixed are set in the combination selection unit 19 (for example, an operator or an operator. It may be set by the designer or the like).

Further, for example, the combination selection unit 19 first selects all the microphone array sets, evaluates each microphone array set based on the information of the microphone array sets selected by the target area sound selection unit 18A, and then evaluates each microphone array set. May select the top three highly rated microphone array sets.

Further, for example, when the combination selection unit 19 is biased in the microphone array set finally selected by the target area sound selection unit 18A (for example, the probability that two microphone array sets are selected is 50% or more of the total). The well-selected (frequently selected) microphone array set may be fixedly selected, and the other may be randomly selected from the remaining microphone array sets.

Hereinafter, the microphone array set most recently selected by the combination selection unit 19 will be referred to as a “selection microphone array set”.

The input level correction unit 12A corrects the levels of the input signals supplied from the microphone arrays MAL, MAR, MAU, and MAB so that they all have the same magnitude.

The directivity forming unit 13A forms directivity toward the target area by BF for each of the microphone arrays MAL, MAR, MAU, and MAB (generates a BF output). In the second embodiment, as in the first embodiment, in each microphone array set, the BF output of any one microphone array is Y _1k (n), and the BF output of the other microphone array is Y _2k (Y 2k (n). It will be described as assuming that each equation is applied as n).

Similar to the first embodiment, the delay correction unit 15A calculates and corrects the delay generated by the difference in the distance between the target area TA (center position PC) and each microphone array for the BF output of each microphone array.

The correction coefficient calculation unit 16A calculates, for each of the selected microphone array sets, a correction coefficient for making the amplitude spectrum of the target area sound component included in each BF output the same, as in the first embodiment. In the sound collecting device 10A, it is assumed that one of the microphone arrays is set as the main microphone array for each microphone array set as in the first embodiment.

The target area sound extraction unit 17A extracts the target area sound for each of the selected microphone array sets with reference to the main microphone array as in the first embodiment.

The target area sound selection unit 18A compares each of the target area sounds extracted by the target area sound extraction unit 17 (target area sounds extracted using each selection microphone array set), and one of the target area sounds. Is selected and output as the final target area sound (output signal). The process itself of selecting one of the target area sounds from the plurality of target area sounds by the target area sound selection unit 18A may apply the same method as in the first embodiment. Further, the target area sound selection unit 18A feeds back the selection result of the target area sound (information on which microphone array set the target area sound is selected) to the combination selection unit 19. When necessary for the processing of the combination selection unit 19, the target area sound selection unit 18A also compares the target area sounds extracted by the target area sound extraction unit 17 (for example, the ranking of the evaluation results) with the combination selection unit. You may want to give feedback to 19.

(B-3) Effect of Second Embodiment According to the second embodiment, the following effects can be obtained in addition to the effect of the first embodiment.

In the sound collecting device 10A of the second embodiment, as compared with the first embodiment, the most candidate among the candidates to which the target area sound extracted by the diagonal combination of microphone arrays (microphone array set) is added. The one with a small average amplitude spectrum is selected and output. As a result, in the sound collecting device 10A of the second embodiment, as described above, it is possible to collect the target area sound more stably while suppressing the mixing of the non-target area sound. For example, in the sound collecting device 10A of the second embodiment, it is possible to suppress the mixing of non-purpose area sounds even in an environment in which a plurality of non-purpose area sounds exist around the sound collecting area.

(C) Third Embodiment Hereinafter, a third embodiment of the sound collecting device, the sound collecting program, and the sound collecting method according to the present invention will be described with reference to the drawings.

(C-1) Configuration of Third Embodiment FIG. 9 is a block diagram showing an overall configuration of the sound collecting device 10B of the third embodiment. In FIG. 9, the same or corresponding reference numerals are given to the same or corresponding parts as those in FIG. 1 described above. Hereinafter, the third embodiment will be described focusing on the differences from the first embodiment.

The sound collecting device 10B of the third embodiment is different from the first embodiment in that the target area sound selection unit 18 is replaced with the frequency-specific target area sound selection unit 20.

(C-2) Operation of the Third Embodiment Next, the operation of the sound collecting device 10B of the third embodiment having the above configuration will be described.

Hereinafter, the operation of the sound collecting device 10B of the third embodiment will be described focusing on the difference from the first embodiment.

As described above, in the sound collecting device 10B of the third embodiment, only the frequency-specific target area sound selection unit 20 is different from the first embodiment. Only will be described.

The target area sound selection unit 18 of the first embodiment combines the target area sound extracted by the target area sound extraction unit 17A and the target area sound extracted by the target area sound extraction unit 17B based on the average amplitude spectrum. One of them was selected and output. On the other hand, the frequency-specific target area sound selection unit 20 of the third embodiment has the target area sound components extracted by the target area sound extraction unit 17A for each frequency and the target area sound extraction unit 17B. Compare with the component of the target area sound, select one of the components of the target area sound for each frequency, and collect the components of each selected frequency to generate one target area sound (output signal) and output it. To do.

The method by which the frequency-specific target area sound selection unit 20 selects one of the two target area sound components is not limited, but for example, the two target area sound components (of the frequencies to be compared). The selection may be made based on the value (power) of the component). For example, the frequency-specific target area sound selection unit 20 may finally select and output the smaller value of the two target area sound components.

(C-3) Effect of Third Embodiment According to the third embodiment, the following effects can be achieved.

In the sound collecting device 10B of the third embodiment, the microphone arrays MAL and MAR (microphone array set MSLR) installed in the left-right direction in front of the target area and the microphone arrays MAU and MUB (microphone array set MSUB) installed in the vertical direction are used. The target area sound is extracted for each of the above, and the one with the smaller amplitude spectrum for each frequency is selected and output from the extracted target area sounds. As a result, in the sound collecting device 10B of the second embodiment, for example, even if there is a non-target area sound in the vertical and horizontal directions of the target area, the sound is not collected, and the sound in the area floating in the air is collected. be able to.

(D) Other Embodiments The present invention is not limited to each of the above embodiments, and modified embodiments as illustrated below can also be mentioned.

(D-1) In the microphone array device ME of the second embodiment, as shown in FIGS. 2 and 3, the microphone arrays MAL and MAR installed in the left-right direction in front of the target area and the microphone arrays installed in the vertical direction Area sound collection was performed using MAU and MUB. In other words, in the microphone array device ME of the second embodiment, four microphone arrays are arranged in a cross shape with the origin P0 as the center to form a microphone array set. However, as shown in FIG. 10, the origin P0 is arranged. The microphone array set may be formed by arranging them in a grid pattern as the center.

FIG. 10 is a diagram showing an arrangement configuration of each microphone array when the direction (−X direction; front direction) from the center position of the target area TA to the origin P0 is viewed in the modified example of the second embodiment. ..

The microphone array device ME shown in FIG. 10 includes a microphone array MARU arranged diagonally above the origin P0 and a microphone array MARB arranged diagonally below the origin P0 when viewed from the destination area TA (center position PC). It has a microphone array MALB arranged diagonally to the left of the origin P0 and a microphone array MALU arranged diagonally to the left of the origin P0.

When the microphone array device ME shown in FIG. 10 is applied, in the sound collecting device 10A of the second embodiment, the microphone array set MS_LU_RU of the combination of the microphone array MARU and MARU, and the microphone array of the combination of the microphone array MARU and MARB are applied. Microphone array set of combination MS_RU_RB, microphone array MARB, MALB Microphone array set of combination MS_RB_LB, microphone array MALB, MALU Microphone array set of combination MS_LU_LB, microphone array MALU, MARB Combination of microphone array set MS_LU_RB, microphone array MARU, MALB The microphone array set MS_RU_LB will be formed (set).

(D-2) In each of the above embodiments, an example in which four microphone arrays are arranged in the microphone array device ME has been shown, but the number of microphone arrays provided in the microphone array device ME may be three or more. If three or more microphone arrays are arranged in the microphone array device ME, two or more (three) microphone array sets can be configured by combining them. In other words, the microphone array device ME may be provided with enough microphones to form two or more microphone array sets. For example, in the microphone array device ME shown in FIGS. 2 and 3, the microphone array MAB on the lower side may be excluded.

(D-3) In the sound collecting device 10A of the second embodiment, a microphone array set in which diagonal microphone arrays are combined may be added to enable processing as in the third embodiment. That is, a sound collecting device may be configured by combining the second embodiment and the third embodiment.

10 ... Sound collecting device, 11 ... Signal input unit, 12 ... Input level correction unit, 13 ... Directivity forming unit, 14 ... Spatial coordinate data storage unit, 15 ... Delay correction unit, 16 ... Correction coefficient calculation unit, 17 ... Purpose Area sound extraction unit, 18 ... Target area sound selection unit, 111, 111A, 111B ... Signal input processing unit, 131, 131A, 131B ... Directivity formation processing unit, 151, 151A, 151B ... Delay correction processing unit, 161, 161A , 161B ... Correction coefficient calculation processing unit, 171, 171A, 171B ... Target area sound extraction processing unit, ME ... Microphone array device, MA, MAL, MAR, MAU, MAB ... Microphone array, MS, MSLR, MSUB ... Microphone array set , M, M1, M2 ... Microphone.

Claims (6)

For each of the input signals supplied from the plurality of microphone arrays, the beam former forms directivity toward the target area where the target area exists, and the target direction signal from the target area direction is acquired for each of the microphone arrays. Directivity forming means and
Three or more microphone array sets formed by a combination of the two microphone arrays are set, and for each of the microphone array sets, the target direction signal of each of the microphone arrays is spectrally subtracted to move toward the target area. A target area sound extraction means that extracts a target area sound by extracting an existing non-purpose area sound and subtracting the spectrum of the extracted non-purpose area sound from any of the target direction signals.
Based on the result of comparing the target area sounds extracted using each of the microphone array sets, one output signal is generated and output from the target area sounds extracted using each of the microphone array sets. A sound collecting device characterized by having an output means. The first aspect of the present invention is characterized in that the output means selects the target area sound having the lowest volume level from the target area sounds extracted using the respective microphone array sets and outputs the target area sound as an output signal. The described sound collector. The output means selects by comparing the component values for each frequency with respect to the target area sound extracted using each of the microphone array sets, and generates an output signal based on the component for each selected frequency. The sound collecting device according to claim 1. The sound collecting device according to claim 3, wherein the output means selects a component having the lowest value for each frequency with respect to the target area sound extracted using each of the microphone array sets. Computer,
For each of the input signals supplied from the plurality of microphone arrays, the beam former forms directivity toward the target area where the target area exists, and the target direction signal from the target area direction is acquired for each of the microphone arrays. Directivity forming means and
Three or more microphone array sets formed by a combination of the two microphone arrays are set, and for each of the microphone array sets, the target direction signal of each of the microphone arrays is spectrally subtracted to move toward the target area. A target area sound extraction means that extracts a target area sound by extracting an existing non-purpose area sound and subtracting the spectrum of the extracted non-purpose area sound from any of the target direction signals.
Based on the result of comparing the target area sounds extracted using each of the microphone array sets, one output signal is generated and output from the target area sounds extracted using each of the microphone array sets. A sound collection program characterized by functioning as an output means. In the sound collecting method performed by the sound collecting device,
The sound collecting device includes a directivity forming means, a target area sound extracting means, and an output means.
The directivity forming means forms directivity for each of the input signals supplied from the plurality of microphone arrays in the direction of the target area where the target area exists by the beam former, and for each of the microphone arrays, from the direction of the target area. Get the destination direction signal of
The target area sound extraction means sets three or more microphone array sets formed by a combination of the two microphone arrays, and for each of the microphone array sets, spectrum subtracts the target direction signal of each of the microphone arrays. By doing so, the non-purpose area sound existing in the target area direction is extracted, and the target area sound is extracted by subtracting the spectrum of the extracted non-purpose area sound from any of the target direction signals.
The output means outputs one output signal from the target area sound extracted using the respective microphone array sets based on the result of comparing the target area sounds extracted using the respective microphone array sets. A sound collection method characterized by generating and outputting.

Images