JP2020120261A - Sound pickup device, sound pickup program, and sound pickup method - Google Patents

Abstract

To provide a sound pickup device, a sound pickup program, and a sound pickup method that improve sound quality when picking up a target area sound.SOLUTION: The present invention relates to a sound pickup device. The sound pickup device of the present invention comprises: non-target area sound extraction means that performs spectrum subtraction on respective input signals input to a plurality of directional microphones to extract a non-target area sound present in a target area direction viewed from the respective directional microphones; and target area sound extraction means that performs spectrum subtraction on the non-target area sound from the input signals to extract a target area sound.SELECTED DRAWING: Figure 1

Description

The present invention relates to a sound collecting device, a sound collecting program, and a sound collecting method, and can be applied to, for example, a sound collecting device and a program that emphasize sounds in a specific area and suppress sounds in other areas.

When a voice communication system or a voice recognition application system is used in a noisy environment, ambient noise mixed with the required target voice interferes with good communication and is a troublesome presence that lowers the voice recognition rate. Conventionally, in such an environment where multiple sound sources exist, a beamformer using a microphone array has been used as a technique for obtaining a desired target sound by separating and collecting only sound in a specific direction to avoid mixing of unnecessary sound. (Beam Former; hereinafter referred to as “BF”; see Patent Document 2). BF is a technique for forming directivity by utilizing the time difference between signals reaching each microphone.

FIG. 5 is a block diagram showing a configuration related to a subtraction type BF when the number of microphones is two in the related art.

Here, as shown in FIG. 5, it is considered that the sound coming from the θ _L direction is received by the two microphones M1 and M2 which are installed at a distance d. The sound wave coming from the θ _L direction is first received by the right microphone M1 close to the sound source. Next, the sound wave arriving from the θ _L direction travels the distance 1 and reaches the left microphone M2. Therefore, the sound reception signal x ₂ (t) at the microphone M2 becomes a signal delayed by the time τ _L required for the sound wave to travel the distance 1 compared to the sound reception signal x ₁ (t) at the microphone M1. There is. That is, in the configuration shown in FIG. 5, the relationships of the following expressions (1) and (2) are established. Note that the sound velocity is expressed as “c” in the expression (1) and below.
τ _L =(dsin θ _L )/c (1)
_{x 2 (t) = x 1} (t-τ L) ... (2)

Therefore, in the configuration shown in FIG. 5, the delay device 21 delays x ₁ (t) by τ _L and adds it to x ₂ (t), thereby enhancing the signal a in which the sound in the θ _L direction is emphasized. (T) can be acquired (see the equation (3)). In the configuration shown in FIG. 5, conversely, by delaying x ₁ (t) by τ _L and subtracting x ₂ (t), the signal soil is canceled and a blind spot is formed in the θ _L direction. The obtained signal b(t) can be obtained (see the equation (4)).
_{a (t) = x 2 (} t) + x 1 (t-τ L) ... (3)
_{b (t) = x 2 (} t) -x 1 (t-τ L) ... (4)

Then, the addition/subtraction processing of the expressions (3) and (4) can be similarly performed in the frequency domain. In that case, the expressions (3) and (4) are respectively expressed by the following expression (5), The formula is changed to the formula (6).

By the way, BFs are roughly classified into two types: addition type and subtraction type. In particular, the subtraction type BF has an advantage that directivity can be formed with a smaller number of microphones than the addition type BF. As a typical method of the subtraction type BF, there is a BF using a spectral subtraction method (Spectral Subtraction; hereinafter also simply referred to as “SS”) (see Non-Patent Document 1).

FIG. 6 is a block diagram showing a configuration related to the conventional SS.

In the configuration shown in FIG. 6, it is assumed that the acoustic signal is processed in the frequency domain. In the configuration shown in FIG. 6, the input signal frequency-converted by the time-frequency converter 31 is used.

In the configuration shown in FIG. 6, the delay device 32 first calculates the time difference τ _L between the signals arriving at the microphone M1 and the microphone M2 from the target direction θ _L , and adds the delay to match the phase of the sound signal in the target sound source direction. Here, when the target sound source exists in the direction of the microphone M1 with respect to the centers of the microphones M1 and M2, the delay device 32 performs delay processing on the input of the microphone M1. After that, the adder 33 performs the addition process according to the equation (5), and the subtractor 34 performs the subtraction process according to the equation (6). As a result, the sound in the target sound source direction is emphasized by the addition processing, and the sound other than the target sound source direction is extracted by the subtraction processing. Further, the spectrum subtractor 35 performs the processing according to the expression (7) using the data subjected to the addition processing and the subtraction processing, whereby the sound in the target sound source direction can be emphasized and the other sounds can be suppressed. In the expression (7), “β” is a coefficient for changing the strength of SS.
Y(ω)=A(ω)−βB(ω) (7)

However, if another sound source exists around an area whose sound is to be collected (hereinafter, referred to as “target area”) only with BF, a sound existing in the target area (hereinafter, referred to as “target area sound”). ) It is difficult to pick up only. Therefore, conventionally, Patent Document 1 and the like have proposed an area sound pickup method that picks up a target area using a plurality of microphone arrays.

FIG. 7 is an explanatory diagram showing a process of collecting a target area sound from a sound source of the target area using the two microphone arrays MA1 and MA2 in the conventional technique.

FIG. 7A is an explanatory diagram showing a configuration example of each microphone array. FIGS. 7B and 7C are graphs (image diagrams) showing the BF outputs of the microphone arrays MA1 and MA2 shown in FIG. 7A in the frequency domain.

In the conventional area sound collection, as shown in FIG. 7A, the directivity of the microphone arrays MA1 and MA2 is crossed and collected from different directions in the desired area (target area). In the state of FIG. 7A, the directivity of each of the microphone arrays MA1 and MA2 includes not only the sound existing in the target area (target area sound) but also noise in the target area direction (non-target area sound). There is. However, as shown in FIGS. 7B and 7C, when comparing the directivities of the microphone arrays MA1 and MA2 in the frequency domain, the target area sound component is included in both outputs, but the non-target area is included. The sound component will be different for each microphone array.

In the conventional area sound collecting technique, by utilizing such a characteristic, only the target area sound can be extracted by suppressing components other than the components commonly included in the BF outputs of the two microphone arrays MA1 and MA2.

JP, 2005-195955, A JP, 2014-72708, A

Futoshi Asano, “Sound Array Signal Processing-Localization/Tracking and Separation of Sound Source”, The Acoustical Society of Japan, Corona Publishing, February 25, 2011

By the way, in the area sound collecting method, it is necessary to process the microphone array based on the BF principle in order to form directivity from different positions toward the target area. In order to pinpoint the directivity in the direction of the target sound, it is necessary to form a sharp directivity. If this is to be realized by the addition type BF, a large number of microphones are required, which is not realistic. Therefore, in the conventional area sound pickup, a process in which the subtraction type BF and the SS are combined (hereinafter referred to as “subtraction type BF+SS process” or simply “BF+SS process”) is used to form sharp directivity with a small number of microphones. ..

However, since SS itself is a non-linear process, processing distortion and occurrence of musical noise cannot be avoided. In addition, the frequency range in which the user is good differs greatly depending on the distance between the microphones of the microphone array.

FIG. 8 shows a polar pattern of the subtraction type BF in which the blind spot is directed in the front direction.

In FIG. 8, it is assumed that the distance between the microphones M1 and M2 is 3 cm. In this case, the subtraction type BF shows good characteristics in the range of 1 kHz to 4 kHz, but almost no gain can be obtained in the low frequency range of 500 Hz or lower. Further, in this case, in the subtraction type BF, the directional characteristic gradually begins to be deformed at 8 kHz or higher, and when the frequency becomes higher, a blind spot occurs in a direction other than the front direction due to spatial aliasing. Therefore, in the subtraction type BF, in order to improve the characteristics in the low frequency region, it is necessary to make the microphone interval larger, and conversely, in order to improve the high frequency region, it is necessary to narrow the microphone interval. Therefore, in the conventional subtraction type BF+SS processing, a microphone interval of about 3 cm is often adopted as a compromise point according to the purpose of voice information transmission. However, even in that case, measures such as low-frequency boosting are taken in order to gain low-frequency gain. In this case, low frequency components increase, but as a side effect, distortion increases. Therefore, the conventional area sound collection method has attracted attention as a method capable of collecting only a sound in a specific area, but it is inevitable that a certain amount of distortion is unavoidable in the emphasized target sound. There was a problem such as missing information.

In view of the above problems, a sound pickup device, a sound pickup program, and a sound pickup method that improve the sound quality when picking up a target area sound are desired.

The sound collecting apparatus of the first aspect of the present invention is (1) present in the direction of the target area when viewed from each of the directional microphones by spectrally subtracting the input signals to which the plurality of directional microphones are input. A non-target area sound extraction means for extracting a non-target area sound; and (2) a target area sound extraction means for extracting a target area sound by spectrally subtracting the non-target area sound from the input signal. And

A sound collecting program according to a second aspect of the present invention allows a computer to perform (1) spectral subtraction of input signals to which a plurality of directional microphones are input, so that the direction of the target area is viewed from each of the directional microphones. A non-target area sound extracting means for extracting a non-target area sound existing in (1), and (2) functioning as a target area sound extracting means for extracting a target area sound by spectrally subtracting the non-target area sound from the input signal. It is characterized by

A third aspect of the present invention is a sound collecting method performed by a sound collecting device, wherein (1) the sound collecting device has a non-target area sound extracting means and a target area sound extracting means, and (2) the non-target area sound. The extracting means extracts the non-target area sound existing in the target area direction as viewed from each of the directional microphones by spectrally subtracting each of the input signals input to the plurality of directional microphones, (3 ) The target area sound extracting means extracts the target area sound by spectrally subtracting the non-target area sound from the input signal.

According to the present invention, it is possible to provide a sound collecting device, a sound collecting program, and a sound collecting method that improve sound quality when collecting a target area sound.

It is the block diagram shown about the functional composition of the sound collection device concerning an embodiment. It is a perspective view of an interference tube type super-directional microphone used in the sound collecting device according to the embodiment. It is explanatory drawing shown about the arrangement example of the interference tube type|mold super-directional microphone which concerns on embodiment. It is the block diagram shown about the example of the hardware constitutions of the sound collection device concerning an embodiment. It is a block diagram which shows the structure which concerns on the subtraction type BF in case the number of conventional microphones is two. It is a block diagram showing the composition concerning conventional SS (spectrum subtraction processing). It is explanatory drawing shown about the example of the signal processed when the directivity by the beam former (BF) of two microphone arrays is directed to a target area from another direction in the conventional sound collection device. It is explanatory drawing shown about the polar pattern at the time of directing a blind spot to the front direction in the conventional subtraction type BF.

(A) Main Embodiment Hereinafter, one embodiment of a sound collecting device, a sound collecting program, and a sound collecting method according to the present invention will be described in detail with reference to the drawings.

First, the outline of the configuration of the sound collecting device in this embodiment will be described.

In order to realize the area sound collection, it is necessary to direct the directivity from different positions toward the target area. In the sound collecting device of this embodiment, it is assumed that directional microphones are used in place of a plurality of microphone arrays to perform area sound collection.

Conventionally, directional microphones include unidirectional microphones, bipolar microphones, superdirectional microphones, and the like.

A unidirectional microphone, even if it is called directional, does not take the sound of the back surface but takes all the sound in the front. Therefore, the unidirectional microphone cannot be used as an area pickup microphone. Further, since the bipolar microphone also does not pick up sounds from the lateral direction, it is not suitable for area sound collection processing as in the case of unidirectionality. The so-called super-directional microphone has sharp directivity to the front compared to other microphones and may be applicable to area sound collection.

Superdirective microphones include an interference tube type that structurally uses an interference tube and a secondary sound pressure gradient type that uses two microphone units. Both structures are generally used as super-directional microphones, but the secondary sound pressure gradient type microphones are only those that perform subtractive BF electrically instead of signal processing, and in principle Problems such as frequency characteristics in sound collection processing cannot be solved. On the other hand, the interference tube type microphone has a long cylinder with slits on the side surface attached to the tip of a dedicated microphone unit to "acoustically" narrow the directivity. Therefore, when the interference tube type microphone is used for the area sound collection processing, there is almost no distortion as in the subtraction type BF+SS processing, and the frequency characteristic can cover almost the entire audible range from the low range to the high range.

Therefore, the sound collecting device of this embodiment is configured to perform the area sound collecting process by using the outputs of a plurality of interference tube type super-directional microphones. Hereinafter, an example of a specific configuration of the sound pickup device of the present invention will be described.

(A-1) Configuration of Embodiment FIG. 1 is a block diagram showing a configuration of a sound collecting device according to a first embodiment of the present invention.

FIG. 1 is a block diagram showing a connection configuration of each device and a functional configuration of a sound collecting device 10 according to this embodiment.

The sound pickup device 10 is a device that picks up and outputs a target area sound having a target area as a sound source, based on the acoustic signals captured by the two super-directional microphones M (M1, M2). Hereinafter, the signal output by the sound collection device 10 will be referred to as an output signal Z.

FIG. 2 is a diagram (perspective view) showing a configuration example of superdirective microphones M1 and M2.

As shown in FIG. 2, the super-directional microphones M1 and M2 of this embodiment have an interference tube type structure configured by using tubes having a plurality of slits MS formed on the side surfaces. In FIG. 2, the directions (directions) of the directional MDs of the super-directional microphones M1 and M2 are indicated by dotted arrows. Superdirective microphones M1 and M2 are not limited to the specific configurations (for example, specific configurations of tubes and slits) shown in FIG. 2 as long as they have an interference tube type configuration.

FIG. 3 is an explanatory diagram showing an example of the arrangement configuration of the super-directional microphones M1 and M2 in this embodiment.

As shown in FIG. 3, superdirective microphones M1 and M2 are arranged at arbitrary places in the space where the target area exists. In FIG. 3, the directivity of superdirective microphone M1 is shown by a dotted line, and the directivity of superdirective microphone M2 is shown by a dashed line.

As shown in FIG. 3, superdirective microphones M1 and M2 may be arranged at positions and orientations (directivity directions) such that the directivity of each superdirective microphone overlaps only in the target area. For example, superdirective microphones M1 and M2 may be arranged opposite to each other with the target area in between. Further, the number of super-directional microphones is not limited to two, and when there are a plurality of target areas, a sufficient number of microphones may be arranged to cover all areas.

Next, the internal configuration of the sound collecting device 10 will be described with reference to FIG.

The sound pickup device 10 includes a signal input unit 101, a frequency conversion unit 102, an amplitude correction coefficient calculation unit 103, and a target area sound extraction unit 104. Details of each element constituting the sound collection device 10 will be described later.

Next, the hardware configuration of the sound collection device 10 will be described with reference to FIG.

The sound pickup device 10 may be entirely configured by hardware (for example, a dedicated chip or the like), or a part or all of the sound collection device 10 may be configured as software (program). The sound collecting device 10 may be configured, for example, by installing a program (including the sound collecting program of the embodiment) in a computer having a processor and a memory.

FIG. 4 is a block diagram showing an example of the hardware configuration of the sound collection device 10.

FIG. 4 shows an example of a hardware configuration when the sound collection device 10 is configured using software (computer).

The sound collecting device 10 illustrated in FIG. 4 includes a computer 200 in which a program (including the sound collecting program according to the embodiment) is installed as a hardware component. If the computer 200 does not include a conversion unit that converts an analog signal (a signal supplied from the super-directional microphones M1 and M2) into a digital signal, the sound collection device 10 includes a conversion unit (not shown). You may do it. Further, the computer 200 may be a computer dedicated to a sound collection program, or may be configured to be shared with a program having another function.

The computer 200 illustrated in FIG. 4 includes a processor 201, a primary storage unit 202, and a secondary storage unit 203. The primary storage unit 202 is a storage unit that functions as a work memory (work memory) of the processor 201. For example, a high-speed operating memory such as a DRAM (Dynamic Random Access Memory) can be applied. The secondary storage unit 203 is a storage unit that records various data such as an OS (Operating System) and program data (including data of the sound collection program according to the embodiment), and is a nonvolatile memory such as a FLASH memory or an HDD. Sex memory can be applied. In the computer 200 of this embodiment, when the processor 201 is activated, the OS and programs recorded in the secondary storage unit 203 (including the sound collection program according to the embodiment) are read and expanded on the primary storage unit 202. Execute.

The specific configuration of the computer 200 is not limited to the configuration shown in FIG. 4, and various configurations can be applied. For example, if the primary storage unit 202 is a non-volatile memory (for example, a FLASH memory or the like), the secondary storage unit 203 may be excluded.

(A-2) Operation of Embodiment Next, the operation of the sound collecting device 10 in this embodiment having the above-described configuration will be described.

The signal input unit 101 converts the acoustic signals picked up by the super-directional microphones M1 and M2 from analog signals into digital signals y ₁ and y ₂ , respectively.

The frequency conversion unit 102 converts the input signals y ₁ and y ₂ from time domain signals into frequency domain signals Y ₁ (n) and Y ₂ (n), respectively. The frequency transforming unit 102 transforms the time domain signals y ₁ and y ₂ into frequency domain signals Y ₁ (n) and Y ₂ (n) using, for example, a fast Fourier transform (FFT). ..

If you want to pick up only the sound that exists in a certain area (target area sound), you can also pick up the sound source (non-target area sound) that exists on the line in the same direction as that area simply by pointing the microphone directivity. Resulting in. Therefore, in the sound collecting device 10 of this embodiment, “a plurality of microphone arrays are used, directivity is directed from different directions to the target area, and the directivity is crossed in the target area, is proposed in Patent Document 2. Therefore, the method of collecting the target area sound (area sound collection processing)” is used. However, since the interference-type super-directional microphones M1 and M2 used in this embodiment have sharp directivity themselves, the sound pickup device 10 of this embodiment causes The process of forming directivity in the target sound direction by BF and SS in Patent Document 2 is not necessary.

In order to extract the target sound by the area sound collection processing, it is necessary that the power of the target sound area sound component included in each of the directional input signals Y ₁ (n) and Y ₂ (n) be the same. .. Therefore, the amplitude correction coefficient calculation unit 103 calculates an amplitude correction coefficient for correcting the difference in the magnitude of the target area sound component caused by the difference in the distance between the superdirective microphones M1 and M2 and the target area. There are various conceivable methods of calculating the correction coefficient in the amplitude correction coefficient calculation unit 103, but here, the ratio of the amplitude spectrum is calculated for each frequency, and the mode value thereof is used as the correction coefficient (the following expression (8), ( 9) Formula reference).

The target area sound extraction unit 104 SSs the time-frequency converted data Y ₁ (n) and Y ₂ (n) of the super-directional microphones M1 and M2 according to the expression (10) or the expression (11), and moves in the direction of the target area. The existing non-target area sounds N ₁ (n) and N ₂ (n) are extracted.
N ₁ (n)=Y ₁ (n)-α ₂ (n)Y ₂ (n) (10)
N ₂ (n)=Y ₂ (n)-α ₁ (n)Y ₁ (n) (11)

After that, the target area sound extraction unit 104 extracts the target area sound by SS of the non-target area sound from each BF output according to the following expressions (12) and (13). In the following equations (12) and (13), γ ₁ (n) and γ ₂ (n) are coefficients for changing the strength during SS.
Z ₁ (n)=Y ₁ (n)−γ ₁ (n)N ₁ (n) (12)
Z ₂ (n)=Y ₂ (n)−γ ₂ (n)N ₂ (n) (13)

(A-3) Effects of the Embodiment According to this embodiment, the following effects can be achieved.

In the sound collection device 10 of this embodiment, an interference tube type super directional microphone is used instead of the microphone array.

Thereby, in the sound collecting device 10 of this embodiment, by using the interference tube type super directional microphone, the processing distortion caused by BF+SS, the lack of the low frequency characteristic component, the restriction of the high frequency region, etc. are improved, and the extraction is performed. The sound quality of the target sound is improved.

Further, in the sound collecting device 10 of this embodiment, by using the interference tube type super directional microphone, the BF+SS process for forming the directivity is not required as compared with the conventional case (when the microphone array is used), and the processing configuration is significantly large. Can be simplified.

(B) Other Embodiments The present invention is not limited to each of the above-described embodiments, but may include modified embodiments as exemplified below.

(B-1) In the above-described embodiment, the super-directional microphones M1 and M2 are connected to the outside of the sound collection device 10, but the super-directional microphones M1 and M2 are connected to the sound collection device 10 itself. It may be mounted.

10... Sound collection device, 101... Signal input part, 102... Frequency conversion part, 103... Amplitude correction coefficient calculation part, 104... Target area sound extraction part, M1, M2... Super directional microphone.

Claims (4)

Non-target area sound extraction means for extracting the non-target area sound existing in the target area direction as viewed from each of the directional microphones by spectrally subtracting the input signals input to the plurality of directional microphones. ,
A target area sound extracting means for extracting a target area sound by spectrally subtracting the non-target area sound from the input signal. The sound pickup device according to claim 1, wherein the directional microphone is an interference tube type. Computer,
Non-target area sound extraction means for extracting the non-target area sound existing in the target area direction as viewed from each of the directional microphones by spectrally subtracting the input signals input to the plurality of directional microphones. ,
A sound collection program which functions as a target area sound extraction means for extracting a target area sound by spectrally subtracting the non-target area sound from the input signal. In the sound collecting method performed by the sound collecting device,
The sound collecting device has a non-target area sound extraction means and a target area sound extraction means,
The non-target area sound extraction means performs spectrum subtraction on the input signals to which a plurality of directional microphones are input, so that the non-target area sounds existing in the target area direction when viewed from each of the directional microphones. Extract and
The target area sound extraction means extracts a target area sound by spectrally subtracting the non-target area sound from the input signal.

Images