JP2020039055A - Sound collection device, program, and method - Google Patents

Abstract

To improve sound quality by recovering a component of a high bandwidth to be lost by area sound collection processing.SOLUTION: A sound collection device includes: target area sound extraction means for acquiring a beam former output of each one of multiple microphone arrays on the basis of input signals input from the microphone arrays, and extracting target area sound with a target area as a sound source by using an acquired beam former output; mixed signal acquisition section for acquiring a mixed signal being the signal based on an input signal that has a higher sampling frequency being more effective than target area sound to be extracted by the target area sound extraction means; signal mixing means for mixing a mixed signal acquired by the mixed signal acquisition means with target area sound extracted by the target area sound extraction means; and output means for outputting post-mixing signal mixed by the signal mixing means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a sound collection device, a program, and a method, and can be applied to, for example, a system that emphasizes sound in a specific area and suppresses sound in other areas.

  In an environment where a plurality of sound sources exist, as a technique for separating and collecting only sound in a specific direction, there is a beam former (hereinafter also referred to as “BF”) using a microphone array. BF is a technique for forming directivity by using a time difference between signals reaching each microphone (see Non-Patent Document 1).

  Conventionally, BFs are roughly classified into two types, namely, an addition type and a subtraction type. In particular, the subtraction type BF has an advantage that directivity can be formed with a smaller number of microphones than the addition type BF.

  FIG. 3 is a block diagram illustrating a configuration of the subtraction type BF 200 when the number of microphones M is two.

  FIG. 4 is an explanatory diagram showing an example of a directional filter formed by a subtraction type BF 200 using two microphones M1 and M2.

  The subtraction type BF 200 first calculates the time difference between the signals that arrive at the microphones M1 and M2 of the sound (hereinafter, referred to as “target sound”) present in the target direction by the delay unit 210, and adds the delay to the target. Adjust the sound phase. The above-mentioned time difference can be calculated by the following equation (1).

Here, d is the distance between the microphones M1 and M2, c is the sound speed, and τ _i is the delay amount. The theta _L is the angle from the vertical direction to the target direction against the line connecting the microphones M (M1, M2).

Also, here, with respect to the center of the blind spot is a microphone M1 M2, when present in the direction of the microphone M1, delayer 210 performs delay processing on an input signal _x 1 of the microphone M1 (t). Thereafter, in the subtraction type BF 200, the subtractor 220 performs processing (subtraction processing) according to the following equation (2).

The processing of the subtraction type BF 200 can be similarly performed in the frequency domain, and in that case, the expression (2) is changed to the following (3).

Here, when θ _L = ± π / 2, the directivity formed by the subtraction type BF 200 is a cardioid type single directivity as shown in FIG. Further, in the case of “θ _L = 0, π”, the directivity formed by the subtraction type BF 200 is a figure-eight bidirectional directivity as shown in FIG.

  Hereinafter, a filter that forms unidirectionality from an input signal is referred to as a “unidirectionality filter”, and a filter that forms bidirectionality is referred to as a bidirectional filter.

  In addition, the subtractor 220 can form a strong directivity in a bidirectional blind spot by using a spectral subtraction method (hereinafter, also simply referred to as “SS”). The directivity by the SS is formed in all frequencies or a designated frequency band according to the following equation (4).

In the following equation (4), and using the input signal X ₁ microphone M1, but it is possible to obtain the same effect input signal X ₂ microphones M2. Here, β is a coefficient for adjusting the strength of SS. When the value becomes negative at the time of subtraction, the subtractor 220 performs flooring processing of replacing it with 0 or a value obtained by reducing the original value. In the processing method of the subtraction type BF 200 as described above, a sound existing in a direction other than the target direction (hereinafter, referred to as a “non-target sound”) is extracted based on the bidirectional characteristics, and an amplitude spectrum of the extracted non-target sound is input. By subtracting from the amplitude spectrum of the signal, the target sound can be emphasized.

  When it is desired to collect only sounds existing in a specific area (hereinafter, referred to as “target area sound”), the sound of a sound source existing around that area (hereinafter, “non (Referred to as “target area sound”). Therefore, in Japanese Patent Application Laid-Open No. H11-163, a method of collecting a target area sound by using a plurality of microphone arrays and directing the directivity from different directions to the target area so that the directivity intersects the target area is described. Sound "). In the area sound collection, first, the ratio of the amplitude spectrum of the target area sound included in the BF output of each microphone array is estimated, and the ratio is used as a correction coefficient.

For example, when two microphone arrays are used, the correction coefficient of the target area sound amplitude spectrum is calculated by a combination of the following equations (5) and (6) or a combination of the following equations (7) and (8). Can be calculated. Where Y _1k (n) is the amplitude spectrum of the BF output of the first microphone array, Y _2k (n) is the amplitude spectrum of the BF output of the second microphone array, N is the total number of frequency bins, k is a frequency. Here, α ₁ (n) and α ₂ (n) are amplitude spectrum correction coefficients for each BF output. Further, here, mode represents the mode, and median represents the median.

By the above processing, the subtracter 220 calculates the correction coefficients α ₁ (n) and α ₂ (n), corrects each BF output with the obtained correction coefficients, and performs SS to obtain the non-existence in the target area direction. Extract the target area sound. Further, the subtracter 220 can extract the target area sound by applying the extracted non-target area sound from the output of each BF.

To extract the non-target area sound N ₁ (n) present in the target area direction as viewed from the first microphone array, as shown in Expression (9), the BF output Y ₁ (n) of the first microphone array. And SS multiplying the BF output Y ₂ (n) of the second microphone array by the amplitude spectrum correction coefficient α ₂ . Similarly, the non-target area sound N ₂ (n) existing in the direction of the target area viewed from the second microphone array is extracted according to the following equation (10).

Thereafter, the subtraction type BF 200 extracts the non-target area sound from each BF output according to the following equation (11) or (12) to extract the target area sound. The following equation (11) shows a process for extracting a target area sound with reference to the first microphone array. The following equation (12) shows a process for extracting a target area sound based on the second microphone array. Here, γ ₁ (n) and γ ₂ (n) are coefficients for changing the strength at the time of SS.

  By the way, when the volume level of the background noise or the non-target area sound is large, the target area sound may be distorted or annoying abnormal noise such as musical noise may occur due to SS performed at the time of extracting the target area sound.

  Therefore, in the method of Patent Document 2, the volume levels of the microphone input signal and the estimated noise are respectively adjusted according to the loudness of the background noise and the non-target area sound, and are mixed with the extracted target area sound.

  Since the musical noise generated by the process of extracting the target area sound becomes stronger as the volume level of the background noise and the non-target area sound increases, the volume level of the sum of the input signal to be mixed and the estimated noise is Increase in proportion to the volume level of the area sound.

  Therefore, in the method of Patent Document 2, the volume level of the background noise is calculated from the estimated noise obtained in the process of suppressing the background noise. Further, in the method of Patent Document 2, the volume level of the non-target area sound is divided into a non-target area sound existing in the target area direction extracted in the process of emphasizing the target area sound and a non-target area sound existing in a direction other than the target area direction. Calculated from the combined sound. Furthermore, in the method of Patent Document 2, the ratio between the input signal to be mixed and the estimated noise is determined from the volume levels of the estimated noise and the non-target area sound.

  When the non-target area sound exists near the target area, if the volume level of the input signal to be mixed is too high, the non-target area sound is mixed with the target area sound, and it is difficult to know which is the target area sound. Therefore, in the method of Patent Document 2, when the non-target area sound is loud, the volume level of the input signal to be mixed is lowered, and the volume level of the estimated noise is increased to mix. That is, in the method of Patent Document 2, when the non-target area sound does not exist or the volume level is low, the ratio of the input signal is increased, and when the non-target area sound is high, the ratio of the estimated noise is increased. Mix.

  As described above, by using the method of Patent Document 2, by mixing the input signal and the estimated noise with the target area sound, the musical noise can be masked, and the sound can be heard without discomfort like ordinary background noise. Further, according to the method of Patent Document 2, distortion of the target area sound can be corrected by the component of the target area sound included in the microphone input signal, and the sound quality can be improved.

JP 2014-072708 A JP 2017-183902 A

Tadashi Asano, "Acoustic Technology Series 16 Array Signal Processing of Sound-Localization / Tracking and Separation of Sound Sources", edited by The Acoustical Society of Japan, Corona, Feb. 25, 2011

  However, in the method of Patent Literature 2, in the processing of Patent Literature 1, since the input signal and the estimated noise are mixed with the area sound processing sound, the frequency band in which the sound quality is improved is limited. In the method of Patent Document 1, the band in which the directivity is guaranteed is limited depending on the microphone interval of the microphone array used. For example, in the method of Patent Document 1, when the microphone interval is set to 3 cm, when the frequency exceeds 6 kHz, aliasing distortion occurs, and sound outside the target area may be collected. Therefore, in the method of Patent Document 2, it is necessary to perform the processing while limiting the band to 6 kHz. Therefore, in the area sound pickup based on the method of Patent Document 2, since a component of 6 kHz or more is unnecessary, an acoustic signal is usually taken in at a sampling frequency of 16 kHz in consideration of omitting wasteful processing (in this case, , 8 kHz, but processing is not performed in the 6 kHz to 8 kHz band). That is, the sound quality improvement by the method of Patent Document 2 is up to 6 kHz (8 kHz at the maximum), and the band beyond that remains lost. Normally, there is no problem even if there is no component up to 6 kHz if the purpose is to understand the contents spoken by humans. However, when the transmission band is increased due to the progress of the communication system, the sound quality is required to be higher with the improvement of the image quality of the video, but it cannot be realized by the conventional method.

  Therefore, a sound collecting device, a program, and a method for recovering a high-band component lost by the area sound collecting process and improving sound quality are desired.

  According to a first aspect of the present invention, (1) beamformer outputs of the respective microphone arrays are obtained based on input signals input from a plurality of microphone arrays, and a target area is set as a sound source using the obtained beamformer outputs. Target area sound extracting means for extracting a target area sound; and (2) a mixed signal which is a signal based on the input signal and which has a higher effective sampling frequency than the target area sound extracted by the target area sound extracting means. (3) signal mixing means for mixing the mixed signal obtained by the mixed signal obtaining means with the target area sound extracted by the target area sound extracting means, and (4) the signal Output means for outputting a mixed signal after the mixing means has been mixed.

  A sound collection program according to a second aspect of the present invention provides a computer which (1) acquires a beamformer output of each microphone array based on input signals inputted from a plurality of microphone arrays, and uses the acquired beamformer output. And (2) a signal based on the input signal, which is more effective than the target area sound extracted by the target area sound extracting means. Mixed signal obtaining means for obtaining a mixed signal having a high sampling frequency; and (3) signal mixing means for mixing the mixed signal obtained by the mixed signal obtaining means with the target area sound extracted by the target area sound extracting means. And (4) functioning as output means for outputting the mixed signal mixed by the signal mixing means.

  According to a third aspect of the present invention, there is provided a sound collecting method performed by the sound collecting apparatus, comprising (1) a target area sound extracting unit, a mixed signal acquiring unit, a signal mixing unit, and an output unit; and (2) the target area sound extracting unit. The means obtains a beamformer output of each of the microphone arrays based on input signals input from a plurality of microphone arrays, and extracts a target area sound having a target area as a sound source using the obtained beamformer outputs, (3) The mixed signal acquiring means acquires a mixed signal having a higher effective sampling frequency than the target area sound extracted by the target area sound extracting means, the mixed signal being a signal based on the input signal. The signal mixing means mixes the mixed signal acquired by the mixed signal acquiring means with the target area sound extracted by the target area sound extracting means, and Output means, and outputs the mixed signal after said signal mixing means is mixed.

  According to the present invention, it is possible to provide a sound collecting device, a program, and a method for improving a sound quality by reducing a component lost in an area sound collecting process.

FIG. 2 is a block diagram illustrating a functional configuration of the sound collection device according to the first embodiment. It is a block diagram showing a functional configuration of a sound collection device according to a second embodiment. FIG. 13 is a block diagram illustrating a configuration related to a conventional subtraction type BF when the number of microphones is two. FIG. 10 is a diagram illustrating a directional characteristic formed by a conventional subtraction-type BF using two microphones.

(A) First Embodiment Hereinafter, a first embodiment of a sound collection device, a program, and a method according to the present invention will be described in detail with reference to the drawings.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing a functional configuration of a sound collection device 100 of this embodiment.

  The sound pickup device 100 performs a target area sound pickup process of picking up a target area sound from a sound source in the target area using two microphone arrays MA (MA1, MA2).

  The microphone arrays MA1 and MA2 are arranged at any location in the air where the target area exists. The positions of the microphone arrays MA1 and MA2 with respect to the target area may be anywhere as long as the directivity overlaps only in the target area. For example, the microphone arrays MA1 and MA2 may be arranged to face each other across the target area. Each microphone array MA includes two or more microphones M, and each microphone M collects an acoustic signal. In this embodiment, a description will be given assuming that two microphones M (M1, M2) that collect sound signals are arranged in each microphone array MA. That is, each microphone array MA constitutes a 2ch microphone array. Note that the number of microphone arrays MA is not limited to two, and when there are a plurality of target areas, it is necessary to arrange the number of microphone arrays MA that can cover all the areas.

  In the example of this embodiment, it is assumed that the number of microphones is two and the distance between the two microphones is 3 cm. The distance between the microphones is not limited.

  The sound collection device 100 includes a sampling frequency setting unit 101, a sound collection processing signal input unit 102, a mixing signal input unit 103, a noise suppression unit 104, a directivity forming unit 105, a delay correction unit 106, spatial coordinate data 107, a correction It has a coefficient calculation unit 108, a target area sound extraction unit 109, a signal mixing unit 110, and a signal output unit 111.

  Detailed processing of each functional block configuring the sound collection device 100 will be described later.

  The sound collection device 100 may be entirely configured by hardware (for example, a dedicated chip or the like), or may be partially or entirely configured as software (program). The sound collection device 100 may be configured by installing a program (including the determination program and the sound collection program of the embodiment) in a computer having a processor and a memory, for example.

(A-2) Operation of First Embodiment Next, an operation (a sound collection method according to the embodiment) of the sound collection device 100 according to the first embodiment having the above-described configuration will be described.

  The sampling frequency setting unit 101 sets a sampling frequency for each use (type) for an input acoustic signal (input signal input from each of the microphones M1 and M2 of each of the microphone arrays MA1 and MA2) and outputs the same. . Here, the sampling frequency setting unit 101 includes a signal for the sound collection processing signal input unit 102 (hereinafter, referred to as an “area sound collection processing signal”) and a signal for the mixing signal input unit 103 (hereinafter, “signal”). The sampling frequency is set for each of the “mixing signals”.

  For example, the sampling frequency setting unit 101 may set the sampling frequency of the mixing signal to be higher than the sampling frequency of the area sound collection processing signal. Specifically, for example, the sampling frequency setting unit 101 may set the sampling frequency of the area sound collection processing signal to 16 kHz and the sampling frequency of the mixing signal to 48 kHz. Note that the sampling frequency of the area sound pickup processing signal is a frequency corresponding to the upper limit of the band (band of the target area sound to be extracted) processed in the area sound pickup processing in the target area sound extraction unit 109 described later. . For example, when the upper limit of the band processed in the area sound pickup processing in the target area sound extraction unit 109 is 8 kHz, the sampling frequency of the area sound pickup processing signal is 16 kHz.

  The sound collection processing signal input unit 102 is configured to input an acoustic signal (input signals input from the microphones M1 and M2 of the microphone arrays MA1 and MA2) based on the sampling frequency set by the sampling frequency setting unit 101. , Converting an analog signal to a digital signal. Then, the sound collection signal input unit 102 converts the converted digital signal (for example, from the time domain to the frequency domain using fast Fourier transform).

  Based on the sampling frequency set by the sampling frequency setting unit 101, the mixing signal input unit 103 converts the input audio signal (input signals input from the microphones M1 and M2 of the microphone arrays MA1 and MA2) into analog signals. Converts a signal to a digital signal. Then, the mixing signal input unit 103 converts the converted digital signal (for example, from the time domain to the frequency domain using fast Fourier transform).

  The noise suppression unit 104 estimates and suppresses a background noise component included in the signal acquired by the sound collection processing signal input unit 102 or the mixing signal input unit 103. That is, the noise suppression unit 104 performs noise suppression processing on the area sound collection processing signal supplied from the sound collection signal input unit 102, and supplies the signal to the directivity forming unit 105. Further, the noise suppression unit 104 performs a noise suppression process on the mixing signal supplied from the mixing signal input unit 103 and supplies the signal to the signal mixing unit 110. Although the specific method of the noise suppression processing performed by the noise suppression unit 104 is not limited, for example, SS and Wiener Filtering (Wiener Filtering) can be used.

  In the sound collecting apparatus 100, when the environment is a quiet environment with almost no background noise (when the input signal is background noise), the noise suppression unit 104 may be omitted. Further, in the sound pickup apparatus 100, when the noise suppression processing is not performed (when the noise suppression unit 104 is excluded), the mixing signal input unit 103 does not convert the signal from the time domain to the frequency domain, and The area may be left as it is.

  The directivity forming unit 105 outputs a signal to the area sound processing signal supplied to each microphone array (in the example of this embodiment, the area sound processing signal whose background noise is suppressed by the noise suppressing unit 104). A signal having directivity in the area direction is obtained. Specifically, the directivity forming unit 105 obtains a signal (BF output) in which the directivity is formed in the direction of the target area by the BF for each of the microphone arrays (MA1, MA2) according to Expression (4).

  The delay correction unit 106 calculates and corrects a delay generated due to a difference in the distance between the target area and each microphone array. First, the position of the target area and the position of the microphone array are acquired from the spatial coordinate data 107, and the difference between the arrival time of the target area sound to each microphone array is calculated. Next, the delay correction unit 106 adds a delay based on the microphone array located farthest from the target area so that the sound of the target area reaches all the microphone arrays at the same time.

  In the case where no delay occurs from the beginning due to the arrangement of the microphone array and the target area sound, the sound collection device 100 may be configured to exclude the delay correction unit 106.

  The spatial coordinate data 107 holds all target areas, each microphone array, and positional information of the microphones constituting each microphone array. The method by which the spatial coordinate data 107 holds the position information of each microphone of each microphone array, and the specific format of the position information held by the spatial coordinate data 107 are not limited, and various data formats may be applied. it can.

  The correction coefficient calculation unit 108 calculates a correction coefficient for equalizing the amplitude spectrum of the target area sound component included in each BF output according to the formulas (5) and (6) or the formulas (7) and (8).

  If it is clear that the difference between the amplitude spectrum of the target area sound of each microphone is small due to the arrangement of the microphone array and the target area sound, the sound collection device 100 may be configured to exclude the correction coefficient calculation unit 108.

  The target area sound extraction unit 109 SSs each BF output data corrected by the correction coefficient calculated by the correction coefficient calculation unit 108 according to the equation (9) or (10), and removes the non-target area sound existing in the target area direction. Extract. Furthermore, the target area sound extraction unit 109 extracts the target area sound by subjecting the extracted noise to SS from the output of each BF according to the formula (10) or (11).

  Then, the target area sound extraction unit 109 performs upsampling on the extracted target area sound at the sampling frequency set for the mixing signal by the sampling frequency setting unit 101 (the acoustic signal raised to the same sampling frequency as the mixing signal). Is performed. Then, the target area sound extraction unit 109 supplies the upsampled target area sound to the signal mixing unit 110. As a result, in the signal mixing section 110, the target area sound becomes an audio signal in a format that can be easily mixed with the mixing signal. Note that the specific processing method of the upsampling by the target area sound extraction unit 109 is not limited, and various methods can be applied.

  The signal mixing section 110 sets the sampling frequency of the up-sampled target area sound supplied from the target area sound extraction section 109 (the sampling frequency of the target area sound extracted by the target area sound extraction section 109 to be the same as the mixing signal). And a signal supplied from the noise suppression unit 104 (mixing signal after noise suppression processing). The details of the mixing process by the signal mixing unit 110 are not limited. The signal mixing unit 110 may, for example, mix the two signals in the frequency domain and then convert the two signals to the time domain, or mix the two signals after converting the frequency domain to the time domain.

  The signal output unit 111 outputs the signal mixed and processed by the signal mixing unit 110 as a final output signal.

  As described above, the sound collection device 100 uses a signal having a higher sampling frequency than the area sound collection processing as a mixed signal.

  As described above, in the sound collection device 100 of the first embodiment, a signal having a sampling frequency higher than that of the area sound collection processing is used as a mixing signal, and mixed with the area sound collection processing sound. Further, in the sound collecting apparatus 100 of the first embodiment, in the area sound collecting processing, an acoustic signal is captured at a sampling frequency at which aliasing does not occur as in the related art. Furthermore, in the sound collection device 100 of the first embodiment, a sound signal is taken in as a mixing signal at a higher sampling frequency than in the area sound collection processing. Furthermore, in the sound pickup apparatus 100 of the first embodiment, after the area sound pickup processing is completed, the signal is up-sampled to have the same sampling frequency as the mixing signal. Then, in the sound pickup apparatus 100 of the first embodiment, the signal after the area sound pickup processing and the mixing signal are mixed and output.

(A-3) Effects of First Embodiment According to the first embodiment, the following effects can be obtained.

  In the sound pickup apparatus 100 of the first embodiment, a signal having a higher sampling frequency than the area sound pickup processing is used as a mixed signal, thereby recovering a high-band component lost by the area sound pickup processing and improving sound quality. can do.

(B) Second Embodiment Hereinafter, a second embodiment of a sound collection device, a program, and a method according to the present invention will be described in detail with reference to the drawings.

(B-1) Configuration of Second Embodiment FIG. 3 is a block diagram showing a functional configuration of a sound collection device 100A according to a second embodiment. Have the same or corresponding reference numerals.

  Hereinafter, differences between the second embodiment and the first embodiment will be described.

  The sound collection device 100A of the second embodiment is different from the first embodiment in that a mixed band selection unit 112 is added.

(B-2) Operation of the Second Embodiment Next, the operation (sound collection method according to the embodiment) of the sound collection device 100A of the second embodiment having the above-described configuration will be described in the first embodiment. The following description focuses on the differences from.

  The mixing band selection unit 112 converts the signal supplied from the preceding stage (the noise suppression unit 104 in the example of this embodiment) (the mixing signal after noise suppression processing in the example of this embodiment) into a target area sound extraction unit. A process of selecting a band (hereinafter, referred to as a “mixing band”) to be mixed with the signal processed in step 109 and extracting only the mixing band from the supplied mixing signal (or suppressing a region other than the mixing band) (hereinafter, “filter”) The signal (hereinafter, also referred to as “filtered mixing signal”) subjected to the “processing” is acquired and supplied to the signal mixing unit 110.

  The range of the mixing zone is not limited. The mixed band may be, for example, a whole band (a whole band corresponding to the sampling frequency), a band of a predetermined frequency ω1 or more (a band passed by processing of a high-pass filter), or a band of a predetermined frequency ω2 or less ( The band may be a band that is passed through the processing of the low-pass filter) or may be a band that is equal to or higher than the predetermined lower limit frequency ω3 and equal to or lower than the upper limit frequency ω4 (the band that is passed by the band-pass filter processing). Further, the mixed band selection unit 112 may perform signal processing using a plurality of filters (for example, processing combining a plurality of bandpass filters). Further, the mixed band selection unit 112 may perform filter processing (band control processing) in the frequency domain, or may perform signal filter processing (band control processing) in the time domain.

  In addition, the mixed band selection unit 112 may set a preset band as the mixed band, or set a dynamic band (for example, to a band corresponding to a supplied signal or other factors). Setting). More specifically, for example, in the mixed band selection unit 112, the quality of the output signal (the quality according to the purpose of use of the output signal) may be deteriorated if the mixed signal is mixed with the signal finally output by the sound pickup device 100A. A band excluding the band may be set as the mixed band.

  Then, the signal mixing section 110 mixes and outputs the signal after the area sound collection processing and the mixed signal (filtered mixing signal) processed by the mixing band selection section 112.

(B-3) Effects of the Second Embodiment According to the second embodiment, the following effects can be obtained as compared with the effects of the first embodiment.

  In the sound collection device 100A (signal mixing unit 110) of the second embodiment, the signal after the area sound collection processing is mixed with the mixed signal (filtered mixing signal) processed by the mixing band selection unit 112. Output. Thereby, in the sound collection device 100A of the second embodiment, there is an effect that the quality of the output signal finally output is improved (deterioration is suppressed). For example, in the sound collection device 100A of the second embodiment, by setting a band excluding a band in which the quality of an output signal may be deteriorated when mixed into a signal to be finally output, as a mixed band, This has the effect of improving the quality of the output signal output to the device (suppressing deterioration).

(C) Other Embodiments The present invention is not limited to the above embodiments, but may include modified embodiments as exemplified below.

  (C-1) In the sound collection device of each of the above embodiments, the number of microphones of each microphone array MA used for sound collection is two, but the number of microphones used for sound collection is three or more. The sound in the direction of the target area may be collected based on the sound. In each of the above embodiments, various existing systems can be applied to the number of microphones for each microphone array MA to be applied and the system for collecting sound in the target sound direction.

  (C-2) In the sampling frequency setting unit 101 of each of the above embodiments, the sampling frequency of the area sound collection processing signal and the sampling frequency of the mixing signal may be the same. In this case, the target area sound extraction unit 109 limits the band in which the area sound pickup processing is performed (the band of the target area sound to be extracted; hereinafter, referred to as the “area sound pickup processing band”) to a band in which aliasing distortion does not occur. Alternatively, a signal in which data of “0” is set as a component of a frequency band higher than the area sound collection processing band may be output as a signal after the area sound collection processing (extracted target area sound).

  In this case, the signal output from the target area sound extraction unit 109 (the extracted target area sound) is a signal having the same sampling frequency as the mixing signal in terms of external shape, but the effective sampling frequency is the area sound collection processing band. (Hereinafter, referred to as "area sound collection processing effective sampling frequency"). As a result, the effective sampling frequency of the target area sound extracted by the target area sound extraction unit 109 (the effective sampling frequency of the area pickup processing) is lower than the frequency of the mixed signal. For example, the sampling frequency of the area sound pickup processing signal and the mixing signal may be set to 48 kHz, and the upper limit of the area sound pickup processing band may be set to 8 kHz (a frequency corresponding to the case where the area sound pickup processing effective sampling frequency is set to 16 kHz).

  In this case, naturally, the target area sound extraction unit 109 supplies the signal after the area sound collection processing (the extracted target area sound) to the signal mixing unit 110 without performing upsampling processing. , A process of mixing with the mixing signal.

  Reference numeral 100: sound collection device, M1, M2: microphone, MA1, MA2: microphone array, sampling frequency setting unit 101, sound collection processing signal input unit 102, mixing signal input unit 103, noise suppression unit 104, directivity forming unit 105, delay correction unit 106, spatial coordinate data 107, correction coefficient calculation unit 108, target area sound extraction unit 109, signal mixing unit 110, and signal output unit 111.

According to a first aspect of the present invention, (1) a beamformer output of each of the microphone arrays is obtained based on input signals input from a plurality of microphone arrays, and a target area is determined as a sound source by using the obtained beamformer outputs. Target area sound extracting means for extracting a target area sound to be reproduced; and (2) a mixed signal which is a signal based on the input signal and which has a higher effective sampling frequency than the target area sound extracted by the target area sound extracting means. (3) a signal mixing means for mixing the target area sound extracted by the target area sound extraction means with the mixed signal obtained by the mixing signal obtaining means; Output means for outputting the mixed signal after the mixing by the signal mixing means.

A sound collection program according to a second aspect of the present invention provides a computer which (1) acquires beamformer outputs of the respective microphone arrays based on input signals input from a plurality of microphone arrays, and outputs the acquired beamformer outputs. A target area sound extracting means for extracting a target area sound using the target area as a sound source, and (2) a signal based on the input signal, which is more effective than the target area sound extracted by the target area sound extracting means. Signal mixing means for obtaining a mixed signal having a high sampling frequency, and (3) signal mixing for mixing the mixed signal obtained by the mixed signal obtaining means with the target area sound extracted by the target area sound extracting means. And (4) functioning as output means for outputting the mixed signal mixed by the signal mixing means.

According to a third aspect of the present invention, there is provided a sound collecting method performed by the sound collecting apparatus, comprising (1) a target area sound extracting unit, a mixed signal acquiring unit, a signal mixing unit, and an output unit; and (2) the target area sound extracting unit. means, based on the input signals input from a plurality of microphone array to obtain the beamformer output of each of the microphone array, a target area extracting a target area sound the sound source using the acquired beamformer output (3) the mixed signal obtaining means obtains a mixed signal having a higher effective sampling frequency than the target area sound extracted by the target area sound extracting means, the mixed signal being a signal based on the input signal; (5) The signal mixing means mixes the target area sound extracted by the target area sound extraction means with the mixed signal acquired by the mixed signal acquisition means, and (5) Serial output means, and outputs the mixed signal after said signal mixing means is mixed.

Claims (5)

Target area sound extraction for obtaining a beamformer output of each of the microphone arrays based on input signals input from a plurality of microphone arrays, and extracting a target area sound having a target area as a sound source using the obtained beamformer outputs. Means,
A mixed signal acquisition unit that is a signal based on the input signal and acquires a mixed signal having a higher effective sampling frequency than the target area sound extracted by the target area sound extraction unit;
A signal mixing unit that mixes the mixed signal acquired by the mixed signal acquisition unit with the target area sound extracted by the target area sound extraction unit,
Output means for outputting a mixed signal mixed by the signal mixing means.   The signal mixing unit mixes the target area sound extracted by the target area sound extraction unit with the mixed signal after performing an up-sampling process at the same sampling frequency as the mixed signal acquired by the mixed signal acquisition unit. The sound pickup device according to claim 1, wherein Mixing band selection means for obtaining a filtered mixed signal subjected to a filtering process for selecting and extracting a component of a part of the frequency band of the mixed signal obtained by the mixed signal obtaining means,
The said signal mixing means mixes the said filtered processed mixed signal which the said mixing band selection means extracted with the said target area sound extracted by the said target area sound extraction means. The said 1 or 2 characterized by the above-mentioned. A sound pickup device as described. Computer
Target area sound extraction for obtaining a beamformer output of each of the microphone arrays based on input signals input from a plurality of microphone arrays, and extracting a target area sound having a target area as a sound source using the obtained beamformer outputs. Means,
A mixed signal acquisition unit that is a signal based on the input signal and acquires a mixed signal having a higher effective sampling frequency than the target area sound extracted by the target area sound extraction unit;
A signal mixing unit that mixes the mixed signal acquired by the mixed signal acquisition unit with the target area sound extracted by the target area sound extraction unit,
A sound collection program, which functions as output means for outputting a mixed signal mixed by the signal mixing means. In the sound pickup method performed by the sound pickup device,
Having a target area sound extraction means, a mixed signal acquisition means, a signal mixing means, and an output means,
The target area sound extraction means obtains a beamformer output of each of the microphone arrays based on input signals input from a plurality of microphone arrays, and uses the obtained beamformer output to set a target area as a sound source to the target area. Extract sound,
The mixed signal acquiring unit is a signal based on the input signal, and acquires a mixed signal having a higher effective sampling frequency than the target area sound extracted by the target area sound extracting unit,
The signal mixing unit mixes the mixed signal acquired by the mixed signal acquisition unit with the target area sound extracted by the target area sound extraction unit,
The sound pickup method, wherein the output means outputs a mixed signal mixed by the signal mixing means.

Images