JPWO2019003716A1 - Sound collecting device, directivity control device, and directivity control method - Google Patents

Abstract

Achieves a good signal-to-noise ratio when providing directivity with a plurality of microphones. In order to provide directivity in a specific direction expressed in N dimensions, N+2 or more omnidirectional microphones 21a, 21b..., A beam former 32, and a noise processing unit 33 are provided. The beam former 32 generates the first directivity signal S1 in the specific direction from the N+1 sound collection signals Sa, Sb,... A second directivity signal S2 in a specific direction is generated from N+1 sound collection signals Se, Sf,... Which are a combination different from the sound collection signals Sa, Sb. The noise processing unit 33 suppresses uncorrelated sound components between the two types of directional signals S1 and S2.

Description

The present invention relates to a directivity control that directs an arbitrary direction.

Various techniques have been proposed in which ambient sound is collected by a plurality of microphones and a sound wave signal having directivity in a specific direction is generated from the plurality of sound collection signals. There are individual differences among multiple microphones, and uncorrelated noise such as circuit noise and wind noise unique to the microphones occurs. Therefore, methods for suppressing these noises are being studied.

As a method of suppressing uncorrelated noise, there is a spectral subtraction method using a nonlinear algorithm (see, for example, Patent Document 1). The spectral subtraction method is a method of estimating the average value of the power spectrum of noise and subtracting it from the power spectrum of the input signal containing noise to reduce the noise.

Further, there is known a method of controlling directivity by using an ambisonic microphone (Ambisonic Microphones) and adding or subtracting recorded signals during reproduction (see, for example, Non-Patent Document 1). The ambisonic microphone is a microphone unit in which four microphones facing outward from each vertex of a regular tetrahedron are arranged. This microphone unit includes a directional microphone and a space portion at the center surrounded by the directional microphone.

In this directivity control method using an ambisonic microphone, first, based on the sound collection signals output by the four unidirectional microphones, the signal is converted into a signal expression called B format. The signal expression called B format includes a non-directional sound collection signal of 0th order and bidirectional sound collection signals of up, down, left, right, and front called first order. Then, the 0th-order sound collection signal and the 1st-order circumferential sound signals are added together at a ratio suitable for a specific direction in which directivity is given. As a result, directivity is imparted in a specific direction using the ambisonic microphone.

Here, in the ambisonic microphone, it is necessary that the phase difference of the sound collection signals is accurately held. This is because if the phase difference of the sound collection signal changes, the effect of emphasizing or attenuating the signal cannot be expected in the calculation during reproduction originally. Therefore, in the sound collection method using the ambisonic microphone, it is difficult to suppress uncorrelated noise by signal processing. Therefore, when the ambisonic microphone is used, the four unidirectional microphones are carefully selected so that the individual difference among the four unidirectional microphones is reduced.

S.F.Boll, ``Suppression of Acoustic Noise in Speech Using Spectral Subtraction'', IEEE Trans.ASSP., April 2, 1979, Vol.27, pp.113-120 DG Malham, "SPATIAL HEARING MECHANISMS and SOUND REPRODUCTION", [online], May 6, 2008, University of York, [June 23, 2017 search], Internet <URL: https://www.york. ac.uk/inst/mustech/3d_audio/ambis2.htm〉

If you try to suppress uncorrelated noise by the spectral subtraction method that uses a non-linearity algorithm, the behavior of each channel is processed individually. , The signal cannot be emphasized or attenuated even if an attempt is made to add or subtract signals having different phase differences to obtain directivity, and the directivity is affected. Therefore, noise cannot be suppressed for each microphone signal of the ambisonic microphone, and the SN ratio cannot be improved.

Further, in the unidirectional microphone, the wind may enter from the acoustic port on the back surface, and turbulent airflow is generated to increase wind noise. Therefore, the noise caused by the wind noise cannot be sufficiently eliminated by the ambisonic microphone, and the SN ratio deteriorates. In addition, if four unidirectional microphones are selected so that the individual difference between the four unidirectional microphones is reduced, a great cost is incurred. The SN ratio deteriorates due to circuit noise or the like peculiar to the unidirectional microphone.

The present invention has been made to solve the above-mentioned problems of the prior art, and an object thereof is to achieve a good SN ratio when directivity is provided using a plurality of microphones. It is to provide a sound collecting device that can perform.

In order to achieve the above-mentioned object, a sound collecting device according to the present invention is a sound collecting device that directs a specific direction expressed in N dimensions and measures a surrounding sound and outputs a sound collecting signal. And at least N+2 omnidirectional microphones, and a beamformer for generating two types of directional signals having unidirectionality in a specific direction based on the sound collection signals output by the omnidirectional microphones; A noise processing unit that suppresses a noise component based on various types of directional signals, and the beam former is configured to output a single signal in a specific direction based on N+1 sound collection signals selected from the outputs of the omnidirectional microphones. The first directivity signal having directivity is generated, and based on N+1 sound collection signals in a combination different from the N+1 sound collection signals used to generate the first directivity signal, The second directional signal having unidirectionality in the same specific direction as the first directional signal is generated, and the noise processing section suppresses a non-correlated sound component between the two types of directional signals. It is characterized by

The noise processing unit may further include a sensitivity correction unit that equalizes the sensitivity of the sound collection signals output from the at least N+2 omnidirectional microphones, and the noise processing unit may provide a non-existence between the two types of directional signals that have been corrected by the sensitivity correction unit. You may make it suppress the sound component of correlation.

Three-dimensional directivity is provided, and at least five or more omnidirectional microphones are provided, and the beam former is based on four sound collection signals selected from the outputs of the omnidirectional microphones. A first directivity signal having a single directivity in a specific direction, and four collections of a combination different from the four sound collection signals used to generate the first directivity signal. A second directional signal having unidirectionality in the same specific direction as the first directional signal may be generated based on the sound signal.

The omnidirectional microphones are three-dimensionally arranged so that two kinds of combinations of the omnidirectional microphones facing the respective vertex directions of the tetrahedron can be selected, and the two kinds of tetrahedrons are not congruent or similar. The positions may be different, the orientations may be different, or a combination of these may be provided.

Two-dimensional directivity is provided, and at least four or more omnidirectional microphones are provided, and the beamformer is based on three sound collection signals selected from the outputs of the omnidirectional microphones. Generating a first directional signal having a unidirectionality in a specific direction, and collecting three signals in a combination different from the three sound collecting signals used to generate the first directional signal. A second directional signal having unidirectionality in the same specific direction as the first directional signal may be generated based on the sound signal.

The beamformer may further include an input unit that receives an input in the specific direction, and the beam former may form directivity in the specific direction received by the input unit.

A panoramic image pickup unit that has a plurality of cameras and captures each direction of the surroundings is further provided, and the at least N+2 omnidirectional microphones are distributed between the plurality of cameras or a user grips supporting the plurality of cameras. It may be attached to a rod for use.

The noise processing unit alternately replaces the first directional signal and the second directional signal for each sample to generate a pair of exchange signals, and a coefficient for one of the exchange signals. After multiplying m, an error signal generator that generates an error signal of the exchange signal, and a recurrence formula calculation for calculating a recurrence formula of the coefficient m including the error signal and updating the coefficient m for each sample A unit and a summing unit that multiplies the first directivity signal or the second directivity signal by the coefficient m that is sequentially updated and outputs the result.

In order to achieve the above-mentioned object, the directivity control device according to the present invention is a directivity control device that gives directivity to a specific direction expressed in N dimensions, and has at least N+2 omnidirectional microphones. A storage unit for storing the respective sound collection signals generated by the storage unit, a beamformer for generating two kinds of directional signals having a single directivity in a specific direction based on the sound collection signals of the storage unit, A noise processing unit that suppresses a noise component based on a directivity signal; and the beamformer has a unidirectionality in a specific direction based on N+1 sound collection signals selected from the storage unit. Of the first directivity signal based on a combination of N+1 sound collection signals that are different from the N+1 sound collection signals used to generate the first directivity signal. A second directivity signal having a unidirectionality in the same specific direction as the above, and the noise processing unit suppresses an uncorrelated sound component between the two types of directivity signals. ..

The noise processing unit further includes a sensitivity correction unit that equalizes the sensitivities of the at least N+2 collected signals, and the noise processing unit suppresses uncorrelated sound components between the two types of directional signals that have been corrected by the sensitivity correction unit. You may do it.

The noise processing unit alternately replaces the first directional signal and the second directional signal for each sample to generate a pair of exchange signals, and a coefficient for one of the exchange signals. After multiplying m, an error signal generator that generates an error signal of the exchange signal, and a recurrence formula calculation for calculating a recurrence formula of the coefficient m including the error signal and updating the coefficient m for each sample A unit and a summing unit that multiplies the first directivity signal or the second directivity signal by the coefficient m that is sequentially updated and outputs the result.

In order to achieve the above-mentioned object, the directivity control device according to the present invention is a directivity control method for giving directivity to a specific direction expressed in N dimensions, and at least N+2 non-directional microphones. A beamforming step of generating two types of directional signals having a unidirectionality in a specific direction based on each of the collected sound signals generated by, and noise suppression for suppressing a noise component based on the two types of directional signals The beamforming step generates a first directional signal having unidirectionality in a specific direction based on at least N+1 sound collection signals of at least N+2 sound collection signals. At the same time, based on N+1 sound collection signals in a combination different from the N+1 sound collection signals used to generate the first directional signal, a single signal is generated in the same specific direction as the first directional signal. A second directivity signal having directivity is generated, and the noise suppressing step suppresses a sound component that is uncorrelated between the two types of directivity signals.

The noise suppression step further includes a sensitivity correction step of equalizing the sensitivities of the at least N+2 sound collection signals, and the noise suppression step suppresses an uncorrelated sound component between the two types of directional signals corrected by the sensitivity correction step. You may do it.

The noise processing step includes an exchange step of generating a pair of exchange signals by alternately exchanging the first directional signal and the second directional signal for each sample, and a coefficient for one of the exchange signals. An error signal generation step of generating an error signal of the exchange signal after multiplying m, and a recurrence formula operation of calculating a recurrence formula of the coefficient m including the error signal and updating the coefficient m for each sample A step and an integrating step of multiplying the first directivity signal or the second directivity signal by the coefficient m that is sequentially updated and outputting the result may be provided.

ADVANTAGE OF THE INVENTION According to this invention, the noise component contained in a 1st directional signal and a 2nd directional signal can be made uncorrelated, and a favorable SN ratio can be achieved by the process which suppresses a non-correlated component. You can

It is a block diagram which shows the whole structure of a sound collection device. It is a schematic diagram which shows a microphone unit. It is a model which shows the example which selects two types of microphone groups from a microphone unit. It is a block diagram which shows the detailed structure of a directivity control apparatus. It is a schematic diagram which shows the original polar pattern of 6 omnidirectional microphones. It is a block diagram which shows the detailed structure of a sensitivity correction part. It is a schematic diagram which shows the control point set around four omnidirectional microphones regarding a multipoint control method. It is a block diagram which shows the structure of a beam former. It is a graph which shows two directional signals which a beam former outputs. It is a block diagram which shows the structural example of a noise processing part. It is a schematic diagram which shows the 1st application example of a sound collection device, (a) is a perspective view, (b) is a side view, (c) is a bottom view. It is a schematic diagram which shows the 2nd application example of a sound collection device, (a) is a perspective view, (b) is a side view, (c) is a bottom view. It is a schematic diagram which shows the 3rd application example of a sound collection device, (a) is a perspective view, (b) is a side view, (c) is a bottom view.

(Constitution)
FIG. 1 is a block diagram showing the overall configuration of the sound collector. The sound collection device 1 includes a microphone unit 2 and a directivity control device 3. This sound collecting device 1 retroactively adds directivity to a specific direction selected by the user after recording ambient sounds. The microphone unit 2 and the directivity control device 3 each have a power source and can be driven separately.

The user carries around the microphone unit 2 to collect sound, and connects the microphone unit 2 to the directivity control device 3 at home or in the studio. The microphone unit 2 transfers the sound collection result to the directivity control device 3, and the directivity control device 3 accepts the setting of the specific direction from the user, and directs the sound collection result in the specific direction. Signal processing.

The microphone unit 2 includes a plurality of omnidirectional microphones 21a, 21b... And a nonvolatile memory 22 such as a flash memory. The microphone unit 2 collects ambient sounds in multiple channels and records each of the collected sound signals Sa to Sf as raw data. The sound collection signals Sa to Sf are formed by synthesizing sound components arriving from the surrounding directions.

When the omnidirectional microphones 21a, 21b,... Are analog outputs, the microphone unit 2 has an amplifier for amplifying the respective sound collecting signals Sa to Sf and converts the sound collecting signals Sa to Sf from analog signals to digital signals. A converter is additionally provided, and the digital sound collection signal that has passed through the amplifier and the converter is recorded in the non-volatile memory 22.

The directivity control device 3 is a dedicated circuit including a microcomputer, a computer, or a DSP that digitally processes the sound collection signals Sa to Sf. The directivity control device 3 stores the sound collection signals Sa to Sf transferred from the microphone unit 2 in a memory 34 such as a hard disk, an SSD or an internal storage in the device. Then, the directivity control device 3 performs signal processing on each of the sound collection signals Sa to Sf to clarify the sound component coming from the specific direction, and circuit noise and wind noise of the omnidirectional microphones 21a, 21b,. To generate the output signal So. The output signal So may be multi-channel according to the number of speakers to be output.

That is, the directivity control device 3 relatively emphasizes the sound component coming to the microphone unit 2 from the specific direction, compared with the sound component coming from the direction other than the specific direction. In other words, the directivity control device 3 largely suppresses the sound component arriving at the microphone unit 2 from a direction other than the specific direction, as the difference between the direction of arrival and the specific direction increases. Further, the directional control device 3 suppresses noise such as circuit system noise and wind noise caused by individual differences of the omnidirectional microphones 21a, 21b,..., And improves the SN ratio.

(Microphone unit)
As shown in FIGS. 2 and 3, the microphone unit 2 includes a cylindrical support member 23. The six omnidirectional microphones 21a to 21f are attached to the surface of the support member 23 while being equally spaced from the other adjacent omnidirectional microphones 21a to 21f. For the omnidirectional microphones 21a, 21b,..., the front of the diaphragm may be opened to the sound field.

Therefore, no space is required behind the omnidirectional microphones 21a, 21b,... Therefore, even if the support member 23 is solid, even if other parts are arranged inside the support member 23 at a position surrounded by the omnidirectional microphones 21a, 21b... The inside and outside do not have to communicate. Further, the supporting member 23 is a cylinder because the orientations of the omnidirectional microphones 21a to 21f are easy to set, but the supporting member 23 may have any shape.

As shown in FIG. 3A, consider a regular hexagonal column inscribed in the support member 23. At this time, the six omnidirectional microphones 21a to 21f are distributed and arranged at six positions of the corners of the regular hexagonal prism. The respective distribution positions are selected so as not to be adjacent to each other with one side of the regular hexagonal column interposed therebetween. That is, around the axis of the regular hexagonal column, select three corners from one bottom surface at intervals of 120 degrees and select three corners from the other bottom surface at intervals of 120 degrees. The three locations on the bottom surface of the above and the other three locations on the other bottom surface are offset by 60 degrees around the axis of the regular hexagonal prism.

This microphone arrangement complies with the following rules. That is, first, omnidirectional microphones 21a, 21b,... Next, as shown in (b) and (c) of FIG. 3, the microphone unit 2 selects two kinds of combinations of four omnidirectional microphones 21a, 21b... Which face the respective vertex directions of the tetrahedron. As possible, at least five or more omnidirectional microphones are three-dimensionally arranged and provided. The two types of tetrahedra are not congruent or similar, have different positions, have different orientations, or have a combination of these. The tetrahedron is preferably a regular tetrahedron, but is not limited to this. In the present embodiment, the omnidirectional microphones 21a, 21c, 21d, 21e are a combination group A arranged in one tetrahedron, and the omnidirectional microphones 21a, 21b, 21d, 21f are arranged in another tetrahedron. It is a combination group B to be arranged.

It is better that there are fewer omnidirectional microphones 21a, 21b... That belong to two types of tetrahedrons, and preferably the microphone unit 2 has eight omnidirectional microphones 21a, 21b. For example, around the axis of a regular octagonal prism, select four corners at intervals of 90 degrees from one bottom surface, and select four corners at intervals of 90 degrees from the other bottom surface. The four locations on the bottom surface of the and the four locations on the other bottom surface are offset by 45 degrees about the axis of the regular octagonal prism. The omnidirectional microphones 21a, 21b... Are arranged at these selection positions so that the omnidirectional microphones 21a, 21b.. Two types of groups of sex microphones 21a, 21b... Can be created.

The rule that the number of omnidirectional microphones 21a, 21b,... Is five or more is applied when relatively emphasizing the sound component from a specific direction represented in three dimensions. When relatively emphasizing the sound component from the specific direction represented by two dimensions, at least four or more omnidirectional microphones 21a, 21b... That is, in the case of relatively emphasizing sound components from a specific direction expressed in N dimensions, at least N+2 omnidirectional microphones 21a, 21b... Two sets of N+1 omnidirectional microphones 21a, 21b,... May be formed so as to reduce the number of 21a, 21b,.

(Directivity control device)
FIG. 4 is a block diagram showing a detailed configuration of the directivity control device 3. First, the sound collection signals from the omnidirectional microphones 21a, 21c, 21d, and 21e having one tetrahedral arrangement relationship are set as one group A, and the other omnidirectional microphone having one tetrahedral arrangement relationship. Collected signals from the microphones 21a, 21b, 21d, and 21f are defined as one group B. The directivity control device 3 includes a sensitivity correction unit 31, a beam former 32, and a noise processing unit 33.

The sensitivity correction unit 31 unifies the sensitivities of the six omnidirectional microphones 21a to 21f. The beam former 32 generates a first directional signal S1 having directivity in a specific direction based on the sound collection signals Sa, Sc, Sd, and Se of the group A, and collects the sound collection signals Sa, Sb of the group B, A second directivity signal S2 having directivity directed in a specific direction is generated based on Sd and Sf.

The noise processing unit 33 generates a coefficient m that emphasizes the correlation component of the first directional signal S1 and the second directional signal S2 and suppresses the non-correlation component, and uses this coefficient m as the first orientation. The output signal So is generated by multiplying the sex signal S1. It is also possible to multiply the coefficient m by the second directional signal S2.

FIG. 5 is a schematic diagram showing original polar patterns of the six omnidirectional microphones 21a to 21f. As shown in FIG. 5, the sensitivity correction unit 31 selects one of the sound collection signals Sa to Sf of the six omnidirectional microphones 21a to 21f, and selects the other sound collection signals with the acoustic power of the selected sound collection signal. Gains Ga to Gf are applied to the other sound collection signals so that the sound powers of the signals Sa to Sf become equal.

For example, the sound collection signal Sa output by the omnidirectional microphone 21a is selected, and other sound collection signals having sound powers Pb, Pc, Pd, Pe and Pf are selected with reference to the sound power Pa of the sound collection signal Sa. All of Sb to Sf are changed to acoustic power Pa. That is, the sound collection signal Sb having the acoustic power Pb is multiplied by the gain Gb, with the gain Gb being the reciprocal of the magnification of the acoustic power Pb with respect to the reference acoustic power Pa.

Further, the sound collection signal Sc having the acoustic power Pc is multiplied as the gain Gc by the reciprocal of the magnification of the acoustic power Pc with respect to the reference acoustic power Pa. The reciprocal of the sound collection signal Sd having the sound power Pd, the sound collection signal Se having the sound power Pe, and the sound collection signal Sf having the sound power Pf are similarly multiplied.

FIG. 6 is a block diagram showing a detailed configuration of the sensitivity correction unit 31. The sensitivity correction unit 31 includes bandpass filters 311a to 311f and power calculation units 312a to 312f for the sound collection signals Sa to Sf input from the six omnidirectional microphones 21a to 21f. The band pass filters 311a to 311f extract a predetermined frequency region from each of the sound collection signals Sa to Sf. The audible range is desirable as the predetermined frequency range. The power calculators 312a to 312f calculate the root mean square from the signals passed through the bandpass filters 311a to 311f, and obtain the power values Pa, Pb, Pc, Pd, Pe and Pf of the sound collecting signals Sa to Sf.

Further, the sensitivity correction unit 31 includes a coefficient calculation unit 313 and multiplication units 314a to 314f. The coefficient calculation unit 313 sets one of the power values of the sound collection signals Sa to Sf as a reference value, and obtains each gain Ga to Gf represented by the reciprocal of the power value multiplication factor with respect to the reference value. The multiplication units 314a to 314f change the gains Ga to Gf obtained by the coefficient calculation unit 313 to the corresponding sound collection so that the sound collection signals Sa to Sf are equal in sound power to the reference sound collection signal. Multiply the signal.

Next, the signal processing of the beam former 32 will be described. First, assume a space in which four omnidirectional microphones M1 to M4 exist. FIG. 7 is a schematic diagram showing control points D1 to Dn set around four omnidirectional microphones M1 to M4 according to the multipoint control method. In the figure, the control points are arranged only on the horizontal plane for convenience of explanation, but it is assumed that the n control points D1 to Dn are equally arranged on the spherical surface equidistant from the centers of the microphones M1 to M4. ..

A transfer function C _ik (ω) is a transfer function that changes the sound on the path from the control point D _i , (i=1 to n) to the omnidirectional microphone M _k , (k=1 to m). A control filter convoluted with the output of the omnidirectional microphone M _k is a transfer function H _k (ω), and a desired response to each control point D _i is a response function A _i (ω). At this time, in order to make the result of collecting the sound coming from each control point D _i by the omnidirectional microphones M1 to M4 the desired response transfer function A _i (ω), the following equation (1) is satisfied. Transfer function H _k (ω) is required.

Here, in the above formula (1), the desired response transfer functions A _j (ω), (i=j) corresponding to the control points D _j in the specific direction are set to “1”, and other control points to be suppressed. If the desired response transfer function A _i (ω), (i≠j) corresponding to D _i is set to zero, the above equation (1) directs the control point D _{j in} the specific direction with the existing direction as the specific direction. This is an expression including a transfer function H _k (ω), (k=1 to m) for directing the sex.

Therefore, when the specific direction is set by the user, the beam former 32 sets the desired response transfer function A _j (ω) corresponding to the control point D _j that matches the specific direction to 1, and sets the other desired response transfer functions. Set A _i (ω) to zero. Then, the beam former 32 obtains the transfer function H _k (ω), (k=1 to m) as a least squares solution using the generalized inverse matrix. Then, the transfer functions H1(ω) to H4(ω) corresponding to the omnidirectional microphones M1 to M4 are convoluted with the sound collection signals of the omnidirectional microphones M1 to M4.

The beam former 32 applies the multi-point control method as described above to generate and apply the transfer function H _k (ω) corresponding to the omnidirectional microphones 21a to 21f, thereby applying the collected sound signal of the group A. A first directivity signal S1 having directivity in a specific direction is generated from Sa, Sc, Sd, and Se, and a second directivity signal S1 having directivity in a specific direction from the sound collection signals Sa, Sb, Sd, and Sf of the group B. Directional signal S2 is generated. Then, the bidirectional signals S1 and S2 have the same polar pattern. The polar pattern may be a unidirectional cardioid type having directivity in a specific direction, or may be bidirectional having directivity in the specific direction and its opposite direction.

The ideal transfer function C _ik (ω) in free space is given by the following expression (2). However, it is desirable to obtain the transfer function C _ik (ω) by actually measuring transfer characteristics such as attenuation and delay between the omnidirectional microphones 21a to 21f and the control points i.

但し、ｒは制御点からマイクまでの距離であり、ｒ／ｃは伝播時間である。

However, r is the distance from the control point to the microphone, and r/c is the propagation time.

FIG. 8 is a block diagram showing the configuration of the beam former 32. The beam former 32 adds the control filters 321a, 321c, 321d, and 321e to which the sound collection signals Sa, Sc, Sd, and Se of the group A are input and the outputs of the control filters 321a, 321c, 321d, and 321e to add the first signal. And an adder 322a for outputting the directional signal S1. The beam former 32 also sums the output of the control filters 321a, 321b, 321d, and 321f and the control filters 321a, 321b, 321d, and 321f to which the sound collection signals Sa, Sb, Sd, and Sf of the group B are input. The adder 322b that outputs the second directional signal S2 is provided.

Further, the beam former 32 includes a specific direction setting unit 323 that generates a transfer function H _k (ω) and sets it in the control filters 321a to 321f. When the specific direction is set, the specific direction setting unit 323 generates the matrix A(ω) of the desired response transfer function A _i (ω) according to the specific direction. The specific direction setting unit 323 also takes the product of the matrix C(ω) of the transfer function C _ik (ω) and the matrix H(ω) of the transfer function H _k (ω).

Then, the specific direction setting unit 323 generates an error e by subtracting the multiplication result of the transfer function matrix C(ω) and the transfer function matrix H(ω) from the desired response transfer function matrix A(ω). The specific direction setting unit 323 solves the transfer function matrix H(ω) that minimizes the error e by the steepest descent method, and sets the transfer function H _k (ω) in the control filters 321a to 321f.

The beam former 32 Fourier-transforms the sound collection signals Sa, Sc, Sd, and Se of the group A into the control filters 321a to 321f in which the corresponding transfer functions H _k (ω) are set, and inputs them. The sound collection signals Sa, Sb, Sd and Sf are Fourier-transformed and input. Then, the beam former 32 adds the sound collection signals Sa, Sc, Sd, and Se of the group A output from the control filters 321a, 321c, 321d, and 321e to generate the first directional signal S1, and the control filter The sound collecting signals Sa, Sb, Sd, and Sf of the group B output from 321a, 321b, 321d, and 321f are added up to generate the second directional signal S1.

9A shows the first directional signal S1 derived from the group A, FIG. 9B shows the second directional signal S2 derived from the group B, and the horizontal axis represents frequency and the vertical axis represents sound pressure. It is a graph taken. As shown in (a) and (b) of FIG. 9, the beamformer emphasizes a single directivity in the first and second directivity signals S1 and S2, and the sensitivity correction unit 31 Since the sensitivities are unified according to, they have a common sound component Sc. On the other hand, noise N1 and N2 are present in each frequency band.

Here, the characteristics of the six omnidirectional microphones 21a to 21f are not uniform, and individual differences among the six omnidirectional microphones 21a to 21f are significant. Therefore, the noises N1 and N2 in the bidirectional signals S1 and S2 originating from the groups A and B having different combinations of four microphones have low correlation. When the four microphones belonging to group A and group B do not overlap, the bidirectional signals S1 and S2 originating from group A and group B have noises N1 and N2 that are not correlated.

Therefore, the noise processing unit 33 performs signal processing that emphasizes a sound component having a high correlation and suppresses the sound component as the correlation of the sound component is lower. The sound components coming from the specific direction, which are included in the bidirectional signals S1 and S2, are equalized to have the same sound pressure by the sensitivity correction unit 31 and the beam former 32. Noises N1 and N2 with small or no noise will be suppressed.

The noise processing unit 33 alternately outputs the first directional signal S1(k) and the second directional signal S2(k) every other sample and outputs the result. That is, the data strings of the exchange signal Sx(k) and the exchange signal Sy(k) are as follows at k=1, 2, 3, 4,...
Sx(k)={S1(1) S2(2) S1(3) S2(4)...}
Sy(k)={S2(1) S1(2) S2(3) S1(4)...}

The noise processing unit 33 calculates the error between the exchange signal Sx(k) and the exchange signal Sy(k), and determines the coefficient m(k) according to the error. The noise processing unit 33 also sequentially updates the coefficient m(k) with reference to the past coefficient m(k−1).

The error signal e(k) of the exchange signal Sx(k) and the exchange signal Sy(k) of the same arrival is defined by the following equation (3).

The noise processing unit 33 uses the error signal e(k) as a function of the coefficient m(k−1) and calculates the recurrence formula between adjacent binomials of the coefficient m(k) including the error signal e(k). , The coefficient m(k) that minimizes the error signal e(k) is searched. The noise processing unit 33 decreases the coefficient m(k) as the time difference between the first directional signal S1(k) and the second directional signal S2(k) is increased by this arithmetic processing. Updating is performed in the direction of making the coefficient m(k) close to 1 if there is no time difference. Then, the noise processing unit 33 multiplies the first directional signal S1(k) or the second directional signal S2(k) by a coefficient m(k) at an arbitrary ratio and outputs the output signal So(k). Output.

FIG. 10 is a block diagram showing a configuration example of the noise processing unit 33. As shown in FIG. 10, the noise processing unit 33 alternately exchanges the signal sequences of the first directional signal S1(k) and the second directional signal S2(k) in the preceding stage to exchange the exchange signal Sx(k). ) A switching unit 331 that is a circuit for generating the switching signal Sy(k) is provided, a plurality of integrators and an adder for updating the coefficient m(k) are provided in the middle stage, and the coefficient m(k) is provided in the second stage. An integrating unit 332 that multiplies the one directional signal S1(k) or the second directional signal S2(k) to generate the output signal So(k) is provided.

The middle stage is a circuit that embodies the recurrence formula between adjacent binomials, and gradually updates the coefficient m(k) with reference to the past coefficient m(k-1). In the noise processing unit 33, an adaptive filter having a long tap number is excluded.

In the noise processing unit 33, in the middle stage, the exchange signal Sy(k) is used as a reference signal to generate an error signal e(k). That is, the exchange signal Sx(k) is input to the integrator 335. The integrator 335 multiplies the exchange signal Sa(k) by -1 times the coefficient m(k-1) one sample before. An adder 336 is connected to the output side of the integrator 335. The signal output from the integrator 335 and the exchange signal Sy(k) are input to the adder 336, and the instantaneous error signal e(k) is obtained by adding these signals.

The error signal e(k) is input to the integrator 337 that multiplies the input signal by μ. The coefficient μ is a step size parameter less than 1. An integrator 338 is connected to the output side of the integrator 337. The exchange signal Sx(k) and the signal μe(k) that has passed through the integrator are input to the integrator 338. This integrator 338 multiplies the exchange signal Sx(k) and the signal μe(k) to obtain the differential signal ∂E(m) ² /∂m of the instantaneous squared error represented by the following equation (4).

An adder 339 is connected to the integrator 338. The adder 339 completes the coefficient m(k) by calculating the following equation (5), and sets the coefficient m(k) in the integrating unit 332. That is, as in the following expression (5), the adder 339 completes the coefficient m(k) by adding the signal β·m(k−1) to the differential signal ∂E(m) ² /∂m. Let

The signal β·m(k−1) is connected to the output side of the adder 339 with a delayer 333 that delays the signal by one sample and an integrator 334 that integrates the constant β, and the signal one sample before. It is generated by multiplying the coefficient m(k−1) updated by the processing by the constant β in the integrator 334.

As a result, the noise processing unit 33 realizes the calculation process of the following recurrence formula (6), the coefficient m(k) is generated, and is gradually updated for each sampling.

That is, the noise processing unit 33 multiplies one of the exchange signals Sx and Sy by a coefficient m and then generates an error signal e of the exchange signals Sx and Sy, and a gradual calculation of the coefficient m including the error signal e. A recurrence formula calculation unit that calculates the formula and updates the coefficient m for each sample is provided. The error signal generation unit multiplies the exchange signal Sa(k) by −1 times the coefficient m(k−1) one sample before, and the signal output from the integrator 335 and the exchange signal Sy. (K) is input and is an adder 336 that adds these signals. The recurrence formula calculation unit is an integrator and an adder other than the integrator 335 and the adder 336.

Then, the noise processing unit 33 alternately calculates the following formula (7) and multiplies the directional signal by the coefficient m(k) obtained by the formula (7).

In Expression (7), the signal squared term acts to reduce uncorrelated components such as white noise with the passage of time. On the other hand, the adjacent term is equivalent to the numerator part of the following formula (8) for sequentially calculating the correlation coefficient, and the influence of the correlation component is reflected in the coefficient m.

That is, when trying to approximate the second directional signal S2 to the first directional signal S1, the uncorrelated component of the first directional signal S1 is in the amplification direction, and the second directional signal S2. The non-correlated component of is the suppression direction. Further, when the first directional signal S1 is attempted to be approximated to the second directional signal S2, the uncorrelated component of the first directional signal S1 is in the amplification direction, and the first directional signal S1 The non-correlated component of is the suppression direction.

Therefore, the noise processing unit 33 has a function of approximating the second directional signal S2 to the first directional signal S1 and performing synchronous addition, and a first directional signal S2 of the first directional signal S2. The function of approximating the directional signal S1 and attempting to perform synchronous addition is alternately repeated. Therefore, the functions of amplifying and suppressing the uncorrelated component are canceled out alternately. That is, the influence of the correlation component is strongly reflected on the coefficient m(k), and from the directional signals S1 and S2 multiplied by the coefficient m(k), the omnidirectional microphones 21a to 21f are output. The inherent noise is suppressed.

(effect)
As described above, the sound collection device 1 imparts directivity to a specific direction expressed in N dimensions, and includes at least N+2 omnidirectional microphones 21a, 21b,..., Beamformer 32, and noise processing. The part 33 is provided. The beam former 32 has two types of directional signals S1 and S2 having unidirectionality in a specific direction, based on the sound collection signals Sa, Sb,... Output by the omnidirectional microphones 21a, 21b,. To generate. The noise processing unit 33 suppresses the noise component based on the two types of directional signals S1 and S2.

This beam former has a first directivity having unidirectionality in a specific direction based on N+1 sound collection signals Sa, Sb,... Selected from the outputs of the omnidirectional microphones 21a, 21b. The signal S1 is generated, and N+1 sound collection signals Se, Sf,... In a combination different from the N+1 sound collection signals Sa, Sb,... Used for generating the first directional signal S1. Based on the above, the second directional signal S2 having unidirectionality in the same specific direction as the first directional signal S1 is generated. Then, the noise processing unit 33 suppresses the uncorrelated sound component between the two types of directional signals S1 and S2.

As a result, the noise components included in the first directional signal S1 and the second directional signal S2 become uncorrelated, so that the uncorrelated sound components are suppressed, so that the omnidirectional microphones 21a, 21b...・The circuit noise and wind noise peculiar to can be suppressed and the SN ratio is improved.

Moreover, the presence of the circuit noise and the wind noise peculiar to the device in which the collected signals Sa, Sb... Are recorded is positively utilized rather, and the omnidirectional microphones 21a, 21b. The work for adjusting the characteristics is not necessary, the labor is dramatically reduced, and the unit price of the sound collector 1 can be reduced.

Further, at least N+2 omnidirectional microphones 21a, 21b... Further comprises a sensitivity correction unit 31 for equalizing the sensitivity of the sound collection signals output, and the noise processing unit 33 is of two types after the correction by the sensitivity correction unit 31. The non-correlated sound components are suppressed between the directional signals S1 and S2.

As a result, the sound components arriving from the specific direction obtained by the omnidirectional microphones 21a, 21b,... Are accurately aligned and included in the first directional signal S1 and the second directional signal S2. The correlation of the sound components in the directions can be made higher. Therefore, the sharpness is increased in the relative emphasis or suppression of the sound component and the noise component in the specific direction, and the SN ratio becomes better.

Here, if the microphones that record the surrounding sounds have directivity that points in different directions, it is unclear whether the sound pressure difference between the microphones originates in the arrival direction or the sensitivity difference, and the sound that arrives from a specific direction is unknown. It is difficult to align the ingredients. However, in this sound collection device 1, since the omnidirectional microphones 21a, 21b,... Are used, it is possible to limit the sound pressure difference between the microphones to the one derived from the sensitivity difference, and the sound component arriving from a specific direction is limited. It is possible to arrange them.

First, the noise processing unit 33 switches the first directional signal S1 and the second directional signal S2 alternately for each sample. Next, one of the exchange signals Sx and Sy is multiplied by a coefficient m, and then an error signal of the exchange signals Sx and Sy is generated. Then, the recurrence formula of the coefficient m including the error signal e is calculated to update the coefficient m for each sample. The sequentially updated coefficient m is multiplied by the first directional signal S1 or the second directional signal S2 and output.

As a result, the function of approximating the second directional signal S2 with respect to the first directional signal S1 to perform synchronous addition and the operation of the first directional signal S1 with respect to the second directional signal S2 are performed. The functions of approximating and performing synchronous addition are repeated alternately. Therefore, the functions of amplifying and suppressing the uncorrelated component are canceled out alternately. That is, the influence of the correlation component is strongly reflected on the coefficient m(k), and from the directional signals S1 and S2 multiplied by the coefficient m(k), the omnidirectional microphones 21a to 21f are output. The inherent noise is suppressed.

(Application example)
An application example of the sound collector 1 is shown in FIGS. 11 and 12. 11 and 12 are schematic diagrams showing an example of an omnidirectional camera capable of photographing each azimuth and each elevation angle. The microphone unit 2 and the panoramic image pickup unit 100 are integrated, and the same support member 23 is used as a housing. A plurality of cameras 101 and omnidirectional microphones 21a to 21f included in the panoramic image pickup unit 100 are attached to the support member 23. ing.

For example, as shown in FIG. 11, the support member 23 has a quadrangular prism shape. Two cameras 101 each formed of a fisheye lens are distributed to a pair of opposing surfaces of the side surfaces of the support member 23. Margins are formed at the four corners of the pair of facing surfaces. The omnidirectional microphones 21a to 21f are distributed in these four corners. For example, the omnidirectional microphones 21a to 21c are distributed one by one to the three corners of the one opposing surface, and the omnidirectional microphones 21d to 21f are distributed one to one of the three corners of the other opposing surface.

Moreover, as shown in FIG. 12, the support member 23 has a spherical shape. Two cameras 101, which are fish-eye lenses, are distributed 180 degrees apart on the support member 23. The omnidirectional microphones 21a to 21f are distributed beside the two cameras 101. For example, assuming that the position directly above one of the cameras 101 is zero degree, the omnidirectional microphones 21a to 21c are allocated to the 45-degree position, the 225-degree position, and the 315-degree position in the clockwise direction when viewed from the direction facing the camera 101. The omnidirectional microphones 21d to 21f are distributed one by one to the 135 degree position, the 225 degree position, and the 315 degree position in the clockwise direction when viewed from the direction facing the camera 101, with the position right above the other camera 101 as zero degree. To be done.

That is, since the camera 101 and the microphone unit 2 are integrally provided on the same support member 23, the omnidirectional camera is compact. The reason is that the sound collector 1 positively uses the omnidirectional microphones 21a to 21f, so that a space like a directional microphone is not required behind the omnidirectional microphones 21a to 21f.

Further, another application example of the sound collector 1 is shown in FIG. The panoramic image capturing unit 100 is attached to a rod-shaped support member 23 held by the user. The microphone unit 2 is built in the support member 23 held by the user. That is, the omnidirectional microphones 21a to 21f are attached to the respective corners of the virtual hexagonal column inscribed in the support member 23. As described above, since the omnidirectional microphones 21a to 21f can be built in the rod held by the user, the omnidirectional camera can be compactly installed. This is because there is no need for a space like a directional microphone behind the omnidirectional microphones 21a to 21f.

The image pickup data of the panoramic image pickup unit 100 and the sound collection signals Sa to Sf are transferred to the user's personal computer 102. The personal computer 102 is installed with an application 103 for implementing image processing for rendering an image in a specific direction from captured data and realizing the directional control device 3. Information indicating a specific direction input by the user to the application 103 using a man-machine interface such as a keyboard or a mouse is passed to the beamformer 32, and the beamformer 32 transmits a desired response transfer function matrix according to the information indicating the specific direction. Generate A(ω).

As described above, the beam former 32 may further include an input unit such as a keyboard or a mouse that receives an input in the specific direction, and the beam former 32 may form directivity in the specific direction received by the input unit.

Further, it has a plurality of cameras 101, and further includes a panoramic image pickup unit 100 for photographing each direction around, and at least N+2 omnidirectional microphones 21a, 21b... Are distributed between the plurality of cameras 101. I was attached. As a result, it becomes difficult for the omnidirectional microphones 21a, 21b,... To be captured in the plurality of cameras 101, and high-quality shooting is possible. The reason why the plurality of cameras 101 can be distributed among the cameras 101 is that no space is required behind the omnidirectional microphones 21a, 21b,....

(Other embodiments)
Although the embodiment of the present invention has been described above, this embodiment is presented as an example and is not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the scope of equivalents thereof.

For example, the noise processing unit 33 multiplies one of the exchange signals by the coefficient m, generates an error signal of the exchange signal, calculates a recurrence formula of the coefficient m including the error signal, and samples the coefficient m by one sample. If it is updated every time, it can be realized in other modes without being limited to the above-mentioned embodiment.

Further, the sound collecting device 1 may be realized as software processing of a CPU or a DSP, or may be configured by a dedicated digital circuit. When implemented as software processing, in a computer including a CPU, an external memory, and a RAM, a program describing the same processing content as the sensitivity correction unit 31, the beam former 32, and the noise processing unit 33 is stored in a ROM, a hard disk, a flash memory, or the like. It may be stored in the external memory of the above, expanded in the RAM as appropriate, and the CPU may perform the operation according to the program.

1 Sound Collection Device 2 Microphone Units 21a to 21f Omnidirectional Microphone 22 Nonvolatile Memory 23 Support Member 3 Directional Control Device 31 Sensitivity Correction Units 311a to 311f Bandpass Filters 312a to 312f Power Calculation Unit 313 Coefficient Calculation Units 314a to 314f Multiply Unit 32 Beamformers 321a to 321f Control filters 322a, 322b Adder 323 Specific direction setting unit 33 Noise processing unit 331 Exchange unit 332 Accumulator 333 Delay unit 334 Accumulator 335 Accumulator 336 Adder 337 Accumulator 338 Accumulator 339 Accumulator 339 Adder 34 memory 100 panoramic image pickup unit 101 camera 102 personal computer 103 application

Claims (14)

A sound collecting device for directing a specific direction expressed in N dimensions,
At least N+2 omnidirectional microphones that measure ambient sounds and output collected signals,
A beamformer that generates two types of directional signals having unidirectionality in a specific direction based on a sound collection signal output by the omnidirectional microphone,
A noise processing unit that suppresses noise components based on the two types of directional signals;
Equipped with
The beam former is
A first directional signal having unidirectionality in a specific direction is generated based on N+1 sound collection signals selected from the output of the omnidirectional microphone,
Based on N+1 sound collection signals in a combination different from the N+1 sound collection signals used to generate the first directional signal, a unidirectional pattern in the same specific direction as the first directional signal. Generate a second directional signal having
The noise processing unit,
Suppressing uncorrelated sound components between the two types of directional signals,
Sound collection device characterized by. Further comprising a sensitivity correction unit that equalizes the sensitivity of the sound collection signals output by the at least N+2 omnidirectional microphones,
The noise processing unit suppresses an uncorrelated sound component between the two types of directional signals that have been corrected by the sensitivity correction unit,
The sound collecting device according to claim 1. Directivity is given to a specific direction expressed in three dimensions,
At least five or more of the omnidirectional microphones are provided,
The beam former is
Based on four sound collection signals selected from the output of the omnidirectional microphone, a first directional signal having unidirectionality in a specific direction is generated, and
Based on four sound collection signals in a combination different from the four sound collection signals used to generate the first directional signal, a unidirectional pattern in the same specific direction as the first directional signal. Generating a second directional signal having
The sound collecting device according to claim 1 or 2. The omnidirectional microphone is
Three-dimensionally arranged so that you can select two types of combinations of the omnidirectional microphones that face each vertex of the tetrahedron,
The two types of tetrahedra are not congruent or similar, have different positions, have different orientations, or have a combination of these.
The sound collecting device according to claim 3, wherein Directivity is added to a specific direction expressed in two dimensions,
At least four or more of the omnidirectional microphones are provided,
The beam former is
Based on three sound collection signals selected from the output of the omnidirectional microphone, a first directional signal having unidirectionality in a specific direction is generated, and
Based on three sound collection signals in a combination different from the three sound collection signals used to generate the first directional signal, a unidirectional pattern in the same specific direction as the first directional signal. Generating a second directional signal having
The sound collecting device according to claim 1 or 2. Further comprising an input unit for receiving an input in the specific direction,
The beam former forms directivity in the specific direction received by the input unit,
The sound collecting device according to any one of claims 1 to 5. Having a plurality of cameras, further equipped with a panoramic image capturing unit for capturing each direction around,
The at least N+2 omnidirectional microphones are distributed between the plurality of cameras or are attached to a user gripping rod that supports the plurality of cameras,
The sound collecting device according to any one of claims 1 to 6, characterized in that: The noise processing unit,
An exchange unit that generates a pair of exchange signals by alternately exchanging the first directional signal and the second directional signal for each sample.
An error signal generation unit that generates an error signal of the exchange signal after multiplying one of the exchange signals by a coefficient m,
A recurrence formula calculation unit that calculates a recurrence formula of the coefficient m including the error signal and updates the coefficient m for each sample;
An accumulator that multiplies the first directivity signal or the second directivity signal by the coefficient m that is sequentially updated, and outputs the product.
Be equipped with,
The sound collecting device according to any one of claims 1 to 7, characterized in that. A directivity control device for directing a specific direction represented by N dimensions,
A storage unit for storing each sound collection signal generated by at least N+2 omnidirectional microphones;
A beamformer for generating two types of directional signals having a unidirectionality in a specific direction based on the sound collection signal in the storage section;
A noise processing unit that suppresses noise components based on the two types of directional signals;
Equipped with
The beam former is
Based on the N+1 sound collection signals selected from the storage unit, a first directional signal having unidirectionality in a specific direction is generated, and
Based on N+1 sound collection signals in a combination different from the N+1 sound collection signals used to generate the first directional signal, a unidirectional pattern in the same specific direction as the first directional signal. Generate a second directional signal having
The noise processing unit,
Suppressing uncorrelated sound components between the two types of directional signals,
Directional control device characterized by. Further comprising a sensitivity correction unit for equalizing the sensitivities of the at least N+2 collected signals,
The noise processing unit suppresses an uncorrelated sound component between the two types of directional signals that have been corrected by the sensitivity correction unit,
The directivity control device according to claim 9, wherein: The noise processing unit,
An exchange unit that generates a pair of exchange signals by alternately exchanging the first directional signal and the second directional signal for each sample.
An error signal generation unit that generates an error signal of the exchange signal after multiplying one of the exchange signals by a coefficient m,
A recurrence formula calculation unit that calculates a recurrence formula of the coefficient m including the error signal and updates the coefficient m for each sample;
An accumulator that multiplies the first directivity signal or the second directivity signal by the coefficient m that is sequentially updated, and outputs the product.
Be equipped with,
The directivity control device according to claim 9 or 10, characterized in that. A directivity control method for directing a specific direction expressed in N dimensions,
A beamforming step of generating two types of directional signals having unidirectionality in a specific direction based on each sound collection signal generated by at least N+2 omnidirectional microphones;
A noise suppression step of suppressing a noise component based on the two types of directional signals,
Including
The beam forming step,
Generating a first directional signal having unidirectionality in a specific direction based on at least N+1 sound collection signals of at least N+2 sound collection signals;
Based on N+1 sound collection signals in a combination different from the N+1 sound collection signals used to generate the first directional signal, a unidirectional pattern in the same specific direction as the first directional signal. Generate a second directional signal having
The noise suppression step,
Suppressing uncorrelated sound components between the two types of directional signals,
Directional control method characterized by. Further comprising a sensitivity correction step of aligning the sensitivities of the at least N+2 collected signals,
The noise suppressing step suppresses a non-correlated sound component between the two types of directional signals that have been corrected by the sensitivity correcting step,
The directivity control method according to claim 12, wherein: The noise processing step,
An exchange step of generating a pair of exchange signals by alternately exchanging the first directional signal and the second directional signal for each sample.
An error signal generating step of generating an error signal of the exchange signal after multiplying one of the exchange signals by a coefficient m;
A recurrence formula calculation step of calculating a recurrence formula of the coefficient m including the error signal and updating the coefficient m for each sample;
An integrating step of multiplying the first directivity signal or the second directivity signal by the coefficient m that is sequentially updated, and outputting the product.
Be equipped with,
The directivity control method according to claim 12 or 13, characterized in that.

Images