JP2020028086A - Sound pickup device - Google Patents

Abstract

To provide a sound pickup device performing area sound pickup efficiently and stably.SOLUTION: A sound pickup device acquires area sound pickup components of all areas covering all sound pickup areas of each pattern, by using means for acquiring area sound pickup components of multiple sound pickup areas on the basis of combination of microphone arrays of more than one pattern, based on an input signal from a microphone array part capable of forming multiple microphone arrays of different directivity, means for acquiring area sound pickup components of independent parts not overlapping other sound pickup area, for one or more sound pickup areas, on the basis of the area sound pickup components of the sound pickup area of each pattern, and the area sound pickup components of the acquired sound pickup area and the independent parts.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a sound collection device, a program, and a method, and can be applied to, for example, a voice communication system used in a noisy environment.

  When a voice communication system or a voice recognition application system is used in a noisy environment, ambient noise mixed with necessary target voice is a troublesome factor that hinders good communication and lowers a voice recognition rate. Conventionally, in an environment where a plurality of sound sources exist, a beamformer using a microphone array has been used as a technique for obtaining a desired target sound by separating and collecting only a sound in a specific direction to avoid mixing of unnecessary sound. (Beam Former; hereinafter also referred to as "BF"; see Patent Document 2). BF is a technique for forming directivity by using a time difference between signals reaching each microphone. However, if there is another sound source around an area for sound collection (hereinafter, referred to as “target area”) using only BF, the sound existing in the target area (hereinafter, referred to as “target area sound”) Only difficult to pick up sound. For this reason, an area sound collecting method for collecting a target area by using a plurality of microphone arrays has been proposed in Patent Document 1 and the like.

  FIG. 23 is an explanatory diagram showing a process of collecting a target area sound from a sound source in the target area using two microphone arrays MA100 and MA200. FIG. 23A is an explanatory diagram showing a configuration example of each of the microphone arrays MA100 and MA200. FIGS. 23B and 23C are diagrams (images in a graph format) showing the BF outputs of the microphone arrays MA100 and MA200 shown in FIG. 23A in the frequency domain. In FIG. 23, each of the microphone arrays MA100 and MA200 includes two microphones ch1 and ch2, respectively.

  In the conventional area sound pickup, as shown in FIG. 23A, sound is picked up by intersecting the directivities of the microphone arrays MA100 and MA200 from different directions in areas (target areas) where sound pickup is desired. In the state of FIG. 23A, the directivity of each of the microphone arrays MA100 and MA200 includes not only the sound existing in the target area (target area sound) but also noise in the target area direction (non-target area sound). I have. However, as shown in FIGS. 23B and 23C, when the directivity of the microphone arrays MA100 and MA200 is compared in the frequency domain, the target area sound component is included in both outputs, but the non-target area is included. The sound component will be different for each microphone array. In the conventional area sound collecting technique, by utilizing such characteristics, only the target area sound can be extracted by suppressing components other than those commonly included in the BF outputs of the two microphone arrays MA100 and MA200.

JP 2012-217315 A JP 2005-195555 A

Tadashi Asano, "Acoustic Technology Series 16 Array Signal Processing of Sound-Localization / Tracking and Separation of Sound Sources", edited by The Acoustical Society of Japan, Corona, Feb. 25, 2011

  By the way, an emergency vehicle is provided with a communication handset (handset) as a means of emergency communication from a fire site where a siren sounds and an emergency site to a command center (fire department). The handset mounted on a conventional emergency vehicle is in a loud noise environment, so communication from the site is drowned out by surrounding noise and sent to the headquarters (for example, the headquarters that commands the crew of the emergency vehicle). Inaccurate information cannot be provided, resulting in erroneous information, which may hinder accurate judgment and delay response. For this reason, the use of various noise removal techniques for handsets has been studied, but there have been many problems in introducing such techniques as securing speech quality and increasing costs. In such a usage environment, the above-described area sound collection technology is expected as an effective solution. For example, two microphone arrays are installed around the mouthpiece of the handset, and the directivity of each of the two microphone arrays intersects in front of the mouthpiece to function as an area sound pickup, thereby providing a large siren or the like. Noise can be eliminated, and only the voice of the sender, such as a firefighter, can be accurately transmitted to the headquarters and other places.

  At least two microphone arrays are required to achieve area sound pickup. On the other hand, the size of the mouthpiece in the handset is as small as about 6 cm in diameter, and when two microphone arrays are mounted to achieve area sound collection, the microphone arrays are placed very close together. There is a need to. As a result, in the area sound collection using the handset, the sound collection area is limited to a very small area immediately near the transmitter. However, when the conventional area sound collection processing is applied to the handset, the manner of holding the handset and the size of the face differ depending on the user (speaker), and the mouth has the above-described narrow sound collection area (set for the handset). Sound pickup area). In this case, if the mouth of the user (speaker) shifts from the sound collection area of the handset, distortion or dropout of the collected sound occurs, and there is a problem that stable sound collection cannot be performed.

  Therefore, a sound collection device, a program, and a method that can stably perform area sound collection are desired.

  According to the first aspect of the present invention, there is provided a sound collecting apparatus comprising: (1) a plurality of microphone arrays based on a combination of two or more patterns based on input signals from a microphone array unit capable of forming a plurality of microphone arrays having different directivities; Area sound pickup means for acquiring an area sound pickup component of a sound pickup area; and (2) one or more of the sound pickup means based on the area sound pickup components of the sound pickup area of each pattern acquired by the area sound pickup means. Independent area component extraction means for acquiring an area sound component of an independent part which does not overlap with the other sound pickup areas, (3) area sound pickup components of the sound pickup area acquired by the area sound pickup means; Using the area sound component of the independent portion extracted by the independent area component extraction means, and the area sound composition of all areas covering all of the sound collection areas acquired by the area sound collection means. And having a partial area integration means for acquiring.

  A sound collection program according to a second aspect of the present invention provides a computer for: (1) combining two or more patterns of the microphone array based on an input signal from a microphone array unit capable of forming a plurality of microphone arrays having different directivities. Area sound collecting means for acquiring area sound collecting components of a plurality of sound collecting areas based on the sound collecting area; and (2) one or a plurality of areas based on the area sound collecting components of the sound collecting area of each pattern acquired by the area sound collecting means. Independent area component extraction means for acquiring an area sound component of an independent portion which does not overlap with the other sound collection areas; and (3) area collection of the sound collection area acquired by the area sound collection means. Using the sound component and the area sound pickup component of the independent portion extracted by the independent area component extraction means, covers all of the sound pickup areas acquired by the area sound pickup means. It characterized in that to function as a partial area integration means for acquiring the area sound-pickup component area.

  According to a third aspect of the present invention, there is provided a sound collection method performed by a sound collection device, comprising: (1) an area sound collection unit to be acquired, an independent area component extraction unit, and a partial area integration unit; Based on an input signal from a microphone array unit capable of forming a plurality of microphone arrays having different directivities, acquiring area sound pickup components of a plurality of sound pickup areas based on a combination of two or more patterns of the microphone array; 3) The independent area component extraction means, based on an area sound collection component of the sound collection area of each pattern acquired by the area sound collection means, for one or more of the sound collection areas and the other sound collection areas. (4) acquiring the area sound component of an independent portion that does not overlap; and (4) the partial area integration means, wherein the area sound component of the sound collection area acquired by the area sound collecting means and the independent area component The area sound pickup component of the entire area covering all of the sound pickup areas obtained by the area sound pickup means is obtained by using the area sound pickup component of the independent portion extracted by the output means. .

  ADVANTAGE OF THE INVENTION According to this invention, the sound collection device which performs area sound collection efficiently and stably can be provided.

FIG. 2 is a block diagram illustrating a configuration of each device according to the first embodiment (including a functional configuration of a sound collection unit (sound collection device) according to the embodiment). FIG. 2 is a diagram (perspective view) illustrating a use state of the handset according to the first embodiment. It is the figure which expanded and showed the mouthpiece part of the handset which concerns on 1st Embodiment. FIG. 3 is an explanatory diagram (image diagram) illustrating a configuration example of a microphone array formed by three microphones. FIG. 9 is an explanatory diagram (image diagram) illustrating an area sound pickup process corresponding to each combination (combination pattern) of microphone arrays formed by three microphones. FIG. 11 is a diagram illustrating a distribution of sensitivity (calculated sensitivity distribution) of area sound pickup when the directivities of two microphone arrays cross each other. It is a block diagram which shows the structure regarding the subtraction type BF when the number of microphones is two. FIG. 4 is a diagram illustrating a directional characteristic formed by a subtraction type BF using two microphones. FIG. 13 is a block diagram illustrating a configuration of each device related to the second embodiment. FIG. 9 is a diagram illustrating an arrangement of six microphones and a configuration example of a microphone array in a microphone array unit according to a second embodiment. FIG. 10 is an explanatory diagram showing a distribution of sound collection areas where a target area sound extraction unit according to the second embodiment performs area sound collection. It is an explanatory view showing an independent area which does not overlap among a plurality of sound collection areas in the sound collection area according to the second embodiment. It is explanatory drawing which showed the composition image (power for every component) of each sound collection area which concerns on 2nd Embodiment in the form of a bar graph. It is an explanatory view showing a procedure of processing by an independent area component calculation part concerning a 2nd embodiment. FIG. 13 is a block diagram illustrating a configuration of each device related to a third embodiment. It is explanatory drawing shown about the image of three sound collection areas concerning 3rd Embodiment. It is explanatory drawing shown about the decomposition | disassembly image of the combination pattern (1st-3rd combination pattern) in three sound collection areas concerning 3rd Embodiment. It is explanatory drawing shown about the image of the independent part produced | generated in area A and D concerning 3rd Embodiment. It is explanatory drawing shown about the image of area A @ D which concerns on 3rd Embodiment. It is explanatory drawing shown about the image of the independent part produced | generated in area B and E concerning 3rd Embodiment. It is explanatory drawing shown about the image of the independent part produced | generated in area C and F concerning 3rd Embodiment. FIG. 9 is an explanatory diagram showing a configuration (a configuration of a modified example according to the embodiment) when the number of microphones in the microphone array unit according to the embodiment is four. FIG. 9 is an explanatory diagram illustrating a configuration example in a case where directivity by a beam former (BF) of two microphone arrays is directed to a target area from different directions in a conventional sound collection device.

(A) First Embodiment Hereinafter, a first embodiment of a sound collection device, a program, and a method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which a sound collection device, a program, and a method of the present invention are applied to a sound collection unit will be described.

  First, the basic principle of the area sound collection processing using the microphone array in this embodiment will be described with reference to FIGS.

  By arranging a microphone at the position of each vertex of the polygon, a plurality of area sound pickups can be constructed in the direction of the center of the polygon.

  For example, when a configuration of area sound collection using three microphones is considered, as shown in FIG. 4, up to three microphone arrays (three microphone arrays having different directivity directions) are formed by combining microphones. Can be set. As shown in FIG. 4, among the three microphones ch1 to ch3, a microphone array MA301 that pairs the microphones ch1 and ch2, a microphone array MA302 that pairs the microphones ch2 and ch3, and a microphone that pairs the microphones ch3 and ch1. The array MA303 can be set.

  Further, in the configuration of the three microphones ch1 to ch3, as shown in FIG. 5, it is possible to perform area sound pickup according to a combination (three combinations of patterns) of the three microphone arrays MA301, MA302, and MA303. .

  In FIG. 5A, the directivity of the microphone array MA301 is shown by a one-dot chain line, and the directivity of the microphone array MA302 is shown by a two-dot chain line. Also, in FIG. 5B, the directivity of the microphone array MA302 is shown by a dashed line, and the directivity of the microphone array MA303 is shown by a two-dot chain line. Further, in FIG. 5C, the directivity of the microphone array MA301 is shown by a one-dot chain line, and the directivity of the microphone array MA303 is shown by a two-dot chain line. Furthermore, in FIG. 5A, the sound collection area A301 corresponding to the combination (pattern) of the microphone arrays MA301 and MA302 is hatched (hatched). In FIG. 5B, the sound collection area A302 corresponding to the combination (pattern) of the microphone arrays MA302 and MA303 is hatched (hatched). Further, in FIG. 5C, the sound collection area A303 corresponding to the combination (pattern) of the microphone arrays MA301 and MA303 is hatched (hatched).

  As shown in FIG. 5, in the configuration of the three microphones ch <b> 1 to ch <b> 3, any microphone array has an angle between the microphone arrays (line segments connecting the positions of the two microphones constituting the microphone array). By intersecting the directivity of each other, it is possible to realize different area sound pickup (area sound pickup in different areas) for each combination.

  On the other hand, the sound collection area of the area sound collection using the microphone array has a property of expanding in front of the microphone array (farther from the microphone array). Hereinafter, the property will be described with reference to FIG.

  FIG. 6 is a diagram illustrating a distribution of sensitivity (calculated sensitivity distribution) of area sound pickup when the directivities of the two microphone arrays MA400 and MA500 intersect at a right angle to each other. In other words, FIG. 6 illustrates the sensitivity of area sound pickup in a region where the directivities of the two microphone arrays MA400 and MA500 intersect and in the vicinity thereof. In FIG. 6, the microphone arrays MA400 and MA500 have two microphones ch1 and ch2, respectively. In FIG. 6, the sensitivity of area sound pickup is divided into five stages (0 to -5 dB, -5 to -10 dB, -10 to -15 dB, -15 to -20 dB, -20 to -25 dB), and Are provided with different patterns (patterns). As shown in FIG. 6, it can be seen that a region with high sensitivity extends in a direction distant from microphone arrays MA400 and MA500 (that is, in the lower right direction).

  Therefore, the combination of FIG. 5A (combination of microphone arrays MA301 and MA302), the combination of FIG. 5B (combination of microphone arrays MA302 and MA303), and the combination of FIG. The sound pickup area (area distribution of sensitivity of area pickup) of the area sound pickup by each combination differs depending on the combination of the microphone arrays, and an overlapping part and a part not overlapping (a part where the sensitivity distribution coincides with a part which does not coincide) are different. Will happen.

  That is, as shown in FIG. 5, in a configuration of three microphones ch1 to ch3, area pickup is performed by a combination of two or three different microphone arrays, and if the respective pickup results are added, one microphone is obtained. It is possible to collect sound in a wider area than the sound collection area realized by the combination of the arrays.

  Therefore, in this embodiment, a combination (combination) of a plurality of different microphone arrays among a plurality of microphone arrays formed by microphones arranged at the corner vertices of a polygon (N polygon; N is an integer of 3 or more). ), And the result of adding or averaging the results of area pickup (output of area pickup) is treated as the final sound pickup result of the target area. As a result, in the area sound pickup processing of this embodiment, as a result, a more robust area sound pickup (more stable) with respect to the difference in the position of the mouth of the speaker (the position of the mouth of the speaker as viewed from the transmitter). Area sound collection).

(A-1) Configuration of Embodiment FIG. 1 is a block diagram showing a configuration of each device related to this embodiment.

  FIG. 1 illustrates a communication device 100 including a sound collection unit 120 according to this embodiment, and a communication device 200. In FIG. 1, the communication devices 100 and 200 are configured to be able to communicate with each other via a communication path P.

  The communication device 100 collects the voice (sound) uttered by the first user U1, transmits the voice data of the collected voice to the communication device 200 via the communication path P, and receives the voice data from the communication device 200. It is a device that outputs a sound based on the sound data (a sound uttered by the second user U2). Further, the communication device 200 collects voice (sound) uttered by the second user U2, transmits voice data of the collected voice to the communication device 100 via the communication path P, and This is a device for outputting a voice based on the received voice data (voice uttered by the first user U1).

  The first user U1 corresponds to, for example, a crew member appearing in an emergency vehicle such as an ambulance or a fire engine, and the second user U2 corresponds to, for example, a remote location (for example, a command center that commands an emergency vehicle). Commanders in charge correspond to this.

  The communication path P is not limited to wired or wireless, and various connection means and connection configurations (network configurations) can be applied.

  Next, an outline of the configuration of the communication device 100 will be described with reference to FIG.

  The communication device 100 includes a handset 110, a sound collection unit 120, a communication unit 130, and an output unit 140.

  The handset 110 includes a microphone array unit 111 composed of three microphones MC1 to MC3 (3ch microphones) and a speaker 112.

  The communication unit 130 is a communication interface for communicating with the communication device 200 via the communication path P.

  The sound collection unit 120 collects a sound (sound) uttered by the first user U1 based on the acoustic signal captured by the microphone array unit 111. Then, the communication unit 130 transmits the audio data of the audio collected by the sound collection unit 120 to the communication device 200 side.

  The output unit 140 acquires voice data (voice data of voice uttered by the second user U2) from the communication device 200 via the communication unit 130, supplies an audio signal based on the voice data to the speaker 112, 112 causes the sound signal to be output as sound.

  Although the hardware configuration of the communication device 100 is not limited, in the example of this embodiment, as illustrated in FIG. It shall be. Note that the communication device 100 does not necessarily need to include the handset 110, and the entire housing (chassis) functions substantially as a handset like a smartphone (for example, the communication device 100 may be provided in a part of the smartphone housing). A configuration in which a talk is set) may be used.

  Next, an outline of the configuration of the communication device 200 will be described with reference to FIG.

  The communication device 200 includes a speaker 210, a microphone 220, a communication unit 230, an output unit 240, and a sound collection unit 250. Although the hardware configuration of the communication device 200 is not limited, for example, various telephone devices (for example, speakerphones and the like) can be applied.

  The communication unit 230 is a communication interface for communicating with the communication device 200 via the communication path P.

  The sound collection unit 250 collects a sound (sound) uttered by the second user U2 based on the acoustic signal captured by the microphone 220. Then, the communication unit 230 transmits the sound data of the sound collected by the sound collection unit 250 to the communication device 100 side.

  The output unit 240 acquires voice data (voice data of voice uttered by the first user U1) from the communication device 100 via the communication unit 230, supplies an audio signal based on the voice data to the speaker 210, The sound signal is output to the sound signal 210.

  Next, a detailed configuration of the sound pickup unit 120 will be described with reference to FIG.

  The sound collection unit 120 includes a signal input unit 121, a frequency conversion unit 122, a directivity forming unit 123, a target area sound extraction unit 124, and an area sound addition unit 125.

  For example, the sound collection unit 120 may cause a computer including a processor, a memory, and the like to execute a program (including the sound collection program according to the embodiment), but even in that case, functionally, It can be shown as in FIG. Details of the processing of each component of the sound pickup unit 120 will be described later.

  Next, the configuration of the handset 110 as a handset will be described with reference to FIGS.

  FIG. 2 is a perspective view showing a state where the handset 110 is being held by the hand U1a of the first user U1.

  As shown in FIG. 2, the handset 110 includes a rod-shaped handle 115 for the first user U1 (hand U1a) to hold, and a mouthpiece 113 (speaker) provided at one end of the handle 115. And an earpiece 114 (receiver) provided at the other end of the handle 115.

  FIG. 3 is an enlarged view of the mouthpiece 113 of the handset 110.

  As shown in FIG. 2, a speaker 112 is arranged in the earpiece 114. As shown in FIGS. 2 and 3, microphone array unit 111 (microphone MC1 to MC3) is arranged in mouthpiece 113 having a circular surface.

  Next, the configuration of the microphone array unit 111 will be described with reference to FIGS.

  In the example of this embodiment, the microphone array unit 111 has a configuration including three microphones MC1 to MC3.

  As shown in FIG. 2, when the first user U1 holds the communication device 100 with the hand U1a and presses the speaker SP to the ear, the first user U1 surrounds the mouthpiece 113 where the mouth of the first user U1 is located (see FIG. 2). Three microphones MC <b> 1 to MC <b> 3 are arranged around the part closest to the mouth of the first user U <b> 1).

  In the handset 110 shown in FIGS. 2 and 3, each position (the center position of each microphone) of the three microphones MC <b> 1 to MC <b> 3 constituting the microphone array unit 111 is similar to the configuration shown in FIGS. 4 and 5 described above. , Around the mouthpiece 113 so as to be the vertices of an equilateral triangle. In FIGS. 2 and 3, each side of the triangle formed by the microphones MC1 to MC3 is set to have the same distance (a triangle formed by the microphones MC1 to MC3 is an equilateral triangle) in order to enlarge the sound pickup area in the same direction. Also, the angles of the angles may not all be the same.

  As shown in FIG. 3, in the following, in the microphone array unit 111, the microphone array paired with the microphones MC1 and MC2 is MA1, the microphone array paired with the microphones MC2 and MC3 is MA2, and the microphones MC3 and MC1 are paired. Is referred to as MA3.

(A-2) Operation of Embodiment Next, an operation of the embodiment having the above-described configuration (a sound collection method according to the embodiment) will be described.

  In the communication device 100, the sound collection unit 120 performs a target area sound collection process of collecting the target area sound of the target area using the acoustic signals supplied from the microphones MC1 to MC3 of the microphone array unit 111.

  The following description focuses on the operation inside the sound pickup unit 120 included in the communication device 100.

  The signal input unit 121 converts an acoustic signal collected by each of the microphones MC <b> 1 to MC <b> 3 from an analog signal to a digital signal, and supplies the digital signal to the frequency conversion unit 122. Then, the frequency conversion unit 122 converts the microphone signal from the time domain to the frequency domain using, for example, fast Fourier transform. The directivity forming unit 123 forms directivity by BF.

  Here, the formation of directivity by BF will be described with reference to FIGS.

  BF is a technique for forming a directivity of sound collection using a time difference between signals reaching respective microphones in a microphone array (see Non-Patent Document 1). BFs are roughly classified into two types, an addition type and a subtraction type. Here, a subtraction type BF that can form directivity with a small number of microphones will be described.

  FIG. 7 is a block diagram showing a configuration relating to the subtraction type BF 600 when the number of microphones is two (MC1, MC2).

  FIG. 8 is a diagram illustrating a directional characteristic formed by a subtraction type BF 600 using two microphones MC1 and MC2.

The subtraction type BF 600 first calculates the time difference between the signals that arrive at the microphones MC1 and MC2 of the sound (hereinafter, referred to as “target sound”) existing in the target direction by the delay unit 610, and adds a delay to the calculated signal. Adjust the sound phase. The time difference is calculated by equation (1). Here, d indicates the distance between the microphones MC1 and MC2, c indicates the speed of sound, and τ _i indicates the amount of delay. Θ _L indicates an angle from a vertical direction to a target direction with respect to a straight line connecting the positions of the microphones MC1 and M2.

Here, when the blind spot is directed toward the microphone MC1 with respect to the centers of the microphones MC1 and MC2, the delay unit 610 performs a delay process on the input signal x ₁ (t) of the microphone MC1. Thereafter, the subtractor 620 performs a subtraction process according to the equation (2). In the subtractor 620, this subtraction processing can be similarly performed in the frequency domain, and in that case, the equation (2) is changed to the equation (3).

Here, when θ _L = ± π / 2, the formed directivity is a cardioid type single directivity as shown in FIG. 8A, and when θ _L = 0, π, the formed directivity is FIG. An eight-shaped bidirectional pattern as shown in FIG. Further, the subtractor 620 can form a directivity that is strong in a bidirectional blind spot by using a process of a spectral subtraction method (hereinafter, also simply referred to as “SS”). The directivity by the SS is formed in all frequencies or a designated frequency band according to the equation (4). (4) In the formula, is used to input signals _{X 1} microphone MC1, it is possible to obtain the same effect input signal _{X 2} microphones MC2. Here, n is a frame number, and β is a coefficient for adjusting the strength of the SS. If the value becomes negative at the time of subtraction, the subtractor 620 may perform flooring processing of replacing the value with 0 or a value obtained by reducing the original value. In this method, a sound existing in a direction other than a target direction (hereinafter, referred to as a “non-target sound”) is extracted due to bidirectional characteristics, and an amplitude spectrum of the extracted non-target sound is subtracted from an amplitude spectrum of an input signal. Thus, the target sound can be emphasized.

  By the way, when it is desired to collect only a target area sound existing in a specific target area, a sound source existing on a line in the same direction as that area (hereinafter, referred to as “non-target area sound”) is used simply by using the subtraction type BF. Call) will also pick up sound.

  Therefore, the directivity forming unit 123 uses the area sound collection processing (a plurality of microphone arrays are used to direct the directivity from different directions to the target area, and intersect the directivity at the target area). In this case, the processing for collecting the target area sound will be described. Specifically, the directivity forming unit 123 may perform the area sound pickup processing by the following processing.

The directivity forming unit 123 forms the directivity of each of the microphone arrays MA1 to MA3 with the BF toward the inside of the triangle (the triangle formed by the microphones MC1 to MC3). Then, the directivity forming unit 123 supplies the BF outputs Y ₁ (n), Y ₂ (n), and Y ₃ (n) of the microphone arrays MA1, MA2, and MA3 to the target area sound extracting unit 124.

The target area sound extraction unit 124 extracts an area sound using the BF outputs Y ₁ (n), Y ₂ (n), and Y ₃ (n) of the microphone arrays MA1, MA2, and MA3 formed by the directivity forming unit 123. I do. As described above, each BF output (Y ₁ (n), Y ₂ (n), Y ₃ (n)) is centered (inside the triangle) from each side of the triangle (a triangle formed by the microphones MC1 to MC3). Direction). Therefore, in each BF output, in any two combinations (combination patterns), the two directivities intersect near the center of the triangle, so that the target area sound extraction unit 124 uses the area sound collection method described below. Thus, it is possible to extract sounds in an area where the directivities cross each other. Here, as a representative, the BF output _Y 1 of the microphone array MA1 (n), will be described using the BF output _Y 2 of the microphone array MA2 (n). The target area sound extraction unit 124 SSs Y ₁ (n) and Y ₂ (n) according to the formula (5) or (6), and non-target area sounds N _1-1 (n) existing in the direction of the target area. , N _1-2 (n). Here, α ₁ and α ₂ are correction coefficients for correcting a difference in signal level caused by a difference in distance between the target area and each microphone array, and should be calculated one by one by a predetermined process. Although described in Document 1, here, for simplicity, the distance between the target area and each microphone array is the same (α ₁ (n) = α ₂ (n) = 1), and (5), (6) Expression) is replaced by Expressions (7) and (8).

Thereafter, the target area sound extraction unit 124 extracts the non-target area sound from each BF output according to the equations (9) and (10) to extract the target area sound. Here, γ ₁ (n) and γ ₂ (n) are coefficients for changing the strength at the time of SS.

In destination area sound extraction unit 124, emphasized sound _Z 1-1 _(n), but may be output to any of the _{Z 1-2} (n), the microphone array _{Z 1-1} (n) here MA1- It is used as an area sound pickup output Z ₁ (n) based on a combination (combination pattern) of the microphone arrays MA2.

The destination area sound extraction unit 124 and similarly, the area sound-pickup output _Z 2 by the combination of the microphone array MA2- microphone array MA3 (n), and the area sound-pickup output _Z 3 by a combination of the microphone array MA3- microphone array MA1 (n ) Is extracted and supplied to the area sound adding unit 125.

As shown in FIG. 2, each of the microphones MC <b> 1 to MC <b> 3 is mounted in a narrow range of several centimeters in the mouthpiece 113 of the handset 110. Therefore, the microphone arrays MA1, MA2, and MA3 are arranged very close (dense), and the sound collection area is also limited to a narrow area in front of the mouthpiece 113. However, as shown in FIG. 6 described above, it has been found that the sound pickup area by the area sound pickup has a characteristic of extending in the far direction of the two microphone arrays. Therefore, if sound pickup areas (sound pickup areas corresponding to Z ₁ (n), Z ₂ (n), and Z ₃ (n)) extending in three different directions are overlapped, a single sound pickup area (single sound pickup area) is obtained. A wider range of area sound collection is possible as compared to a sound collection area corresponding to any one of Z ₁ (n), Z ₂ (n), and Z ₃ (n).

Therefore, the area sound adding unit 125 adds or averages the _three outputs Z ₁ (n), Z ₂ (n), and Z ₃ (n) of the collected sound to generate a final output W (n). The sound pickup unit 120 outputs the sound as a sound pickup result. The area sound addition unit 125 considers that there is a part where the areas overlap in the addition processing, and the addition value (Z ₁ (n) + Z ₂ (n), + Z ₃ ( n)) may be averaged or multiplied by a gain adjustment coefficient α as shown in equation (11). The area sound adding unit 125 adds (or averages) only two or more outputs of three area sound pickup outputs (Z ₁ (n), Z ₂ (n), and Z ₃ (n)). ) May be performed. For example, the area sound addition unit 125 may perform a process of adding (or averaging) only two outputs of the three area sound pickup outputs.

  As described above, the sound pickup unit 120 outputs the final output W (n) as the target sound picked up from the enlarged area. At this time, the sound pickup unit 120 may output W (n) as frequency-time converted audio data.

  Then, the communication unit 130 transmits the audio data based on the final output W (n) to the communication device 200 via the communication path P.

  Then, the communication unit 230 of the communication device 200 supplies the output unit 140 with the audio data (audio data based on W (n)) received from the communication device 100. The output unit 140 supplies an audio signal based on the received audio data to the speaker 210 to output a sound (a sound output to the second user U2).

(A-3) Effects of Embodiment According to this embodiment, the following effects can be obtained.

  In the sound pickup unit 120 of this embodiment, area sound pickup is performed from different directions, and by adding them, an area sound pickup wider than the conventional area pickup using one set (two) microphone arrays is performed. It is possible to form a sound collecting area having a characteristic (enlarged sound collecting area). With this, in the sound pickup unit 120, when performing area sound pickup using the microphones MC1 to MC3 attached to the mouthpiece 113 of the handset 110, the mouth of the speaker (first user U1) and the mouthpiece are performed. Even when the relative position with respect to 113 is shifted, stable voice pickup is possible.

(B) Second Embodiment Hereinafter, a second embodiment of a sound collection device, a program, and a method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which a sound collection device, a program, and a method of the present invention are applied to a sound collection unit will be described.

(B-1) Configuration of Second Embodiment FIG. 9 is a block diagram showing a configuration of each device related to the second embodiment. In FIG. 9, the same or corresponding portions as those in FIG. 1 described above are denoted by the same reference numerals or corresponding reference numerals.

  In the second embodiment, the communication device 100 is replaced with a communication device 100A. In the communication device 100A according to the second embodiment, the microphone array unit 111 and the sound pickup unit 120 are replaced with the microphone array unit 111A and the sound pickup unit 120A.

  Next, an internal configuration of the sound pickup unit 120A according to the second embodiment will be described.

  As described above, the sound pickup unit 120 of the first embodiment performs area sound pickup from different directions and superimposes (adds) them to use a conventional set (two) of microphone arrays. The sound pickup area (enlarged sound pickup area) which is wider than the sound pickup area and has the same directionality is formed.

  However, in the area sound pickup processing as in the first embodiment, if the sound pickup areas to be superimposed are independent from each other, there is no problem even if the sound pickup areas are simply superimposed. When there is an overlap, it is difficult to obtain a uniform sound collecting characteristic in the compounded (unified) sound collecting area. When the sound pickup results of a plurality of sound pickup areas having overlapping portions are added, the gain of the overlapping portion is further emphasized by adding components to those of the non-overlapping portion. Regarding the extended sound collection area, the sound collection characteristics in the sound collection area may be non-uniform as a result, and may differ from the original characteristics of the target sound source in the sound collection area. In particular, when the sound source position straddles the overlapping portion and the non-overlapping portion, the characteristic is likely to be distorted.

  Therefore, in the sound pickup unit 120A of the second embodiment, when sound pickup is performed from a plurality of sound pickup areas, even if there is an overlap between the sound pickup areas, uniform sound pickup characteristics are realized and distortion is reduced. An object is to provide a stable sound collection method.

  Specifically, in the sound pickup unit 120A of the second embodiment, the area sound pickup components (area sound output) of a plurality of sound pickup areas having overlapping portions are compared with the area sound pickup components of the respective sound pickup areas. , The area sound pickup component of the overlapping portion and the area sound pickup component of the non-overlapping portion are individually calculated, and the individually calculated area sound pickup components are integrated to form the entire range of the plurality of sound pickup areas. The sound pickup component of the area to be covered is obtained and output. Thus, the sound pickup unit 120A of the second embodiment does not double-count an area sound pickup component of an overlapped portion as in the first embodiment even if an overlapped portion exists in a plurality of sound pickup areas. In addition, the uniformity of the sound pickup characteristics can be maintained over the entire range of the plurality of sound pickup areas.

  Next, the internal configuration of the sound pickup unit 120A will be described with reference to FIG.

  The sound collecting unit 120A is different from the first embodiment in that the target area sound extracting unit 124 is replaced with the target area sound extracting unit 124A, and the area sound adding unit 125 is excluded. The sound pickup unit 120A is different from the first embodiment in that an independent area component calculation unit 126 and a partial area integration unit 127 are added.

  Next, the configuration of the microphone array unit 111A according to the second embodiment will be described.

  As shown in FIG. 9, in the second embodiment, the microphone array unit 111A has six microphones MC1 to MC6.

  FIG. 10 is a diagram illustrating an arrangement of six microphones MC1 to MC6 in the microphone array unit 111A and a configuration example of the microphone array.

  As shown in FIG. 10, the six microphones MC1 to MC6 constituting the microphone array unit 111A include three microphone arrays MA1 (a microphone array having the microphones MC1 and MC2 as a pair) and MA2 (a microphone array having two microphones as a pair). The microphones MC3 and MC4 constitute a pair of microphone arrays, and the MA3 (microphones MC5 and MC6 constitute a pair of microphone arrays). In the second embodiment, in order to easily explain the principle of the sound collection method according to the second embodiment, a configuration for performing area sound pickup of two overlapping areas is used.

(B-2) Operation of Second Embodiment Next, an operation (a sound collection method according to the embodiment) of the second embodiment having the above-described configuration will be described.

The signal input unit 121 converts an acoustic signal collected by the six microphones from an analog signal to a digital signal (x ₁ to x ₆ ), and supplies the digital signal to the frequency conversion unit 122.

The frequency conversion unit 122 converts the microphone signal from the time domain to the frequency domain using, for example, a fast Fourier transform (X _{1 to} X ₆ ).

The directivity forming unit 123 forms directivity by BF using the input signal of each microphone that has been time-frequency converted by the frequency converting unit 122. In the second embodiment, the BF output by the microphone array MA1 _{Y 1,} the BF output by the microphone array MA2 _{Y 2,} and the BF output _{Y 3} by the microphone array MA3. The directivity of the BF outputs Y ₁ , Y ₂ , and Y ₃ is as shown in FIG. As in the second embodiment shown in FIG. 10, the microphone array MA1~MA3 is disposed at the position of each vertex of a triangle, BF Output _{Y 1,} Y 2, _{Y 3} directional (microphone array MA1~MA3 Are directed inside the triangle.

The target area sound extraction unit 124A performs an area sound collection process using the BF output generated by the directivity forming unit 123. The area sound pickup is for directing the directivity of the BF from different directions and separating and extracting the components (area sounds) of the areas where the directivities intersect. BF Output _Y 1, BF combination of the output _{Y 2,} and BF Output _Y 1, BF respectively from the area voice collecting the combination of the outputs _{Y 3} can be realized.

  FIG. 11 is an explanatory diagram showing the distribution of sound collection areas where the target area sound extraction unit 124A performs area sound collection.

As shown in FIG. 6 described above, the area sound pickup has a characteristic that the sound pickup area spreads in a direction far from the microphone array. Therefore, an area sound pickup area (in the second embodiment, referred to as “area 1” or “sound pickup area 1”) by the microphone arrays MA1-MA2 and an area sound pickup area (the second sound pickup area) by the microphone arrays MA2-MA3. In the embodiment, “area 2” or “sound collection area 2” has an image as shown in FIG. In the second embodiment, the area sound pickup component (area sound pickup output) of the sound pickup area 1 is Z ₁ , and the area sound pickup component (area sound pickup output) of the area ₂ is Z ₂ .

  Each sound pickup area is divided into a part where the two sound pickup areas overlap as shown in FIG. 12 and an independent part where the two sound pickup areas do not overlap.

  In FIG. 12, an area overlapping in areas 1 and 2 is referred to as an overlapping area OL. In FIG. 12, in the area 1, an independent area excluding the overlapping area OL (an area that does not overlap with another sound collection area) is defined as an independent area A. Further, in FIG. 12, an independent area other than the overlapping area OL in the area 2 is defined as an independent area B. Note that an independent area (independent portion) generated from one sound collection area may be divided into a plurality of areas as shown in FIG. 12, but in this specification, an independent area generated from one sound collection area is used. The areas are collectively indicated by one code. For example, in FIG. 12, the independent area A is divided (divided) into two areas by the overlapping area OL, but here, these two areas are collectively referred to as an independent area A.

As described above, the area 1 is composed of the overlapping area OL and the independent area A (the area excluding the overlapping area OL from the area 1), and the area 2 is the overlapping area OL and the independent area B (the area excluding the overlapping area OL from the area 2). ). An area sound-pickup output Z ₁ of the area 1, and superimposing the area sound-pickup output Z ₂ of area 2 (summing), can be collected sound from a wide range of areas, that the components of the overlapping area OL is applied to double , And it is not possible to obtain a uniform sound collecting characteristic over the entire sound collecting area. Therefore, if the sound sources of the overlapping area OL and the independent areas A and B can be separately separated and extracted, the respective areas can be integrated without overlapping so that a uniform sound collection over the entire range of the areas 1 and 2 can be achieved. Characteristics will be obtained.

  In the independent area component calculation unit 126, the two area sound components having the overlapping area OL (here, the area sound components of the areas 1 and 2) are separated from the area sound components of the independent area (here, the independent areas A, B area sound pickup component).

  FIG. 13 is an explanatory diagram showing a composition image (power for each component) of each area shown in FIG. 12 in the form of a bar graph.

13 (a) shows the composition image of the area voice collecting component _{Z 1} in area 1, FIG. 13 (b) shows a composition image of the area sound-pickup component _{Z 2} of area 2. Further, FIG. 13 (c), the composition image of the area sound-pickup component Z ₁ shown in FIG. 13 (a), the components of the overlapping area OL illustrates hatched (shaded pattern). Further, FIG. 13 (d) the composition image area sound-pickup output Z ₂ shown in FIG. 13 (b), the components of the overlapping area OL illustrates hatched (shaded pattern).

Duplicate area OL in area 1 and area 2, since it is common to duplicate literally included as identical components each in Z ₁ and Z _2. Therefore, the target area sound extraction unit 124A separates each component by using the spectrum subtraction method (SS) based on the same principle as the area sound pickup.

Independent area component calculation unit 126 SS the area sound-pickup output _{Z 2} from the area sound-pickup output _{Z 1.} The independent area component calculation unit 126 clips a component that becomes negative at the time of SS to 0. In doing so, the destination area sound extraction unit 124A, area sound-pickup component overlapping area OL is removed from the area sound-pickup output Z _1, the area sound-pickup component (first embodiment of the independent area A "V _A "). Similarly, independent area component calculation unit 126, an area sound-pickup output Z ₁ from the area sound-pickup output Z ₂ By SS, the area sound-pickup component (first embodiment of the independent area B, "V _B" ) Can be separated.

  FIG. 14 is an explanatory diagram illustrating a procedure of processing by the independent area component calculation unit 126.

Figure 14 (a) ~ FIG. 14 (c), independent area component calculation unit 126, the area sound-pickup output _{Z 1} of the area 1, the independent area A the area sound-pickup output _{Z 2} of area 2 and SS Area The processing for extracting the sound pickup component _VA (processing corresponding to the following equation (21)) is shown. Figure 14 (a) ~ FIG. 14 (c), area sound-pickup component _{Z 1} of each area 1, area sound-pickup component _{Z 2} of area 2, represents a composition image of the area sound-pickup component _{V A} separate area A I have.

Further, FIG. 14 (d) ~ FIG 14 (f) is independent area component calculation unit 126, the area sound-pickup component _{Z 2} of area 2, independently Area B the area sound-pickup component _{Z 1} in area 1 and SS shows the process of extracting the area sound-pickup component V _B (process equivalent to the following equation (22)). Figure 14 (d) ~ FIG 14 (f), respectively, area sound-pickup component _{Z 2} of area 2, area sound-pickup component _{Z 1} in area 1, represents a composition image of the area sound-pickup component _{V B} of the independent area B ing.

In each composition image shown in FIG. 14, the area sound-pickup component overlapping area OL, and area sound-pickup component V _A separate area A, in the area sound-pickup component V _B of the independent area B, and different patterns respectively This is shown in FIG.

  As described above, the independent area component calculation unit 126 separates and separates the sound pickup components of the independent area included in only one sound pickup area from the two areas 1 and 2 including the overlapping area OL. Extract.

Partial area integrating unit 127 integrates the area voice collecting component of each independent area separated (V A or V _B,) and the area sound components that do not contain the independent area component (Z _2, or Z ₁₎ and Then, a final final output W is calculated and output. The partial area integration unit 127 calculates the final output W using, for example, the following equation (23) or (24).

  As described above, the sound pickup unit 120A outputs the final output W (n) as the target sound picked up from the enlarged area.

  Then, the communication unit 130 transmits the audio data based on the final output W (n) to the communication device 200 via the communication path P.

  Then, the communication unit 230 of the communication device 200 supplies the output unit 140 with the audio data (audio data based on W (n)) received from the communication device 100. The output unit 140 supplies an audio signal based on the received audio data to the speaker 210 to output a sound (a sound output to the second user U2).

(B-3) Effects of Second Embodiment According to the second embodiment, the following effects can be obtained.

  The sound pickup unit 120A of the second embodiment performs area sound pickup from different directions, and forms a sound pickup area wider and isotropic in comparison with the conventional area sound pickup using a pair of microphone arrays. can do.

  In the final output W of the second embodiment, since only one area sound output is selected and output for the same frequency component, the uniformity of the sound pickup characteristics is maintained even when the area is expanded. Thus, for example, even when the sound pickup unit 120A of the second embodiment is applied to a handset, stable sound pickup is possible even if the mouth of the user is shifted from the mouthpiece.

(C) Third Embodiment Hereinafter, a third embodiment of a sound collection device, a program, and a method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which a sound collection device, a program, and a method of the present invention are applied to a sound collection unit will be described.

(C-1) Configuration of Third Embodiment FIG. 15 is a block diagram showing the configuration of each device related to the third embodiment.

  15, the same or corresponding parts as those in FIG. 1 or FIG. 9 are denoted by the same reference numerals or corresponding reference numerals. Hereinafter, the third embodiment will be described focusing on differences from the first and second embodiments.

  The third embodiment is different from the second embodiment in that a communication device 100A is replaced by a communication device 100B. Further, the communication device 100B of the third embodiment is different from the second embodiment in that the microphone array unit 111A is replaced by a microphone array unit 111 (the same configuration as in the first embodiment). Further, the communication device 100B of the third embodiment is different from the second embodiment in that the sound collection unit 120A is replaced with the sound collection unit 120B.

  Next, an internal configuration of the sound pickup unit 120B according to the third embodiment will be described.

  In the sound pickup unit 120B of the third embodiment, the target area sound extraction unit 124A, the independent area component calculation unit 126, and the partial area integration unit 127 are connected to the target area sound extraction unit 124 (the same configuration as in the first embodiment). ), The independent area component calculation unit 126B and the partial area integration unit 127B are different from the second embodiment.

  As described above, the microphone array unit 111 according to the third embodiment includes three microphones MC1 to MC3, as in the first embodiment. In the third embodiment, the three microphones MC1 to MC3 are arranged on the surface of the mouthpiece 113 of the handset 110 so as to form an equilateral triangle, as in FIGS. 2 and 3 described above. In this embodiment, the microphones MC1 to MC3 are arranged in an equilateral triangle in order to enlarge the sound collection area in the same direction, but the microphones MC1 to MC3 are not necessarily limited to the equilateral triangle.

  In the third embodiment, similarly to the first embodiment, three microphone arrays MA1 to MA3 are configured from three microphones MC1 to MC3 (see FIG. 4 described above). Although details will be described later, the communication device 100B (sound pickup unit 120B) of the third embodiment performs area sound pickup for three sound pickup areas by a combination of the microphone arrays MA1 to MA3.

(C-2) Operation of Third Embodiment Next, an operation (a sound collection method according to the embodiment) of the third embodiment having the above-described configuration will be described.

The signal input unit 121 converts the acoustic signals collected by the three microphones MC1 to MC3 from analog signals to digital signals (x ₁ to x ₃ ) and supplies the digital signals to the frequency conversion unit 122.

The frequency conversion unit 122, for example, to convert from the area microphone signal time to the frequency domain using a fast Fourier transform _(X 1 ~X _3).

The directivity forming unit 123 forms directivity by BF using the input signal of each microphone that has been time-frequency converted by the frequency converting unit 122. In the third embodiment, the BF output by the microphone array MA1 _{Y 1,} the BF output by the microphone array MA2 _{Y 2,} and the BF output _{Y 3} by the microphone array MA3.

The target area sound extraction unit 124 uses the BF outputs Y ₁ , Y ₂ , and Y ₃ formed by the directivity forming unit 123 and uses a combination of Y ₁ -Y ₂ , Y ₂ -Y ₃ , and Y ₃ -Y ₁ . , Respectively.

In the third _embodiment, Y 1 "1" area (sound-pickup area) by a combination of -Y _2, combining area by a combination of Y 2 -Y ₃ (the sound-pickup area) of _2, Y 3 -Y ₁ Area (sound collection area) is referred to as “3”.

As shown in FIG. 6 described above, the area sound pickup has a characteristic that the sound pickup area spreads in a direction far from the microphone array. Therefore, BF outputs _Y 1 -Y ₂ area by area according to (microphone array MA1-MA2) 1, BF output _Y 2 -Y ₃ (microphone array MA2-MA3) 2, BF output _Y 3 -Y ₁ (microphone array MA3- The distribution of the sound collection area according to MA1) has an image as shown in FIG. In the third embodiment, the area sound pickup components (area sound pickup outputs) of the areas ₁ , ₂ , and ₃ are Z ₁ , Z ₂ , and Z ₃ .

  In the second embodiment, two sound pickup areas are considered to overlap, but in the third embodiment, three sound pickup areas are provided. Therefore, in the third embodiment, the overlapping pattern is more compared to the second embodiment, such as a portion where three sound collection areas overlap, a portion where two areas overlap, and an independent portion without overlap. It gets complicated.

Using the partial area integrating unit 127B in the area sound-pickup output Z _1, Z _2, Z _3, part 3 horn sound collecting areas overlap, part 2 horns sound collecting areas overlap, without duplication independent parts area sound-pickup of Each component is calculated. When calculating the area sound pickup component of each part, the partial area integration unit 127B sets each combination of the two sound pickup areas (combination of areas 1, 2; combination of areas 2, 3; combination of areas 3, 1). (Hereinafter, referred to as a “combination pattern”), the same method as in the second embodiment can be used. Specifically, the process of separating the area sound pickup components of the two sound pickup areas having the overlap area into the overlap area part and the independent area part in the partial area integration unit 127B is the same as in the first embodiment. It is.

  Hereinafter, the combination pattern of the areas 1 and 2 is referred to as a “first combination pattern”, the combination pattern of the areas 2 and 3 is referred to as a “second combination pattern”, and the combination pattern of the areas 3 and 1 is referred to as a “third combination pattern”. Combination pattern ".

  FIG. 17 is an explanatory diagram (image diagram) showing an exploded image of a combination pattern (first to third combination patterns) of two sound collection areas for three areas 1 to 3.

  FIG. 17A is a diagram in which three areas 1 to 3 are overlapped. FIGS. 17B to 17D are explanatory diagrams showing images decomposed into first to third combination patterns, respectively.

  First, the first combination pattern (combination pattern of areas 1 and 2) shown in FIG. 17B will be described as a representative example from the three combination patterns shown in FIGS. 17B to 17D.

Independent area component calculation unit 126B, by the SS an area sound-pickup output Z ₂ from the area sound-pickup output Z _1, the independent parts (the embodiment for areas 2 in area 1, referred to as "area A" 17B) is obtained (referred to as “ _VA ” in the third embodiment). Also, independent area component calculation unit 126B, by the SS an area sound-pickup output Z ₁ from the area sound-pickup output Z _2, the independent parts (the embodiment for areas 1 area 2, "area B" 17 (b)) (refer to FIG. 17 (b)) (referred to as “V _B ” in the third embodiment). At isolation area component calculation unit 126B, as in the second embodiment, the above equation (21) can be obtained by calculation formula (22), the area voice collecting component _V A, the _{V B.}

In the independent area component calculation unit 126B, the second combination pattern (the combination pattern of the areas 2 and 3) is similarly a part independent of the area 3 of the area 2 (referred to as “area C” in this embodiment). and ones; and FIG. 17 (c) refer) area sound-pickup components (in the third embodiment is referred to as "V _C"), independent part with respect to the area 2 of the area 3 (in the third embodiment , "Area D"; see FIG. 17 (c)) to obtain an area sound pickup component (referred to as "V _D " in the third embodiment). In the independent area component calculation unit 126B, the third combination pattern (the combination pattern of the areas 3 and 1) is similarly independent of the area 3 of the area 1 (in this embodiment, “area E”). The area sound pickup component (referred to as “ _VE ” in the third embodiment) of FIG. 17D) and a portion independent of the area 3 of the area 1 (in this embodiment, , is referred to as "area F"; the area sound-pickup component (the third embodiment of FIG. 17 (d) refer) can be obtained is called) and "V _F".

At isolation area component calculation unit 126B, can be obtained by (31) to (34) of the formula below, area sound-pickup component _{V C, V D, V E} , the _{V F.}

As described above, the independent area component calculation unit 126B calculates the area of the independent portion (areas A to F) generated by a combination of any two areas (any combination pattern) among the area 1, the area 2, and the area 3. it can be calculated for sound collection component _(V A ~V _F). Thus, independent area component calculation unit 126B, based on the area sound-pickup component of the independent part _(V A _{~V F),} calculates the area sound-pickup component independent area when the overlapping area 1-3 simultaneously be able to.

  Here, first, a process in which the independent area component calculation unit 126B obtains an area sound pickup component of an independent portion generated in the areas A and D will be described.

  FIG. 18 is an explanatory diagram showing an image of an independent portion generated in areas A and D.

Independent area component calculation unit 126B can be obtained and the area sound-pickup component V _A of the area A in the above calculations, the area sound-pickup Ingredient V _D of the area D.

Then, independent area component calculation unit 126B is the same calculation method as heretofore, by SS the area sound-pickup Ingredient V _D from the area voice collecting component V _A, and independently of the area D of the area A part An area sound pickup component (that is, an independent portion within the area A; referred to as “area Ad” in this embodiment) (referred to as “V _Ad ” in the third embodiment) can be obtained. In the independent area component calculation unit 126B, the area sound pickup component _VAd can be obtained by the following equation (35).

Thus, the independent area component calculation unit 126B, as shown in the following equation (36), by adding the area sound-pickup component V _Ad and area sound-pickup component V _D, across a range of areas A and D ( Hereinafter, an area sound _pickup component (hereinafter, referred to as “ _VA∨D ”) of “area A∨D” can be obtained.

  FIG. 19 is an explanatory diagram showing an image of areas A∨D in areas 1 to 3.

Note that, as _shown in the following equations (37) and (38), the area sound _pickup component VAＶD of the area _A∨D is a part independent of the area A of the area D (that is, the area A∨D). Independent portion; In this embodiment, an area sound pickup component (referred to as “area Da”) (referred to as “V _Da ” in the third embodiment) and an area sound pickup component VA of area _A are added. You may make it obtain.

As shown in FIG. 19, the area A∨D (the entire area between the area A and the area D) is the part that protrudes from the area 2 (the whole part that protrudes from the area 2 in the entire area of the areas 1 to 3). Therefore, as will be described later, the partial area integration unit 127B can calculate the area sound _pickup component (final output W) of the entire area 1 to 3 using the area sound _pickup component _VA∨D .

As described above, the independent area component calculation unit 126B can obtain the area sound _pickup component VA∨D of the area A∨D (the entire range of the area A and the area D).

In the examples of FIGS. 18 and 19 described above, the area A∨D (the entire range of the areas A and D; the part protruding from the area 3) is obtained by using the area sound pickup component of the independent part generated in the area A or the area D. Although the example in which the area sound _pickup component _VA∨D is obtained has been described, similarly, the entire range of the areas B and E (protruding from the area 1) is obtained using the area sound _pickup component of the independent area generated in the area B or the area E. Part; hereinafter, referred to as an area sound _pickup component (hereinafter, referred to as “ _VB∨E ”) or an area sound _pickup component of an independent area generated in the area C or the area F. , And the areas C and F (the part protruding from the area 3; hereinafter, referred to as “area C∨F”) may be obtained as the area pickup component (hereinafter, referred to as “ _VC∨F ”). .

  FIG. 20 is an explanatory diagram showing an image of an independent portion generated in areas B and E.

The independent area component calculation unit 126B calculates a portion independent of the area E of the area B (that is, an independent area within the area B; in this embodiment, the “area”) by a calculation method similar to that of the above-described areas A and D. Be ") (in the third embodiment, referred to as" _VBe "), and a portion of area E that is independent of area B (that is, an independent area within area E; In the embodiment, an area sound pickup component (referred to as “V _Eb ” in the third embodiment) of “area Eb” can be obtained (a specific calculation formula is as follows: Equation, Equation (41)). Then, independent area component calculation unit 126B, by using the area sound-pickup component _{V Be} or area sound-pickup component _{V Eb,} obtain area sound-pickup component _{V B∨E} area B∨E (portion protruding from the area 1) (Specific formulas are shown in the following formulas (40) and (42)).

  FIG. 21 is an explanatory diagram showing an image of an independent part generated in areas C and F.

The independent area component calculation unit 126B calculates a portion independent of the area F of the area C (that is, an independent area in the area C; in this embodiment, the “area”) by the same calculation method as in the above-described areas A and D. Cf ”(referred to as“ V _Cf ”in the third embodiment) and a portion independent of the area C of the area F (ie, an independent area in the area F; In the embodiment, an area sound pickup component (referred to as “V _Fc ” in the third embodiment) of “area Fc” can be obtained (the specific calculation formula is as follows (43)). Equation (45)). Then, the independent area component calculation unit 126B obtains an area sound _pickup component V _{C ∨F} of the area C∨F (the portion protruding from the area 3) by using the area sound _pickup component V _Cf or the area sound pickup component V _Fc. (Specific calculation formulas are shown in the following formulas (44) and (46)).

As described above, the independent area component calculation unit 126B calculates the area sound component of an area independent of the overlap of the areas 1 to 3 and then calculates the area sound component (V _A∨D , _VB∨E , or _VC∨F ).

In the partial area integration unit 127B, the area sound _pickup component (one of _VA∨D , _VB∨E , and _{VC 部分 F} ) of the part that protrudes from any one of the areas 1 to 3 calculated by the independent area component calculation unit 126B. ) And the area sound pickup components (corresponding area sound pickup components among Z1, Z2 and Z3) of the corresponding sound pickup areas are integrated (added) to cover the three areas (areas 1 to 3). An area sound pickup component (final output W) of the entire range can be obtained. As a result, the final output W calculated by the partial area integration unit 127B is a component that uniformly collects the entire range covered by the three sound collection areas (areas 1 to 3).

  The partial area integration unit 127B calculates the final output W by any of the following equations (47) to (49), depending on which area sound pickup component is calculated by the independent area component calculation unit 126B.

For example, when the independent area component calculation unit 126B calculates the area sound _pickup component VA∨D of the part (area A∨D) that protrudes from the area 2, the partial area integration unit 127B calculates the following equation (47). as shown, by adding the area sound-pickup component Z ₂ of area sound-pickup component V _A∨D and area 2, it is possible to obtain a final output W. Also, when the independent area component calculation unit 126B calculates the area sound _pickup component V _{B の} E of the part (area B∨E) that protrudes from the area 1, the partial area integration unit 127B calculates the following equation (48). As shown, the final output W can be obtained by adding the area sound _pickup component V _B∨E and the area sound _pickup component Z ₁ of the area 1. Further, when the independent area component calculation unit 126B calculates the area sound _pickup component VC∨F of the portion (area C∨F) that protrudes from the area 3, the partial area integration unit 127B calculates the following equation (49). as shown, by adding the area sound-pickup component Z ₃ of area sound-pickup component V _C∨F and area 3, it is possible to obtain a final output W.

As described above, in the sound pickup unit 120B, the independent area component calculation unit 126B picks up the area of an independent portion (any one of Ad, Da, Be, Eb, Cf, and Fc) independent of the overlap of the areas 1 to 3. The components (V _Ad , V _Da , V _Be , V _Eb , V _Cf , and V _Fc ) are separated, and three sound pickup areas (areas 1 to 3) are formed by using the separated sound pickup components of the independent portions. An area sound pickup component in which the entire range to be covered is uniformly picked up by the area is obtained. Thereby, the sound pickup unit 120B outputs the final output W (n) as the target sound picked up from the enlarged area.

  Then, the communication unit 130 transmits the audio data based on the final output W (n) to the communication device 200 via the communication path P.

  Then, the communication unit 230 of the communication device 200 supplies the output unit 140 with the audio data (audio data based on W (n)) received from the communication device 100. The output unit 140 supplies an audio signal based on the received audio data to the speaker 210 to output a sound (a sound output to the second user U2).

(C-3) Effects of Third Embodiment According to the third embodiment, the following effects can be obtained.

  In the sound pickup unit 120B of the third embodiment, the components of the respective areas overlapping with the independent areas with respect to the three or more sound pickup areas overlapping each other are calculated, and the components of each part are comprehensively calculated without duplication. Because of the integration, it is possible to simultaneously increase the sound pickup area and ensure uniformity of the sound pickup characteristics.

(D) Other Embodiments The present invention is not limited to the above embodiments, but may include modified embodiments as exemplified below.

  (D-1) In each of the above embodiments, the sound collection unit has been described as constituting a part of the communication device, but may be configured as an independent device. Further, in each of the above-described embodiments, the sound collecting unit is not configured to include the microphone array unit. However, the sound collecting unit and the microphone array unit may be configured as an integrated device.

  (D-2) In each of the above embodiments, an example is described in which the sound pickup device (sound pickup unit) of the present invention is applied to a device such as a handset or the like having a handheld transmitter (transmitter / receiver). The sound collection device of the present invention is applied to a headset or a wearable device (for example, a head-mounted display with a microphone, a neckband headphone with a microphone, or the like), and is attached to the first user U1 when worn by the first user U1. The area where the mouth is located is set as the target area, and microphones are installed at the respective vertices of the polygon (N-gon) around the mouth (mouthpiece), and the area sound pickup processing may be performed in the same manner as in the above embodiment. .

  (D-3) In the first and third embodiments, an example of area sound collection using three microphones MC1 to MC3 has been described. However, the number of microphones (microphones are arranged in the microphone array unit 111) is described. The number of sides (corners) of the polygon is not limited. For example, even if area pickup is performed in three or four directions, the number of microphones increases only slightly, and as a result, the increase in the amount of processing is limited. Specifically, for example, in the first and third embodiments, when four microphones are arranged at the corner vertices of a quadrangle, the number of microphones is the same as that of a conventional microphone even though four areas are picked up. Since it can be realized by the same four microphones as the two microphone array × 2 which is the minimum configuration of area sound pickup, it can be easily mounted on a device having a limited space such as the handset 110 with a simple configuration and a small processing amount.

  As described above, if the number of microphones (the number of polygons formed by the positions of the microphones) installed in the microphone array unit 111 increases, the direction of directivity (the direction of directivity of BF output) becomes diversified. The stability against the fluctuation of the mouth of the speaker (first user U1) (the fluctuation of the relative position between the mouthpiece 113 of the handset 110 and the mouth of the first user U1) is further improved.

  FIG. 22 is an explanatory diagram showing a configuration when the number of microphones in the microphone array unit 111 is four.

  In FIG. 22, four microphones MC1 to MC4 are arranged at the positions of the corner vertices of a square (square). The four microphones MC1 to MC4 are combined with adjacent microphones to form a microphone array MA701 formed by a pair of microphones MC1 and MC2, a microphone array MA702 formed by a pair of microphones MC2 and MC3, and a microphone MC3. A microphone array MA703 formed by a pair of MC4 and a microphone array MA704 formed by a pair of microphones MC4 and MC1 are formed. Further, these microarrays can collect sound in four areas by combination with adjacent microphone arrays (combination of microphone arrays sharing some microphones). For example, when the configuration of the four microphones MC1 to MC4 is applied to the microphone array unit 111, the sound collection unit 120 collects an area by combining the microphone arrays MA701 and MA702 and an area collection by combining the microphone arrays MA702 and MA703. It is possible to acquire each output (output of four area sound collections) of the sound, the area sound collection by the combination of the microphone arrays MA703 and MA704, and the area sound collection by the combination of the microphone arrays MA704 and MA701. Then, the sound pickup unit 120 can acquire a sound pickup result (for example, an added value or an average value of the outputs of the four area sound pickups) based on the output of the four area sound pickups.

  100, 100A, 100B: communication device, 110: handset, 111: microphone array, MC1 to MC6: microphone, 112: speaker, 113: mouthpiece, 114: earpiece, 115: handle, 120, 120A, 120B ... Sound collection unit, 121 ... Signal input unit, 122 ... Frequency conversion unit, 123 ... Directivity formation unit, 124, 124A ... Target area sound extraction unit, 125 ... Area sound addition unit, 126, 126B ... Independent area component calculation unit 127, 127B: partial area integration unit, 130: communication unit, 140: output unit, 200: communication device, 210: speaker, 220: microphone, 230: communication unit, 240: output unit, 250: sound collection unit, U1 .., The first user, U1a, the hand of the listener, U2, the second user, P, the communication path.

Claims (5)

Area sound pickup for acquiring area sound pickup components of a plurality of sound pickup areas based on a combination of two or more patterns of the microphone arrays based on input signals from a microphone array unit capable of forming a plurality of microphone arrays having different directivities. Means,
Based on an area sound pickup component of the sound pickup area of each pattern obtained by the area sound pickup means, for one or a plurality of the sound pickup areas, an area sound pickup component of an independent part which does not overlap with the other sound pickup areas. An independent area component extracting means to be obtained;
Using the area sound pickup component of the sound pickup area obtained by the area sound pickup means and the area sound pickup component of the independent part extracted by the independent area component extraction means, the area sound pickup means And a partial area integrating means for acquiring an area sound pickup component of an entire area covering the entire sound pickup area. The area sound pickup unit acquires area sound pickup components of the two sound pickup areas based on an input signal from the microphone array unit,
The independent area component extraction means is configured to determine the second sound collection area based on an area sound pickup component of the first sound pickup area and an area sound pickup component of the second sound pickup area acquired by the area sound pickup means. Obtains an area sound pickup component of an independent part that does not overlap with the first sound pickup area,
The partial area integration means integrates an area sound pickup component of the first sound pickup area and an area sound pickup component of an independent part of the second sound pickup area, thereby obtaining an area sound pickup of the entire area. The sound collection device according to claim 1, wherein a component is acquired. The area sound pickup means acquires area sound pickup components of the three sound pickup areas based on an input signal from the microphone array unit,
The independent area component extraction means integrates an area sound pickup component of a portion of the entire area that protrudes from a first sound pickup area with an area sound pickup component of the first sound pickup area, and The sound pickup device according to claim 1, wherein an area sound pickup component of the area is acquired. Computer
Area sound pickup for acquiring area sound pickup components of a plurality of sound pickup areas based on a combination of two or more patterns of the microphone arrays based on input signals from a microphone array unit capable of forming a plurality of microphone arrays having different directivities. Means,
Based on an area sound pickup component of the sound pickup area of each pattern obtained by the area sound pickup means, for one or a plurality of the sound pickup areas, an area sound pickup component of an independent part which does not overlap with the other sound pickup areas. An independent area component extracting means to be obtained;
Using the area sound pickup component of the sound pickup area obtained by the area sound pickup means and the area sound pickup component of the independent part extracted by the independent area component extraction means, the area sound pickup means A sound collection program characterized by functioning as partial area integration means for acquiring area sound pickup components of all areas covering all sound pickup areas. In the sound pickup method performed by the sound pickup device,
Comprising an area sound pickup means to acquire, an independent area component extraction means, and a partial area integration means,
The area sound pickup unit is configured to pick up an area sound of a plurality of sound pickup areas based on a combination of two or more patterns of the microphone arrays based on input signals from a microphone array unit capable of forming a plurality of microphone arrays having different directivities. Get the ingredients,
The independent area component extraction unit does not overlap one or more of the sound collection areas with another of the sound collection areas based on an area sound collection component of the sound collection area of each pattern acquired by the area sound collection unit. Get the area sound pickup component of the independent part,
The partial area integration means uses the area sound pickup component of the sound pickup area acquired by the area sound pickup means and the area sound pickup component of the independent part extracted by the independent area component extraction means, and A sound collection method characterized by obtaining area sound pickup components of all areas covering all of the sound pickup areas obtained by the sound pickup means.

Images