JP2020048038A - Sound collection device, program, and method - Google Patents

Abstract

To provide a sound collection device for efficiently and stably perform area sound collection.SOLUTION: A sound collection device includes: means for acquiring area sound collection components of multiple sound collection areas on the basis of two or more patterns of combination of microphone arrays in dependence upon an input signal from a microphone array part capable of forming multiple different kinds of directional microphone arrays; partial area component calculation means for performing acquisition on the basis of area sound collection components of the respective sound collection areas concerning respective area sound collection components of a partial area where two or more sound collection areas divided from a whole area for covering all of the acquired sound collection areas are overlapped and also partial areas where the sound collection areas are not overlapped; and means for selecting area sound collection components of one or multiple partial areas from the area sound collection components of the respective partial areas, so as to acquire a sound collection result based on the selected area sound collection components.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a sound collection device, a program, and a method, and can be applied to, for example, a voice communication system used in a noisy environment.

  When a voice communication system or a voice recognition application system is used in a noisy environment, ambient noise mixed with necessary target voice is a troublesome factor that hinders good communication and lowers a voice recognition rate. Conventionally, in an environment where a plurality of sound sources exist, a beamformer using a microphone array has been used as a technique for obtaining a desired target sound by separating and collecting only a sound in a specific direction to avoid mixing of unnecessary sound. (Beam Former; hereinafter also referred to as “BF”; see Patent Documents 2 and 3). BF is a technique for forming directivity by using a time difference between signals reaching each microphone. However, if there is another sound source around an area for sound collection (hereinafter, referred to as “target area”) using only BF, the sound existing in the target area (hereinafter, referred to as “target area sound”) Only difficult to pick up sound. For this reason, an area sound collecting method for collecting a target area by using a plurality of microphone arrays has been proposed in Patent Document 1 and the like.

  FIG. 25 is an explanatory diagram (graph) showing a process of collecting a target area sound from a sound source in the target area using two microphone arrays MA100 and MA200.

  FIG. 25A is an explanatory diagram showing a configuration example of each of the microphone arrays MA100 and MA200. FIGS. 25 (b) and 25 (c) are diagrams (images in the form of graphs) showing the BF outputs of the microphone arrays MA100 and MA200 shown in FIG. 25 (a) in the frequency domain. In FIG. 25, each of the microphone arrays MA100 and MA200 includes two microphones ch1 and ch2, respectively.

  In the conventional area sound pickup, as shown in FIG. 25A, sound is picked up by intersecting the directivities of the microphone arrays MA100 and MA200 in different areas (target areas) to be picked up from different directions. In the state of FIG. 25A, the directivity of each of the microphone arrays MA100 and MA200 includes not only the sound existing in the target area (target area sound) but also noise in the target area direction (non-target area sound). I have. However, as shown in FIGS. 25B and 25C, when the directivity of the microphone arrays MA100 and MA200 is compared in the frequency domain, the target area sound component is included in both outputs, but the non-target area The sound component will be different for each microphone array. In the conventional area sound collecting technique, by utilizing such characteristics, only the target area sound can be extracted by suppressing components other than those commonly included in the BF outputs of the two microphone arrays MA100 and MA200.

JP 2014-072708 A JP 2005-195555 A JP-A-2006-127457

  By the way, an emergency vehicle is provided with a communication handset (handset) as a means of emergency communication from a fire site where a siren sounds and an emergency site to a command center (fire department). The handset mounted on a conventional emergency vehicle is in a loud noise environment, so communication from the site is drowned out by surrounding noise and sent to the headquarters (for example, the headquarters that commands the crew of the emergency vehicle). Inaccurate information cannot be provided, resulting in erroneous information, which may hinder accurate judgment and delay response. For this reason, the use of various noise removal techniques for handsets has been studied, but there have been many problems in introducing such techniques as securing speech quality and increasing costs. In such a usage environment, the above-described area sound collection technology is expected as an effective solution. For example, two microphone arrays are installed around the mouthpiece of the handset, and the directivity of each of the two microphone arrays intersects in front of the mouthpiece to function as an area sound pickup, thereby providing a large siren or the like. Noise can be eliminated, and only the voice of the sender, such as a firefighter, can be accurately transmitted to the headquarters and other places.

  At least two microphone arrays are required to achieve area sound pickup. On the other hand, the size of the mouthpiece in the handset is as small as about 6 cm in diameter, and when two microphone arrays are mounted to achieve area sound collection, the microphone arrays are placed very close together. There is a need to. As a result, in the area sound collection using the handset, the sound collection area is limited to a very small area immediately near the transmitter. However, when the conventional area sound collection processing is applied to the handset, the manner of holding the handset and the size of the face differ depending on the user (speaker), and the mouth has the above-described narrow sound collection area (set for the handset). Sound pickup area). In this case, if the mouth of the user (speaker) shifts from the sound collection area of the handset, distortion or dropout of the collected sound occurs, and there is a problem that stable sound collection cannot be performed.

  Therefore, a sound collection device, a program, and a method that can efficiently and stably perform area sound collection are desired.

  According to the first aspect of the present invention, there is provided a sound collecting apparatus comprising: (1) a plurality of microphone arrays based on a combination of two or more patterns based on input signals from a microphone array unit capable of forming a plurality of microphone arrays having different directivities; Area pickup means for acquiring an area pickup component of the sound pickup area; and (2) two or more sound pickup areas divided from all areas covering all of the sound pickup areas acquired by the area pickup means. The sound pickup components of each of the overlapping partial areas and the partial areas that do not overlap with each other are obtained based on the sound pickup components of the sound pickup areas of the respective patterns obtained by the area sound pickup means. A partial area component calculating means; and (3) extracting an area sound collecting component of one or more partial areas from the area sound collecting components of the partial area calculated by the partial area component calculating means. -Option is characterized by having a partial area selecting means for obtaining a sound collecting results based on the selected area sound-pickup component.

  A sound collection program according to a second aspect of the present invention provides a computer for: (1) combining two or more patterns of the microphone array based on an input signal from a microphone array unit capable of forming a plurality of microphone arrays having different directivities. And (2) two or more sound pickup areas separated from all areas covering all of the sound pickup areas acquired by the area sound pickup means. For each of the area sound pickup components of the partial area where the sound area overlaps and the partial area where the sound pickup areas do not overlap each other, based on the area sound pickup component of the sound pickup area of each pattern acquired by the area sound pickup means. And (3) one or a plurality of partial area components from the area sound component of the partial area calculated by the partial area component calculating means. Select area sound-pickup components A, characterized in that to function as a partial area selecting means for obtaining a sound collecting results based on the selected area sound-pickup component.

  According to a third aspect of the present invention, there is provided a sound collecting method performed by a sound collecting device, comprising: (1) an area sound collecting unit, a partial area component calculating unit, and a partial area selecting unit; (3) acquiring an area sound pickup component of a plurality of sound pickup areas based on a combination of two or more patterns of the microphone array based on an input signal from a microphone array unit capable of forming a microphone array having different directivities; The partial area component calculating means includes: a partial area in which two or more sound collecting areas divided from an entire area covering all of the sound collecting areas acquired by the area sound collecting means overlap each other; For each area sound pickup component of a non-overlapping partial area, the sound pickup component is obtained based on the area sound pickup component of the sound pickup area of each pattern obtained by the area sound pickup means. The partial area selecting means selects an area sound collecting component of one or a plurality of partial areas from the area sound collecting components of the partial area calculated by the partial area component calculating means, and collects sound based on the selected area sound collecting component. The method is characterized in that a result is obtained.

  ADVANTAGE OF THE INVENTION According to this invention, the sound collection device which performs area sound collection efficiently and stably can be provided.

FIG. 2 is a block diagram illustrating a configuration of each device according to the first embodiment (including a functional configuration of a sound collection unit (sound collection device) according to the embodiment). FIG. 2 is a diagram (perspective view) illustrating a use state of the handset according to the first embodiment. It is the figure which expanded and showed the mouthpiece part of the handset which concerns on 1st Embodiment. FIG. 3 is an explanatory diagram (image diagram) illustrating a configuration example of a microphone array formed by three microphones. FIG. 9 is an explanatory diagram (image diagram) illustrating an area sound pickup process corresponding to each combination (combination pattern) of microphone arrays formed by three microphones. FIG. 11 is a diagram illustrating a distribution of sensitivity (calculated sensitivity distribution) of area sound pickup when the directivities of two microphone arrays cross each other. It is a block diagram which shows the structure regarding the subtraction type BF when the number of microphones is two. FIG. 4 is a diagram illustrating a directional characteristic formed by a subtraction type BF using two microphones. FIG. 13 is a block diagram illustrating a configuration of each device related to the second embodiment. FIG. 9 is a diagram illustrating an arrangement of six microphones and a configuration example of a microphone array in a microphone array unit according to a second embodiment. FIG. 10 is an explanatory diagram showing a distribution of sound collection areas where a target area sound extraction unit according to the second embodiment performs area sound collection. It is explanatory drawing which showed the independent area which does not overlap in a plurality of sound collection areas, and the overlap area which overlaps in a plurality of sound collection areas in the sound collection area which concerns on 2nd Embodiment. It is explanatory drawing which showed the composition image (power for every component) of each sound collection area which concerns on 2nd Embodiment in the form of a bar graph. FIG. 14 is an explanatory diagram showing a procedure of a process (part 1: a process of acquiring an area sound pickup component of an independent area) by the independent area component calculation unit according to the second embodiment. FIG. 11 is an explanatory diagram showing a procedure of a process (part 2: a process of acquiring an area sound pickup component of an overlapping area) by the independent area component calculation unit according to the second embodiment. FIG. 13 is a block diagram illustrating a configuration of each device related to a third embodiment. FIG. 14 is a diagram illustrating a plan view of a communication device (smart phone) according to a third embodiment. It is explanatory drawing shown about the image of three sound collection areas concerning 3rd Embodiment. It is explanatory drawing shown about the decomposition | disassembly image of the combination pattern (1st-3rd combination pattern) in three sound collection areas concerning 3rd Embodiment. It is explanatory drawing shown about the image of the independent part produced | generated in area A and D concerning 3rd Embodiment. It is explanatory drawing shown about the image of the independent part produced | generated in area B and E concerning 3rd Embodiment. It is explanatory drawing shown about the image of the independent part produced | generated in area C and F concerning 3rd Embodiment. It is explanatory drawing shown about the image of the overlapping part of three areas which concerns on 3rd Embodiment. FIG. 9 is an explanatory diagram showing a configuration (a configuration of a modified example according to the embodiment) when the number of microphones in the microphone array unit according to the embodiment is four. FIG. 9 is an explanatory diagram illustrating a configuration example in a case where directivity by a beam former (BF) of two microphone arrays is directed to a target area from different directions in a conventional sound collection device.

(A) First Embodiment Hereinafter, a first embodiment of a sound collection device, a program, and a method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which a sound collection device, a program, and a method of the present invention are applied to a sound collection unit will be described.

  First, the basic principle of the area sound collection processing using the microphone array in this embodiment will be described with reference to FIGS.

  By arranging a microphone at the position of each vertex of the polygon, a plurality of area sound pickups can be constructed in the direction of the center of the polygon.

  For example, when a configuration of area sound collection using three microphones is considered, as shown in FIG. 4, up to three microphone arrays (three microphone arrays having different directivity directions) are formed by combining microphones. Can be set. As shown in FIG. 4, among the three microphones ch1 to ch3, a microphone array MA301 that pairs the microphones ch1 and ch2, a microphone array MA302 that pairs the microphones ch2 and ch3, and a microphone that pairs the microphones ch3 and ch1. The array MA303 can be set.

  Further, in the configuration of the three microphones ch1 to ch3, as shown in FIG. 5, it is possible to perform area sound pickup according to a combination (three combinations of patterns) of the three microphone arrays MA301, MA302, and MA303. .

  In FIG. 5A, the directivity of the microphone array MA301 is shown by a one-dot chain line, and the directivity of the microphone array MA302 is shown by a two-dot chain line. Also, in FIG. 5B, the directivity of the microphone array MA302 is shown by a dashed line, and the directivity of the microphone array MA303 is shown by a two-dot chain line. Further, in FIG. 5C, the directivity of the microphone array MA301 is shown by a one-dot chain line, and the directivity of the microphone array MA303 is shown by a two-dot chain line. Furthermore, in FIG. 5A, the sound collection area A301 corresponding to the combination (pattern) of the microphone arrays MA301 and MA302 is hatched (hatched). In FIG. 5B, the sound collection area A302 corresponding to the combination (pattern) of the microphone arrays MA302 and MA303 is hatched (hatched). Further, in FIG. 5C, the sound collection area A303 corresponding to the combination (pattern) of the microphone arrays MA301 and MA303 is hatched (hatched).

  As shown in FIG. 5, in the configuration of the three microphones ch <b> 1 to ch <b> 3, any microphone array has an angle between the microphone arrays (line segments connecting the positions of the two microphones constituting the microphone array). By intersecting the directivity of each other, it is possible to realize different area sound pickup (area sound pickup in different areas) for each combination.

  On the other hand, the sound collection area of the area sound collection using the microphone array has a property of expanding in front of the microphone array (farther from the microphone array). Hereinafter, the property will be described with reference to FIG.

  FIG. 6 is a diagram illustrating a distribution of sensitivity (calculated sensitivity distribution) of area sound pickup when the directivities of the two microphone arrays MA400 and MA500 intersect at a right angle to each other. In other words, FIG. 6 illustrates the sensitivity of area sound pickup in a region where the directivities of the two microphone arrays MA400 and MA500 intersect and in the vicinity thereof. In FIG. 6, the microphone arrays MA400 and MA500 have two microphones ch1 and ch2, respectively. In FIG. 6, the sensitivity of area sound pickup is divided into five stages (0 to -5 dB, -5 to -10 dB, -10 to -15 dB, -15 to -20 dB, -20 to -25 dB), and Are provided with different patterns (patterns). As shown in FIG. 6, it can be seen that a region with high sensitivity extends in a direction distant from microphone arrays MA400 and MA500 (that is, in the lower right direction).

  Therefore, the combination of FIG. 5A (combination of microphone arrays MA301 and MA302), the combination of FIG. 5B (combination of microphone arrays MA302 and MA303), and the combination of FIG. The sound pickup area (area distribution of sensitivity of area pickup) of the area sound pickup by each combination differs depending on the combination of the microphone arrays, and an overlapping part and a part not overlapping (a part where the sensitivity distribution coincides with a part which does not coincide) are different. Will happen.

  That is, as shown in FIG. 5, in a configuration of three microphones ch1 to ch3, area pickup is performed by a combination of two or three different microphone arrays, and if the respective pickup results are added, one microphone is obtained. It is possible to collect sound in a wider area than the sound collection area realized by the combination of the arrays.

  Therefore, in this embodiment, a combination (combination) of a plurality of different microphone arrays among a plurality of microphone arrays formed by microphones arranged at the corner vertices of a polygon (N polygon; N is an integer of 3 or more). ), And the result of adding or averaging the results of area pickup (output of area pickup) is treated as the final sound pickup result of the target area. As a result, in the area sound pickup processing of this embodiment, as a result, a more robust area sound pickup (more stable) with respect to the difference in the position of the mouth of the speaker (the position of the mouth of the speaker as viewed from the transmitter). Area sound collection).

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing a configuration of each device related to this embodiment.

  FIG. 1 illustrates a communication device 100 including a sound collection unit 120 according to this embodiment, and a communication device 200. In FIG. 1, the communication devices 100 and 200 are configured to be able to communicate with each other via a communication path P.

  The communication device 100 collects the voice (sound) uttered by the first user U1, transmits the voice data of the collected voice to the communication device 200 via the communication path P, and receives the voice data from the communication device 200. It is a device that outputs a sound based on the sound data (a sound uttered by the second user U2). Further, the communication device 200 collects voice (sound) uttered by the second user U2, transmits voice data of the collected voice to the communication device 100 via the communication path P, and This is a device for outputting a voice based on the received voice data (voice uttered by the first user U1).

  The first user U1 corresponds to, for example, a crew member appearing in an emergency vehicle such as an ambulance or a fire engine, and the second user U2 corresponds to, for example, a remote location (for example, a command center that commands an emergency vehicle). Commanders in charge correspond to this.

  The communication path P is not limited to wired or wireless, and various connection means and connection configurations (network configurations) can be applied.

  Next, an outline of the configuration of the communication device 100 will be described with reference to FIG.

  The communication device 100 includes a handset 110, a sound collection unit 120, a communication unit 130, and an output unit 140.

  The handset 110 includes a microphone array unit 111 composed of three microphones MC1 to MC3 (3ch microphones) and a speaker 112.

  The communication unit 130 is a communication interface for communicating with the communication device 200 via the communication path P.

  The sound collection unit 120 collects a sound (sound) uttered by the first user U1 based on the acoustic signal captured by the microphone array unit 111. Then, the communication unit 130 transmits the audio data of the audio collected by the sound collection unit 120 to the communication device 200 side.

  The output unit 140 acquires voice data (voice data of voice uttered by the second user U2) from the communication device 200 via the communication unit 130, supplies an audio signal based on the voice data to the speaker 112, 112 causes the sound signal to be output as sound.

  Although the hardware configuration of the communication device 100 is not limited, in the example of this embodiment, as illustrated in FIG. It shall be. Note that the communication device 100 does not necessarily need to include the handset 110, and the entire housing (chassis) functions substantially as a handset like a smartphone (for example, the communication device 100 may be provided in a part of the smartphone housing). A configuration in which a talk is set) may be used.

  Next, an outline of the configuration of the communication device 200 will be described with reference to FIG.

  The communication device 200 includes a speaker 210, a microphone 220, a communication unit 230, an output unit 240, and a sound collection unit 250. Although the hardware configuration of the communication device 200 is not limited, for example, various telephone devices (for example, speakerphones and the like) can be applied.

  The communication unit 230 is a communication interface for communicating with the communication device 200 via the communication path P.

  The sound collection unit 250 collects a sound (sound) uttered by the second user U2 based on the acoustic signal captured by the microphone 220. Then, the communication unit 230 transmits the sound data of the sound collected by the sound collection unit 250 to the communication device 100 side.

  The output unit 240 acquires voice data (voice data of voice uttered by the first user U1) from the communication device 100 via the communication unit 230, supplies an audio signal based on the voice data to the speaker 210, The sound signal is output to the sound signal 210.

  Next, a detailed configuration of the sound pickup unit 120 will be described with reference to FIG.

  The sound collection unit 120 includes a signal input unit 121, a frequency conversion unit 122, a directivity forming unit 123, a target area sound extraction unit 124, and an area sound addition unit 125.

  For example, the sound collection unit 120 may cause a computer including a processor, a memory, and the like to execute a program (including the sound collection program according to the embodiment), but even in that case, functionally, It can be shown as in FIG. Details of the processing of each component of the sound pickup unit 120 will be described later.

  Next, the configuration of the handset 110 as a handset will be described with reference to FIGS.

  FIG. 2 is a perspective view showing a state where the handset 110 is being held by the hand U1a of the first user U1.

  As shown in FIG. 2, the handset 110 includes a rod-shaped handle 115 for the first user U1 (hand U1a) to hold, and a mouthpiece 113 (speaker) provided at one end of the handle 115. And an earpiece 114 (receiver) provided at the other end of the handle 115.

  FIG. 3 is an enlarged view of the mouthpiece 113 of the handset 110.

  As shown in FIG. 2, a speaker 112 is arranged in the earpiece 114. As shown in FIGS. 2 and 3, microphone array unit 111 (microphone MC1 to MC3) is arranged in mouthpiece 113 having a circular surface.

  Next, the configuration of the microphone array unit 111 will be described with reference to FIGS.

  In the example of this embodiment, the microphone array unit 111 has a configuration including three microphones MC1 to MC3.

  As shown in FIG. 2, when the first user U1 holds the communication device 100 with the hand U1a and presses the speaker SP to the ear, the first user U1 surrounds the mouthpiece 113 where the mouth of the first user U1 is located (see FIG. 2). Three microphones MC <b> 1 to MC <b> 3 are arranged around the part closest to the mouth of the first user U <b> 1).

  In the handset 110 shown in FIGS. 2 and 3, each position (the center position of each microphone) of the three microphones MC <b> 1 to MC <b> 3 constituting the microphone array unit 111 is similar to the configuration shown in FIGS. 4 and 5 described above. , Around the mouthpiece 113 so as to be the vertices of an equilateral triangle. In FIGS. 2 and 3, each side of the triangle formed by the microphones MC1 to MC3 is set to have the same distance (a triangle formed by the microphones MC1 to MC3 is an equilateral triangle) in order to enlarge the sound pickup area in the same direction. Also, the angles of the angles may not all be the same.

  As shown in FIG. 3, in the following, in the microphone array unit 111, the microphone array paired with the microphones MC1 and MC2 is MA1, the microphone array paired with the microphones MC2 and MC3 is MA2, and the microphones MC3 and MC1 are paired. Is referred to as MA3.

(A-2) Operation of First Embodiment Next, the operation of this embodiment having the above-described configuration (sound collection method according to the embodiment) will be described.

  In the communication device 100, the sound collection unit 120 performs a target area sound collection process of collecting the target area sound of the target area using the acoustic signals supplied from the microphones MC1 to MC3 of the microphone array unit 111.

  The following description focuses on the operation inside the sound pickup unit 120 included in the communication device 100.

  The signal input unit 121 converts an acoustic signal collected by each of the microphones MC <b> 1 to MC <b> 3 from an analog signal to a digital signal, and supplies the digital signal to the frequency conversion unit 122. Then, the frequency conversion unit 122 converts the microphone signal from the time domain to the frequency domain using, for example, fast Fourier transform. The directivity forming unit 123 forms directivity by BF.

  Here, the formation of directivity by BF will be described with reference to FIGS.

  BF is a technique for forming a directivity of sound collection using a time difference between signals reaching respective microphones in a microphone array (see Non-Patent Document 1). BFs are roughly classified into two types, an addition type and a subtraction type. Here, a subtraction type BF that can form directivity with a small number of microphones will be described.

  FIG. 7 is a block diagram showing a configuration relating to the subtraction type BF 600 when the number of microphones is two (MC1, MC2).

  FIG. 8 is a diagram illustrating a directional characteristic formed by a subtraction type BF 600 using two microphones MC1 and MC2.

The subtraction type BF 600 first calculates the time difference between the signals that arrive at the microphones MC1 and MC2 of the sound (hereinafter, referred to as “target sound”) existing in the target direction by the delay unit 610, and adds a delay to the calculated signal. Adjust the sound phase. The time difference is calculated by equation (1). Here, d indicates the distance between the microphones MC1 and MC2, c indicates the speed of sound, and τ _i indicates the amount of delay. Θ _L indicates an angle from a vertical direction to a target direction with respect to a straight line connecting the positions of the microphones MC1 and M2.

Here, when the blind spot exists in the direction of the microphone MC1 with respect to the centers of the microphones MC1 and MC2, the delay unit 610 performs a delay process on the input signal x ₁ (t) of the microphone MC1. Thereafter, the subtractor 620 performs a subtraction process according to the equation (2). In the subtractor 620, this subtraction processing can be similarly performed in the frequency domain, and in that case, the equation (2) is changed to the equation (3).

Here, when θ _L = ± π / 2, the formed directivity is a cardioid type single directivity as shown in FIG. 8A, and when θ _L = 0, π, the formed directivity is FIG. An eight-shaped bidirectional pattern as shown in FIG. Further, the subtractor 620 can form a directivity that is strong in a bidirectional blind spot by using a process of a spectral subtraction method (hereinafter, also simply referred to as “SS”). The directivity by the SS is formed in all frequencies or a designated frequency band according to the equation (4). (4) In the formula, is used to input signals _{X 1} microphone MC1, it is possible to obtain the same effect input signal _{X 2} microphones MC2. Here, n is a frame number, and β is a coefficient for adjusting the strength of the SS. If the value becomes negative at the time of subtraction, the subtractor 620 may perform flooring processing of replacing the value with 0 or a value obtained by reducing the original value. In this method, a sound existing in a direction other than a target direction (hereinafter, referred to as a “non-target sound”) is extracted due to bidirectional characteristics, and an amplitude spectrum of the extracted non-target sound is subtracted from an amplitude spectrum of an input signal. Thus, the target sound can be emphasized.

  By the way, when it is desired to collect only a target area sound existing in a specific target area, a sound source existing on a line in the same direction as that area (hereinafter, referred to as “non-target area sound”) is used simply by using the subtraction type BF. Call) will also pick up sound.

  Therefore, the directivity forming unit 123 uses the area sound collection processing (a plurality of microphone arrays are used to direct the directivity from different directions to the target area, and intersect the directivity at the target area). In this case, the processing for collecting the target area sound will be described. Specifically, the directivity forming unit 123 may perform the area sound pickup processing by the following processing.

The directivity forming unit 123 forms the directivity of each of the microphone arrays MA1 to MA3 with the BF toward the inside of the triangle (the triangle formed by the microphones MC1 to MC3). Then, the directivity forming unit 123 supplies the BF outputs Y ₁ (n), Y ₂ (n), and Y ₃ (n) of the microphone arrays MA1, MA2, and MA3 to the target area sound extracting unit 124.

The target area sound extraction unit 124 extracts an area sound using the BF outputs Y ₁ (n), Y ₂ (n), and Y ₃ (n) of the microphone arrays MA1, MA2, and MA3 formed by the directivity forming unit 123. I do. As described above, each BF output (Y ₁ (n), Y ₂ (n), Y ₃ (n)) is centered (inside the triangle) from each side of the triangle (a triangle formed by the microphones MC1 to MC3). Direction). Therefore, in each BF output, in any two combinations (combination patterns), the two directivities intersect near the center of the triangle, so that the target area sound extraction unit 124 uses the area sound collection method described below. Thus, it is possible to extract sounds in an area where the directivities cross each other. Here, as a representative, the BF output _Y 1 of the microphone array MA1 (n), will be described using the BF output _Y 2 of the microphone array MA2 (n). The target area sound extraction unit 124 SSs Y ₁ (n) and Y ₂ (n) according to the formula (5) or (6), and non-target area sounds N _1-1 (n) existing in the direction of the target area. , N _1-2 (n). Here, α ₁ and α ₂ are correction coefficients for correcting a difference in signal level caused by a difference in distance between the target area and each microphone array, and should be calculated one by one by a predetermined process. Although described in Document 1, here, for simplicity, the distance between the target area and each microphone array is the same (α ₁ (n) = α ₂ (n) = 1), and (5), (6) Expression) is replaced by Expressions (7) and (8).

Thereafter, the target area sound extraction unit 124 extracts the non-target area sound from each BF output according to the equations (9) and (10) to extract the target area sound. Here, γ ₁ (n) and γ ₂ (n) are coefficients for changing the strength at the time of SS.

In destination area sound extraction unit 124, emphasized sound _Z 1-1 _(n), but may be output to any of the _{Z 1-2} (n), the microphone array _{Z 1-1} (n) here MA1- It is used as an area sound pickup output Z ₁ (n) based on a combination (combination pattern) of the microphone arrays MA2.

The destination area sound extraction unit 124 and similarly, the area sound-pickup output _Z 2 by the combination of the microphone array MA2- microphone array MA3 (n), and the area sound-pickup output _Z 3 by a combination of the microphone array MA3- microphone array MA1 (n ) Is extracted and supplied to the area sound adding unit 125.

As shown in FIG. 2, each of the microphones MC <b> 1 to MC <b> 3 is mounted in a narrow range of several centimeters in the mouthpiece 113 of the handset 110. Therefore, the microphone arrays MA1, MA2, and MA3 are arranged very close (dense), and the sound collection area is also limited to a narrow area in front of the mouthpiece 113. However, as shown in FIG. 6 described above, it has been found that the sound pickup area by the area sound pickup has a characteristic of extending in the far direction of the two microphone arrays. Therefore, if sound pickup areas (sound pickup areas corresponding to Z ₁ (n), Z ₂ (n), and Z ₃ (n)) extending in three different directions are overlapped, a single sound pickup area (single sound pickup area) is obtained. A wider range of area sound collection is possible as compared to a sound collection area corresponding to any one of Z ₁ (n), Z ₂ (n), and Z ₃ (n).

Therefore, the area sound adding unit 125 adds or averages the _three outputs Z ₁ (n), Z ₂ (n), and Z ₃ (n) of the sound pickup and outputs the final output (sound pickup result) W ( n) is generated and output as a sound collection result of the sound collection unit 120. The area sound addition unit 125 considers that there is a part where the areas overlap in the addition processing, and the addition value (Z ₁ (n) + Z ₂ (n), + Z ₃ ( n)) may be averaged or multiplied by a gain adjustment coefficient α as shown in equation (11). The area sound adding unit 125 adds (or averages) only two or more outputs of three area sound pickup outputs (Z ₁ (n), Z ₂ (n), and Z ₃ (n)). ) May be performed. For example, the area sound addition unit 125 may perform a process of adding (or averaging) only two outputs of the three area sound pickup outputs.

  As described above, the sound pickup unit 120 outputs the final output W (n) as the target sound picked up from the enlarged area. At this time, the sound pickup unit 120 may output W (n) as frequency-time converted audio data.

  Then, the communication unit 130 transmits the audio data based on the final output W (n) to the communication device 200 via the communication path P.

  Then, the communication unit 230 of the communication device 200 supplies the output unit 140 with the audio data (audio data based on W (n)) received from the communication device 100. The output unit 140 supplies an audio signal based on the received audio data to the speaker 210 to output a sound (a sound output to the second user U2).

(A-3) Effects of First Embodiment According to this embodiment, the following effects can be obtained.

  In the sound pickup unit 120 of this embodiment, area sound pickup is performed from different directions, and by adding them, an area sound pickup wider than the conventional area pickup using one set (two) microphone arrays is performed. It is possible to form a sound collecting area having a characteristic (enlarged sound collecting area). With this, in the sound pickup unit 120, when performing area sound pickup using the microphones MC1 to MC3 attached to the mouthpiece 113 of the handset 110, the mouth of the speaker (first user U1) and the mouthpiece are performed. Even when the relative position with respect to 113 is shifted, stable voice pickup is possible.

(B) Second Embodiment Hereinafter, a second embodiment of a sound collection device, a program, and a method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which a sound collection device, a program, and a method of the present invention are applied to a sound collection unit will be described.

  As described above, the sound pickup unit 120 of the first embodiment performs area sound pickup from different directions and superimposes (adds) them to use a conventional set (two) of microphone arrays. The sound pickup area (enlarged sound pickup area) which is wider than the sound pickup area and has the same directionality is formed.

  However, as in the sound pickup unit 120 of the first embodiment, an attempt to expand the sound pickup area, on the other hand, suppresses surrounding unnecessary sounds by picking up sound only in a specific area, thereby reducing the target sound. There is a possibility that the effect of emphasizing the area sound collection may be reduced.

  Therefore, in the sound pickup unit 120A of the second embodiment, in order to solve the above-described problem in the first embodiment, the sound pickup area is expanded and the deterioration of the target sound enhancement performance is suppressed. ing.

(B-1) Configuration of Second Embodiment FIG. 9 is a block diagram showing a configuration of each device related to the second embodiment. In FIG. 9, the same or corresponding portions as those in FIG. 1 described above are denoted by the same reference numerals or corresponding reference numerals.

  In the second embodiment, the communication device 100 is replaced with a communication device 100A. In the communication device 100A according to the second embodiment, the microphone array unit 111 and the sound pickup unit 120 are replaced with the microphone array unit 111A and the sound pickup unit 120A.

  Next, an internal configuration of the sound pickup unit 120A according to the second embodiment will be described.

  Next, the internal configuration of the sound pickup unit 120A will be described with reference to FIG.

  The sound collecting unit 120A is different from the first embodiment in that the target area sound extracting unit 124 is replaced with the target area sound extracting unit 124A, and the area sound adding unit 125 is excluded. The sound pickup unit 120A is different from the first embodiment in that a partial area component calculation unit 126 and a partial area selection unit 127 are added.

  Next, the configuration of the microphone array unit 111A according to the second embodiment will be described.

  As shown in FIG. 9, in the second embodiment, the microphone array unit 111A has six microphones MC1 to MC6.

  FIG. 10 is a diagram illustrating an arrangement of six microphones MC1 to MC6 in the microphone array unit 111A and a configuration example of the microphone array.

  As shown in FIG. 10, the six microphones MC1 to MC6 constituting the microphone array unit 111A include three microphone arrays MA1 (a microphone array having the microphones MC1 and MC2 as a pair) and MA2 (a microphone array having two microphones as a pair). The microphones MC3 and MC4 constitute a pair of microphone arrays, and the MA3 (microphones MC5 and MC6 constitute a pair of microphone arrays).

  From the viewpoint of increasing the number of divisions and extracting the target sound from the pinpoint area, it is desirable to have a configuration of three or more areas. However, in the second embodiment, in order to easily understand the principle of the present invention, the overlap will be described. An example will be described in which area sound pickup is performed for the two areas that are provided. A configuration using three overlapping areas will be described in a third embodiment described later.

(B-2) Operation of Second Embodiment Next, an operation (a sound collection method according to the embodiment) of the second embodiment having the above-described configuration will be described.

Signal input unit 121, a sound signal picked up by the six microphones MC1 to MC6, respectively converted from analog signals to digital signals _x 1 ~x _6, and supplies the frequency conversion section 122.

The frequency conversion unit 122 converts the microphone signals x _{1 to} x ₆ from time domain to frequency domain signals X _{1 to} X ₆ using, for example, fast Fourier transform.

The directivity forming unit 123 forms directivity by BF using the input signal of each microphone that has been time-frequency converted by the frequency converting unit 122. In the second embodiment, the BF output by the microphone array MA1 _{Y 1,} the BF output by the microphone array MA2 _{Y 2,} and the BF output _{Y 3} by the microphone array MA3. The directivity of the BF outputs Y ₁ , Y ₂ , and Y ₃ is as shown in FIG. As in the second embodiment shown in FIG. 10, the microphone array MA1~MA3 is disposed at the position of each vertex of a triangle, BF Output _{Y 1,} Y 2, _{Y 3} directional (microphone array MA1~MA3 Are directed inside the triangle.

The target area sound extraction unit 124A performs an area sound collection process using the BF output generated by the directivity forming unit 123. The area sound pickup is for directing the directivity of the BF from different directions and separating and extracting the components (area sounds) of the areas where the directivities intersect. Area sound pickup can be realized from each of the combination of the BF outputs Y ₁ and Y _{2 and} the combination of the BF outputs Y ₁ and Y ₃ .

  FIG. 11 is an explanatory diagram showing the distribution of sound collection areas where the target area sound extraction unit 124A performs area sound collection.

As shown in FIG. 6 described above, the area sound pickup has a characteristic that the sound pickup area spreads in a direction far from the microphone array. Therefore, an area sound pickup area (in the second embodiment, referred to as “area 1” or “sound pickup area 1”) by the microphone arrays MA1-MA2 and an area sound pickup area (the second sound pickup area) by the microphone arrays MA2-MA3. In the embodiment, “area 2” or “sound collection area 2” has an image as shown in FIG. In the second embodiment, the area sound pickup component (area sound pickup output) of the sound pickup area 1 is Z ₁ , and the area sound pickup component (area sound pickup output) of the area ₂ is Z ₂ .

  Each sound pickup area is divided into a part where the two sound pickup areas overlap as shown in FIG. 12 and an independent part where the two sound pickup areas do not overlap.

  In FIG. 12, the overlapping area in areas 1 and 2 is referred to as overlapping area 1 エ リ ア 2. In FIG. 12, in the area 1, an independent area excluding the overlapping area 1∧2 (an area that does not overlap with another sound collection area) is defined as an independent area A. Further, in FIG. 12, an independent area other than the overlapping area 1∧2 in the area 2 is defined as an independent area B. Note that an independent area (independent portion) derived from one sound collection area may be divided into a plurality of areas as shown in FIG. 12, but in this specification, an independent area generated from one sound collection area is divided. The areas are collectively indicated by one code. For example, in FIG. 12, the independent area A is divided (divided) into two areas by the overlapping area 1∧2, but here, these two areas are collectively referred to as an independent area A.

As described above, the area 1 is composed of the overlapping area 1∧2 and the independent area A (the area excluding the overlapping area 1∧2 from the area 1), and the area 2 is the overlapping area 1∧2 and the independent area B (the overlapping area Area excluding area 1∧2). An area sound-pickup output Z ₁ of the area 1, superimposing area sound-pickup output Z ₂ of area 2 (summing), can be collected sound from a wide range of areas, the component is a double overlap area 1∧2 Therefore, uniform sound pickup characteristics cannot be obtained for the entire sound pickup area. Therefore, if the sound sources of the overlapping area 1 音源 2 and the independent areas A and B can be separated and extracted individually, the respective areas are integrated without overlapping so that the entire area of the areas 1 and 2 is uniform. Sound pickup characteristics can be obtained.

  The partial area component calculation unit 126 calculates a sound pickup component of the overlap area 1∧2 and an independent area from the two area sound pickup components (here, the area sound pickup components of the areas 1 and 2) having the overlap area 1∧2. (Here, the sound pickup components of the independent areas A and B) are separated.

  FIG. 13 is an explanatory diagram showing a composition image (power for each component) of each area shown in FIG. 12 in the form of a bar graph.

13 (a) shows the composition image of the area voice collecting component _{Z 1} in area 1, FIG. 13 (b) shows a composition image of the area sound-pickup component _{Z 2} of area 2. Further, FIG. 13 (c), the composition image of the area sound-pickup component Z ₁ shown in FIG. 13 (a), the components of the overlapping area 1∧2 illustrates hatched (shaded pattern). Further, FIG. 13 (d) the composition image area sound-pickup output Z ₂ shown in FIG. 13 (b), the components of the overlapping area 1∧2 illustrates hatched (shaded pattern).

Duplicate area of Area 1 and Area 2 1∧2, since a common overlapping literally included as identical components each in Z ₁ and Z _2. Therefore, the target area sound extraction unit 124A separates each component by using the spectrum subtraction method (SS) based on the same principle as the area sound pickup.

Partial area component calculation unit 126 SS the area sound-pickup output _{Z 2} from the area sound-pickup output _{Z 1.} The partial area component calculation unit 126 clips a component that becomes negative at the time of SS to 0. In doing so, the destination area sound extraction unit 124A, area sound-pickup component overlapping area 1∧2 from area sound-pickup output Z ₁ is removed, the area sound-pickup component independent area A (first embodiment " V _A "). Similarly, partial area component calculation unit 126, an area sound-pickup output Z ₁ from the area sound-pickup output Z ₂ By SS, the area sound-pickup component (first embodiment of the independent area B, "V _B" ) Can be separated.

Figure 14 is an explanatory view showing a procedure of a process for calculating the partial area component calculation unit 126 independent area area (independent areas A, B) of the sound pickup component _{(V A,} V _B).

Figure 14 (a) in whole to FIG 14 (c), partial area component calculation unit 126, the area 1 from area sound-pickup output _{Z 1,} independent area A the area sound-pickup output _{Z 2} of area 2 and SS The process of extracting the area sound pickup component _VA (processing corresponding to the following equation (21)) is shown. Each of the graphs in FIGS. Z _1, represents the composition image of the area sound-pickup component V _a of the area sound-pickup component area 2 Z _2, and independent area a.

Similarly, in the entirety of FIGS. 14D to 14F, the partial area component calculation unit 126 sets the area sound component Z1 of the area ₁ to the SS from the area sound component Z2 of the area ₂ to be independent. shows the process of extracting the area sound-pickup component _{V B} of the area B (process equivalent to the following equation (22)), the individual graphs, respectively Area2 in FIG 14 (d) ~ FIG 14 (f) sound pickup component Z _2, which represents a composition image of the area sound-pickup component V _B of the area voice collecting component Z _1, and independent area B of area 1.

In each composition image shown in FIG. 14, the area sound-pickup component overlapping area 1∧2, the area sound-pickup component V _A separate area A, in the area sound-pickup component V _B of the independent area B, different from each other It is illustrated with a pattern.

The partial area component calculation unit 126, based on the area sound-pickup component V _B of the area voice collecting component V _A or an independent area B of the independent area A, area sound-pickup component overlapping area 1∧2 (hereinafter, "V _{1∧ 2} "). For example, partial area component calculation unit 126 SS the area sound-pickup component _{V A} separate area A from area sound-pickup output _{Z 1.} The partial area component calculation unit 126 clips a component that becomes negative at the time of SS to 0. In doing so, the destination area sound extraction unit 124A, area sound-pickup component V _A separate area A from area sound-pickup output Z ₁ is removed, the area voice collecting component V _1∧2 overlapping area 1∧2 separation Is done. Similarly, the partial area component calculation unit 126 separates the area sound component V 1 ₂ of the overlap area 1∧2 by SS the area sound component V _B of the independent area B from the area sound output _Z2 . can do.

FIG. 15 is an explanatory diagram showing a procedure of a process in which the partial area component calculation unit 126 calculates the area sound _pickup component V 1 _# 2 of the overlap area 1 # 2.

FIG. 15 (a), the FIG. 15 (b), the FIG. 15 (e), the partial area component calculation unit 126, overlap area the area sound-pickup component _{V A} separate area A from area sound-pickup output _{Z 1} and SS The processing for extracting the 1 _{音 2} area sound _pickup component V 1 ₂ (processing corresponding to the following equation (23)) is shown. FIG. 15 (c), the FIG. 15 (d), the FIG. 15 (e), the partial area component calculation unit 126, overlap area the area sound-pickup component _{V B} of the independent area B from the area sound-pickup output _{Z 2} and SS The processing for extracting the 1 _{音 2} area sound _pickup component V _1∧2 (processing corresponding to the following equation (24)) is shown. As described above, in the partial area component calculation unit 126, the area sound component V _{A of} the independent area _A , the area sound component V _B of the independent area _B , and the area sound component V _1∧2 of the overlap area _1∧2. Can be separated and extracted.

Partial area selection unit 127, in addition to the area sound-pickup component _Z 1, _{Z 2,} area sound-pickup component _V 1-2 of the partial area _divided, V A, choose one of the _{V B} output ( (Output as final sound collection result W). The selection processing method by the partial area selection unit 127 is not limited.

  As described above, the sound pickup unit 120A outputs the area sound pickup component selected by the partial area selection unit 127 as the final output W (n).

  Then, the communication unit 130 transmits the audio data based on the final output W (n) to the communication device 200 via the communication path P.

  Then, the communication unit 230 of the communication device 200 supplies the output unit 140 with the audio data (audio data based on W (n)) received from the communication device 100A. The output unit 140 supplies an audio signal based on the received audio data to the speaker 210 to output a sound (a sound output to the second user U2).

(B-3) Effects of the Second Embodiment According to the second embodiment, the following effects can be obtained as compared with the first embodiment.

  In the sound pickup unit 120A of the second embodiment, for two area sound pickup outputs having overlapping areas, the overlapping of the areas is utilized to separate and extract each area component of an independent area that does not overlap with the overlapping area. Thereby, the whole area is divided into a plurality of small areas. Then, the sound pickup unit 120A of the second embodiment selects the most suitable area as the target sound pickup area from the divided small areas, thereby covering the entire area of a plurality of areas. Then, it is possible to extract the emphasized sound from the pinpoint area including the target sound.

(C) Third Embodiment Hereinafter, a second embodiment of a sound collection device, a program, and a method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which a sound collection device, a program, and a method of the present invention are applied to a sound collection unit will be described.

(C-1) Configuration of Third Embodiment FIG. 16 is a block diagram showing the configuration of each device related to the third embodiment.

  In FIG. 16, the same parts or corresponding parts as those of Y1 described above are denoted by the same reference numerals or corresponding reference numerals. Hereinafter, the third embodiment will be described focusing on differences from the second embodiment.

  The third embodiment is different from the second embodiment in that a communication device 100A is replaced by a communication device 100B. Further, the communication device 100B of the third embodiment differs from the second embodiment in that the microphone array unit 111A is replaced by the microphone array unit 111B. Further, the communication device 100B of the third embodiment is different from the second embodiment in that the sound collection unit 120A is replaced with the sound collection unit 120B.

  Next, an internal configuration of the sound pickup unit 120B according to the third embodiment will be described.

  In the sound pickup unit 120B of the third embodiment, the target area sound extraction unit 124A, the partial area component calculation unit 126, and the partial area selection unit 127 are respectively connected to the target area sound extraction unit 124B, the partial area component calculation unit 126B, It differs from the second embodiment in that it is replaced with an area selector 127B.

  Next, the configuration of the microphone array unit 111B will be described with reference to FIG.

  In the example of this embodiment, as shown in FIG. 17, the communication device 100B has a hardware configuration of a smartphone (a smartphone owned by the speaker U1). In the example of the third embodiment, the microphone array unit 1B has a configuration including three microphones MC1 to MC3.

  As shown in FIG. 17, in the example of this embodiment, since the communication device 100 has a configuration of a smart phone, the three microphones MC1 to MC3 are used as portions (speakers SP) that normally function as mouthpieces in the smart phone. Is desirably disposed around the end opposite to the portion where the is disposed. In other words, in the communication device 100, the three microphones MC1 to MC3 may be arranged around a portion facing the mouth of the speaker U1 (a portion closest to the mouth of the speaker U1) when the communication device 100 is used. desirable. In FIG. 17, when the speaker U1 grips the communication device 100 with his hand and presses the speaker SP against his ear, the portion where the mouth of the speaker U1 is located (the lower portion as viewed from the direction of FIG. 17) is shown. Three microphones MC <b> 1 to MC <b> 3 are arranged around (around the part closest to the mouth of the speaker U <b> 1).

  In the communication device 100 (microphone array unit 1) illustrated in FIG. 17, the three microphones MC1 to MC3 are arranged so that each position (the center position of each microphone) becomes the vertex of an equilateral triangle. In this embodiment, three microphone arrays MA1 to MA3 are configured by a combination of three microphones MC1 to MC3. In the following, as shown in FIG. 17, a microphone array pairing microphones MC1 and MC2 is called MA1, a microphone array pairing microphones MC2 and MC3 is called MA2, and a microphone array pairing microphones MC3 and MC1 is called MA3. Shall be.

  In this embodiment, the microphones MC1 to MC3 are arranged in an equilateral triangle in order to enlarge the area in the same direction, but are not necessarily limited to the equilateral triangle. That is, the distances between the sides of the triangle and the angles of the corners of the microphones MC1 to MC3 need not be all the same.

  As described above, in the third embodiment, as shown in FIG. 18, three microphone arrays (MA1 to MA3) are configured from three microphones (MC1 to MC3), and three microphone arrays (MA1 to MA3) are formed by combining the microphone arrays. The area is picked up.

(C-2) Operation of Third Embodiment Next, an operation (a sound collection method according to the embodiment) of the third embodiment having the above-described configuration will be described.

The signal input unit 121 converts the acoustic signals collected by the three microphones MC <b> _{1 to} MC <b> ₃ from analog signals to digital signals x <b> _{1 to} x <b> ₃ and supplies the digital signals to the frequency conversion unit 122.

The frequency conversion unit 122, for example, to convert the microphone signals from the time domain to the signal X ₁ to X ₃ in the frequency domain using a fast Fourier transform.

The directivity forming unit 123 forms directivity by BF using the input signal of each microphone that has been time-frequency converted by the frequency converting unit 122. In the third embodiment, the BF output by the microphone array MA1 _{Y 1,} the BF output by the microphone array MA2 _{Y 2,} and the BF output _{Y 3} by the microphone array MA3.

The target area sound extraction unit 124 uses the BF outputs Y ₁ , Y ₂ , and Y ₃ formed by the directivity forming unit 123 and uses a combination of Y ₁ -Y ₂ , Y ₂ -Y ₃ , and Y ₃ -Y ₁ . , Respectively.

In the third _embodiment, Y 1 "1" area (sound-pickup area) by a combination of -Y _2, combining area by a combination of Y 2 -Y ₃ (the sound-pickup area) of _2, Y 3 -Y ₁ Area (sound collection area) is referred to as “3”.

As shown in FIG. 6 described above, the area sound pickup has a characteristic that the sound pickup area spreads in a direction far from the microphone array. Therefore, BF outputs _Y 1 -Y ₂ area by area according to (microphone array MA1-MA2) 1, BF output _Y 2 -Y ₃ (microphone array MA2-MA3) 2, BF output _Y 3 -Y ₁ (microphone array MA3- The distribution of the sound collection area according to MA1) has an image as shown in FIG. In the third embodiment, the area sound pickup components (area sound pickup outputs) of the areas ₁ , ₂ , and ₃ are Z ₁ , Z ₂ , and Z ₃ .

The partial area component calculation unit 126B uses the area sound pickup outputs Z ₁ , Z ₂ , and Z ₃ to calculate a portion where the three areas overlap, a portion where the two areas overlap, and an independent portion without overlap. I do. In the second embodiment, the overlap of two sound pickup areas is considered. However, in the third embodiment, there are three sound pickup areas. Therefore, the overlap pattern is more complicated than in the second embodiment. Become. The calculation of each part of the overlapping area of the three sound pickup areas is performed by decomposing the area sound pickup components into a combination of two known area sound pickup components, thereby using the same method (calculation method) as in the second embodiment. It becomes possible.

  Specifically, when calculating the area sound pickup component of each part, the partial area component calculation unit 126B calculates each combination of the two sound pickup areas (the combination of the areas 1 and 2, the combination of the areas 2 and 3, the area The same method as in the second embodiment can be used by decomposing the pattern into a combination (3, 1) (hereinafter, referred to as a “combination pattern”). That is, in the partial area component calculation unit 126B, the processing of separating the area sound pickup components of the two sound pickup areas having the overlap area into the overlap area part and the independent area part is the same as in the second embodiment. .

  Hereinafter, the combination pattern of the areas 1 and 2 is referred to as a “first combination pattern”, the combination pattern of the areas 2 and 3 is referred to as a “second combination pattern”, and the combination pattern of the areas 3 and 1 is referred to as a “third combination pattern”. Combination pattern ".

  FIG. 19 is an explanatory diagram (image diagram) showing an exploded image of a combination pattern (first to third combination patterns) of two sound collection areas for three areas 1 to 3.

  FIG. 19A is a diagram in which three areas 1 to 3 are overlapped. FIGS. 19B to 19D are explanatory diagrams showing images decomposed into first to third combination patterns, respectively.

  First, from the three combination patterns shown in FIGS. 19B to 19D, the first combination pattern (combination pattern of areas 1 and 2) shown in FIG. 19B will be described as a representative example.

Partial area component calculation unit 126B, by the SS an area sound-pickup output Z ₂ from the area sound-pickup output Z _1, the independent parts (the embodiment for areas 2 in area 1, referred to as "area A" It is assumed that an area sound pickup component (referred to as “ _VA ” in the third embodiment) of FIG. 19B is obtained. Further, the partial area component calculation unit 126B, by SS the area sound-pickup output Z ₁ from the area sound-pickup output Z _2, the independent parts (the embodiment for areas 1 area 2, "area B" 19 (b)) (refer to FIG. 19 (b)) (referred to as “V _B ” in the third embodiment). The partial area component calculation unit 126B, as in the second embodiment, the above equation (21), can be obtained by equation (22) equation, the area voice collecting component _V A, _{V B.}

In the partial area component calculation unit 126B, the second combination pattern (the combination pattern of the areas 2 and 3) is also a part independent of the area 3 of the area 2 (referred to as “area C” in this embodiment). and ones; and FIG. 19 (c) refer) area sound-pickup components (in the third embodiment is referred to as "V _C"), independent part with respect to the area 2 of the area 3 (in the third embodiment , "Area D"; see FIG. 19 (c)) to obtain an area sound pickup component (called "V _D " in the third embodiment). Similarly, in the partial area component calculation unit 126B, the third combination pattern (the combination pattern of the areas 3 and 1) is also independent of the area 3 of the area 1 (in this embodiment, “area E”). An area sound pickup component (referred to as “ _VE ” in the third embodiment) of FIG. 19D) and a portion independent of the area 3 of the area 1 (in this embodiment, , is referred to as "area F"; the area sound-pickup component (the third embodiment of FIG. 19 (d) refer) can be obtained is called) and "V _F".

The partial area component calculation unit 126B, may be obtained by the following equation (31) - (34) equation, the area voice collecting component _{V C, V D, V E} , the _{V F.}

When the area sound pickup component VA of the area _A or the area sound pickup component VB of the area _B is known, the partial area component calculation unit 126B determines a portion where the areas 1 and 2 overlap (in this embodiment, "overlap area 1"). 19B (refer to FIG. 19 (b)) (in this embodiment, referred to as “V 1 _{で は 2} ”). Specifically, partial area component calculation unit 126B, as shown in the following expression (35), by SS the area sound-pickup component _{V A} of the area A from the area sound-pickup output _{Z 1,} Area 1∧2 Area sound pickup components can be obtained. The partial area component calculation unit 126B, as shown in the following equation (36), also by SS the area sound-pickup component _{V B} of the area B from the area sound-pickup output _{Z 2,} the area 1∧2 area yield Sound components can be obtained.

Similarly, the partial area component calculator 126B calculates the area sound component VC of the area _C or the area sound component VD of the area _D based on the following formulas (37) and (38). An area sound _pickup component (referred to as “V 2 _∧3 ” in this embodiment) of an area where areas 2 and 3 overlap (referred to as “overlap area 2 ∧ 3” in this embodiment; see FIG. 19C). Obtainable. Further, the partial area component calculation unit 126B, the following equation (39), (40) as shown in the expression on the basis of the area sound-pickup component _{V F} of the area sound-pickup component _{V E} or area F of area E, the area An area sound _pickup component (referred to as “V 3 _∧1 ” in this embodiment) of an area where 3 and 1 overlap (referred to as “overlap area 3 ∧ 1” in this embodiment; see FIG. _19D ). be able to.

In the partial area component calculation unit 126B, the area sound pickup components (V _A , V _B , V _C , V _D , V _{E, V F)} and overlapping parts _{(V 1∧2, V 2∧3,} when all _{V 3∧1)} is calculated, based on their area sound-pickup components, when stacked 3 areas simultaneously The area sound pickup component of each partial area can be calculated.

FIG. 20 to FIG. 23 are explanatory diagrams showing the respective partial areas when three areas of area 1, area 2, and area 3 are overlapped at the same time. For example, in the above calculation, the area sound pickup component _VA of the area _A and the area sound pickup component VD of the area _D are known, so the partial area component calculation unit 126B calculates the following equation (41). as shown, the area sound-pickup component V _D of the area D from the area voice collecting component V _a of the area a, by SS by the same calculation as far independent portion of the area a (hereinafter, the "area Ad"Call; see FIG. 20) (hereinafter, referred to as “V _Ad ”). Further, the partial area component calculation unit 126B, as shown in the following equation (42), by the area sound-pickup component V _D of the area A from the area sound-pickup component V _D of the area D, the same calculation as far SS By doing so, an area sound pickup component (hereinafter, referred to as “V _Da ”) of an independent portion of the area D (hereinafter, referred to as “area Da”; see FIG. 20) can be obtained.

Then, when the area sound pickup component VAd of the area _Ad or the area sound pickup component VDa of the area _Da is obtained, the partial area component calculation unit 126 </ b> B overlaps the area A and the area D (hereinafter, “area A∧D ; See FIG. 20) (hereinafter, referred to as “ _VA∧D ”).

More specifically, the partial area component calculation unit 126B calculates the area sound pickup component V _Ad of the area Ad from the area sound pickup output V _D of the area D as shown in the following equation (43). By performing the SS by the calculation method, it is possible to calculate the area sound _pickup component VA∧D of the area A∧D. In addition, the partial area component calculation unit 126B also performs the area sound component V _Da of the area Da from the area sound output V _A of the area A, as shown in the following equation (44), so that the area A∧D it can be calculated in the area sound-pickup component V _A∧D.

Similarly, in the partial area component calculation unit 126B, as shown in the following equations (45) and (47), the area sound pickup of an independent part of the area B (hereinafter, referred to as “area Be”; see FIG. 21). component (hereinafter, referred to as _{"V be")} and, independent portions of the area E (hereinafter, referred to as "area Eb"; see FIG. 21) area sound-pickup components (hereinafter, referred to as _{"V Eb")} to obtain Can be. Then, the partial area component calculation unit 126B, the following expression (46), (48) as shown in the expression on the basis of the area sound-pickup component _{V Eb} of area sound-pickup component _{V Be} or area Eb area Be, Area An area sound _pickup component (hereinafter, referred to as “V _BＶE ”) of an overlapping portion of B and area E (hereinafter, referred to as “area B∧E”; see FIG. 21) can be calculated.

In addition, the partial area component calculation unit 126B calculates the area sound pickup component of an independent part of the area C (hereinafter, referred to as “area Cf”; see FIG. 22) as shown in the following equations (49) and (51). (Hereinafter, referred to as “V _Cf ”) and an area sound pickup component (hereinafter, referred to as “V _Fc ”) of an independent portion of the area F (hereinafter, referred to as “area Fc”; see FIG. 22). it can. Then, the partial area component calculation unit 126B calculates the area based on the area sound component V _{Cf of the} area _Cf or the area sound component V _Fc of the area Fc as shown in the following equations (50) and (52). An area sound _pickup component (hereinafter, referred to as “ _VC∧F ”) of an overlapping portion of C and the area F (hereinafter, referred to as “area C∧F”; see FIG. 22) can be calculated.

Then, when the area sound _pickup component V1 _{# 2} and the area sound _pickup component _{VC # F} are known, the partial area component calculation unit 126B determines the area (the area 1, the area 2, and the area 3) of the common part of the three areas. Hereinafter, an area sound _pickup component (hereinafter, referred to as “ _V1∧2∧3 ”) of “area 1∧2∧3; see FIG. 23” can be acquired. More specifically, the partial area component calculation unit 126B calculates the area sound _pickup component V _{C ∧F} from the area sound _pickup component V _1∧2 by the same calculation method as described above, as shown in the following equation (53). By performing SS, it is possible to obtain an area sound _pickup component V1∧2∧3 of the area 1∧2∧3.

Similarly, when the area _pickup component _V2∧3 and the area _pickup component _VB∧E are known, the partial area component calculation unit 126B similarly calculates the area 1∧2∧3 by the following equation (54). V _1∧2∧3 can be obtained. Furthermore, the partial area component calculation unit 126B, the area sound-pickup component _{V 3∧1} and area sound-pickup component _{V A∧D} is known, by the following equation (55), _V the area 1∧2∧3 _{1∧ 2∧3} can be obtained.

By the above processing, the partial area component calculation unit 126B _obtains the area sound components (V _1∧2∧3 ) of the portion where the three areas overlap with each other, based on the area sound collection of the three areas including the overlap, and Area sound _pickup components ( _VA∧D , _VB∧E , _VC∧F ) of a part where two areas overlap, and area sound _{pickup of} an independent part without overlapping with other areas in the three areas component _{(V Ad, V Da, V} be, V Eb, V Cf, V Fc) can be separated and extracted.

  The partial area selection unit 127B collects the area sound components of the area estimated to contain the most target sound components from among the area sound components of the area decomposed into the respective parts as described above. Select (acquire) as result W. Hereinafter, each area divided from all areas (all areas covered by areas 1, 2, and 3) in the partial area selection unit 127B is referred to as a "partial area". For example, in the partial area selection unit 127B, as partial areas, areas 1, 2, 3, A, B, C, D, E, F, 1∧2, 2∧3, 3∧1, 1∧2∧3, Some or all of Ad, Da, A∧D, Be, Eb, B∧E, Cf, Fc, and C∧F may be set.

  The method in which the partial area selection unit 127B selects one of the respective partial areas (area sound pickup components) is not limited. Hereinafter, the partial area selected by the partial area selection unit 127B is also referred to as a “selected area”.

  For example, the partial area selection unit 127B may select a partial area having the largest power (for example, an area having the largest average power spectrum obtained by averaging the frequency components constituting the area sound pickup component of the partial area). Good. At this time, the partial area selection unit 127B may select any one of the partial areas based on the evaluation value obtained by normalizing the power of each partial area with the area and evaluating the power. For example, for each partial area, the partial area selection unit 127B calculates an evaluation value such that an area having a smaller area has a higher evaluation even with the same power, and selects a partial area having the highest evaluation (evaluation value). It may be.

  When selecting a partial area, the partial area selection unit 127B does not have to select all the partial areas as selection targets. For example, the partial area selection unit 127B excludes an area whose area (area) is smaller than other partial areas from selection targets (for example, there is only one third or less the area as compared with the largest partial area). Partial areas may be excluded from selection targets), and selection may be made from remaining partial areas (partial areas not excluded).

  Further, the partial area selection section 127B may integrate the area including the area adjacent to the area selected by the above-described method (the area adjacent to the boundary) and select the integrated area as one partial area. For example, the partial area selection unit 127B may select an area having the highest power and an area adjacent to the area.

  Furthermore, the partial areas to be selected by the partial area selection unit 127B need not necessarily be independent of each other. For example, the partial area selection unit 127B may select a partial area having an overlapping area as the selection target, such as the area D and the area A shown in FIG. In other words, in the partial area selection unit 127B, the area to be selected is smaller than the entire area (the entire area covered by the areas 1, 2, and 3), and the partial area selection unit 127B does not include an area unnecessary for extracting the target sound. An area may be selected.

  Then, the partial area selection unit 127B acquires the area sound pickup component of the partial area selected by the above processing as the final sound pickup result W (n). Note that the above-described method of selecting the partial area (the area sound pickup component of the partial area) in the partial area selection unit 127B may be applied to the partial area selection unit 127A of the second embodiment.

  As described above, the sound pickup unit 120B outputs the area sound pickup component selected by the partial area selection unit 127B as the final output W (n).

  Then, the communication unit 130 transmits the audio data based on the final output W (n) to the communication device 200 via the communication path P.

  Then, the communication unit 230 of the communication device 200 supplies the output unit 140 with the audio data (audio data based on W (n)) received from the communication device 100B. The output unit 140 supplies an audio signal based on the received audio data to the speaker 210 to output a sound (a sound output to the second user U2).

(C-3) Effects of Third Embodiment According to the third embodiment, the following effects can be obtained.

  In the communication device 100B (sound pickup unit 120B) of the third embodiment, for the three sound pickup areas having overlaps, an independent area that does not overlap with any area, an area where two areas overlap, and 3 The area in which all three areas overlap is calculated, and the components of each area are calculated. Then, in the communication device 100B (sound pickup unit 120B) of the third embodiment, the most appropriate area as the target sound pickup area is selected from the divided areas, and the area sound pickup component of the selected area is finalized. Is output as a typical sound pickup result. Thus, in the communication device 100B (sound pickup unit 120B) of the third embodiment, the selected area is an area that is much smaller than the entire area while covering the entire area based on the plurality of area sound pickup results. The emphasized voice can be extracted from the pinpoint area including the target sound without including unnecessary area components.

(D) Other Embodiments The present invention is not limited to the above embodiments, but may include modified embodiments as exemplified below.

  (D-1) In each of the above embodiments, the sound collection unit has been described as constituting a part of the communication device, but may be configured as an independent device. Further, in each of the above-described embodiments, the sound collecting unit is not configured to include the microphone array unit. However, the sound collecting unit and the microphone array unit may be configured as an integrated device.

  (D-2) In each of the above embodiments, an example is described in which the sound pickup device (sound pickup unit) of the present invention is applied to a device such as a handset or the like having a handheld transmitter (transmitter / receiver). The sound collection device of the present invention is applied to a headset or a wearable device (for example, a head-mounted display with a microphone, a neckband headphone with a microphone, or the like), and is attached to the first user U1 when worn by the first user U1. The area where the mouth is located is set as the target area, and microphones are installed at the respective vertices of the polygon (N-gon) around the mouth (mouthpiece), and the area sound pickup processing may be performed in the same manner as in the above embodiment. .

  (D-3) In the first and third embodiments, an example of area sound collection using three microphones MC1 to MC3 has been described. However, the number of microphones (microphones are arranged in the microphone array unit 111) is described. The number of sides (corners) of the polygon is not limited. For example, area sound pickup may be performed from three or four directions. For example, in the first and third embodiments, four microphones may be arranged at the corner vertices of a rectangle.

  FIG. 24 is an explanatory diagram showing a configuration when the number of microphones in the microphone array unit 111 is four.

  In FIG. 24, four microphones MC1 to MC4 are arranged at the corner vertices of a square (square). The four microphones MC1 to MC4 are combined with adjacent microphones to form a microphone array MA701 formed by a pair of microphones MC1 and MC2, a microphone array MA702 formed by a pair of microphones MC2 and MC3, and a microphone MC3. A microphone array MA703 formed by a pair of MC4 and a microphone array MA704 formed by a pair of microphones MC4 and MC1 are formed. Further, these microarrays can collect sound in four areas by combination with adjacent microphone arrays (combination of microphone arrays sharing some microphones). For example, when a configuration of four microphones MC1 to MC4 is applied to the microphone array unit 111, the sound collection unit 120B collects an area by combining the microphone arrays MA701 and MA702 and an area by the combination of the microphone arrays MA702 and MA703. It is possible to acquire each output (output of four area sound collections) of the sound, the area sound collection by the combination of the microphone arrays MA703 and MA704, and the area sound collection by the combination of the microphone arrays MA704 and MA701. The sound pickup unit 120B can acquire a sound pickup result based on the output of the above-described four area sound pickups.

  (D-4) In the sound collection unit 120B (partial area selection unit 127B) of the third embodiment, one area including the target sound most is selected from a plurality of areas, but a plurality of areas are selected. You may make it. In this case, the sound pickup unit 120B (partial area selection unit 127B) of the third embodiment integrates (adds) area sound pickup components of a plurality of selected areas and acquires the sound pickup result W. Good. However, in this case, in the sound pickup unit 120B (partial area selection unit 127B) of the third embodiment, the selected areas are selected so as not to have a common part (overlapping part) with each other. It is necessary to consider the area in advance.

  100, 100A, 100B: communication device, 110: handset, 111: microphone array, MC1 to MC6: microphone, 112: speaker, 113: mouthpiece, 114: earpiece, 115: handle, 120, 120A, 120B ... Sound collection unit, 121 ... Signal input unit, 122 ... Frequency conversion unit, 123 ... Directivity forming unit, 124, 124A, 124B ... Target area sound extraction unit, 125 ... Area sound addition unit, 126, 126B ... Partial area component Calculation unit, 127, 127B: partial area selection unit, 130: communication unit, 140: output unit, 200: communication device, 210: speaker, 220: microphone, 230: communication unit, 240: output unit, 250: sound collection unit U1, first user, U1a, listener's hand, U2, second user, P, communication channel.

Claims (6)

Area sound pickup for acquiring area sound pickup components of a plurality of sound pickup areas based on a combination of two or more patterns of the microphone arrays based on input signals from a microphone array unit capable of forming a plurality of microphone arrays having different directivities. Means,
Each of a partial area in which two or more of the sound collecting areas are divided from an entire area covering all of the sound collecting areas acquired by the area sound collecting means, and a partial area in which the sound collecting areas do not overlap with each other For a sound pickup component, a partial area component calculation means for acquiring based on an area sound pickup component of the sound pickup area of each pattern acquired by the area sound pickup means,
A partial area in which one or more partial area sound pickup components are selected from the area sound pickup components of the partial area calculated by the partial area component calculation means, and a sound pickup result based on the selected area sound pickup component is obtained. A sound pickup device comprising: a selection unit.   2. The partial area selection unit according to claim 1, wherein the partial area selection unit selects an area sound collection component of one or more partial areas based on a result of comparing the power of the area sound collection components of the respective partial areas. 3. Sound pickup device.   The sound pickup apparatus according to claim 2, wherein the partial area selection means selects an area sound pickup component having the highest power from the area sound pickup components of the partial area calculated by the partial area component calculation means. .   The said partial area selection means acquires the result which added the selected some area | region sound-pickup component as a sound-pickup result, when the area sound-pickup component of a some partial area is selected. A sound pickup device according to claim 1. Computer
Area sound pickup for acquiring area sound pickup components of a plurality of sound pickup areas based on a combination of two or more patterns of the microphone arrays based on input signals from a microphone array unit capable of forming a plurality of microphone arrays having different directivities. Means,
Each of a partial area in which two or more of the sound collecting areas are divided from an entire area covering all of the sound collecting areas acquired by the area sound collecting means, and a partial area in which the sound collecting areas do not overlap with each other For a sound pickup component, a partial area component calculation means for acquiring based on an area sound pickup component of the sound pickup area of each pattern acquired by the area sound pickup means,
A partial area in which one or more partial area sound pickup components are selected from the area sound pickup components of the partial area calculated by the partial area component calculation means, and a sound pickup result based on the selected area sound pickup component is obtained. A sound collection program characterized by functioning as selection means. In the sound pickup method performed by the sound pickup device,
Area pickup means, partial area component calculation means, and partial area selection means,
The area sound pickup unit is configured to pick up an area sound of a plurality of sound pickup areas based on a combination of two or more patterns of the microphone arrays based on input signals from a microphone array unit capable of forming a plurality of microphone arrays having different directivities. Get the ingredients,
The partial area component calculating means includes: a partial area in which two or more sound collecting areas divided from an entire area covering all of the sound collecting areas acquired by the area sound collecting means overlap each other; For each area sound pickup component of the non-overlapping partial area, acquiring based on the area sound pickup component of the sound pickup area of each pattern obtained by the area sound pickup means,
The partial area selecting means selects an area sound collecting component of one or more partial areas from the area sound collecting components of the partial area calculated by the partial area component calculating means, and collects sound based on the selected area sound collecting component. A sound collection method characterized by acquiring sound results.

Images