JP7176316B2 - SOUND COLLECTION DEVICE, PROGRAM AND METHOD - Google Patents

Description

The present invention relates to a sound collecting device, program and method, and can be applied, for example, to a speech communication system used in a noisy environment.

When using a speech communication system or a speech recognition application system in a noisy environment, ambient noise mixed in with the required target speech is a troublesome existence that hinders good communication and lowers the speech recognition rate. Conventionally, in an environment where multiple sound sources exist, a beamformer using a microphone array has been used as a technology that separates and collects only the sound from a specific direction to avoid mixing in unwanted sounds and obtain the necessary target sound. (Beam Former; hereinafter also referred to as “BF”; see Patent Documents 2 and 3). BF is a technique of forming directivity using the time difference between signals reaching each microphone. However, with only BF, if there are other sound sources around the area for which sound is to be collected (hereinafter referred to as "target area"), the sound existing in the target area (hereinafter referred to as "target area sound") It is difficult to capture only For this reason, conventionally, Japanese Patent Laid-Open No. 2002-200001 and the like propose an area sound pickup method that picks up sound in a target area using a plurality of microphone arrays.

FIG. 25 is an explanatory diagram (graph) showing processing for picking up target area sound from a sound source in the target area using two microphone arrays MA100 and MA200.

FIG. 25(a) is an explanatory diagram showing a configuration example of each of the microphone arrays MA100 and MA200. FIGS. 25(b) and 25(c) are diagrams (image diagrams in graph form) showing in the frequency domain the BF outputs of the microphone arrays MA100 and MA200 shown in FIG. 25(a), respectively. In FIG. 25, each of the microphone arrays MA100 and MA200 is composed of two microphones ch1 and ch2.

In the conventional area sound pickup, as shown in FIG. 25(a), the directivity of the microphone arrays MA100 and MA200 is crossed in areas (target areas) where sound is to be picked up from different directions. In the state of FIG. 25(a), the directivity of each of the microphone arrays MA100 and MA200 includes not only the sound existing in the target area (target area sound) but also the noise in the direction of the target area (non-target area sound). there is However, as shown in FIGS. 25(b) and 25(c), when the directivity of the microphone arrays MA100 and MA200 is compared in the frequency domain, the target area sound component is included in both outputs, but the non-target area The sound component will be different for each microphone array. In the conventional area sound pickup technique, using such characteristics, it is possible to extract only the target area sound by suppressing components other than those commonly included in the BF outputs of the two microphone arrays MA100 and MA200.

JP 2014-072708 A JP 2005-195955 A JP 2016-127457 A

By the way, emergency vehicles are equipped with handsets (transceivers) for communication as a means of emergency communication from a fire site where a siren sounds or an emergency site to a command center (fire department). Conventional handsets installed in emergency vehicles are used in a noisy environment, so communication from the site is drowned out by the surrounding noise, resulting in a lack of communication from the headquarters (for example, the headquarters that directs the crew of the emergency vehicle). Accurate information cannot be conveyed, resulting in erroneous information. For this reason, the use of various noise reduction technologies for handsets has been investigated, but many problems have been encountered in the introduction of such technologies, such as securing call quality and increasing costs. In such a usage environment, the area sound pickup technology described above is expected to be an effective solution. For example, two microphone arrays are installed around the mouthpiece of a handset, and the directivity of each of the two microphone arrays is crossed in front of the mouthpiece to function as area sound pickup, thereby enabling a loud sound such as a siren. It is possible to eliminate noise and to accurately transmit only the voice of the caller such as a firefighter to the headquarters or the like.

At least two microphone arrays are required to achieve area sound pickup. On the other hand, the size of the mouthpiece of a handset is as small as 6 cm in diameter, and when two microphone arrays are attached to achieve area sound pickup, the two microphone arrays are placed very close to each other. There is a need to. As a result, in area sound pickup using the handset, the sound pickup area is limited to a very narrow area in the immediate vicinity of the transmitter. However, when conventional area sound pickup processing is applied to a handset, the way of holding the handset and the size of the face differ depending on the user (speaker), and the mouth area is narrowly limited as described above (set for the handset). sound pickup area). In this case, if the mouth of the user (speaker) deviates from the sound pickup area of the handset, there is a problem that the sound picked up is distorted or dropped, and the sound cannot be picked up stably.

Therefore, there is a demand for a sound collecting device, program, and method capable of efficiently and stably performing area sound collection.

A sound collecting device according to a first aspect of the present invention provides: (1) a plurality of microphone arrays based on a combination of two or more patterns based on an input signal from a microphone array section capable of forming a plurality of microphone arrays with different directivities; and (2) two or more of the sound collection areas divided from the whole area covering all of the sound collection areas acquired by the area sound collection means. Area sound pickup components of overlapping partial areas and non-overlapping partial areas between the sound pickup areas are acquired based on the area sound pickup components of the sound pickup areas of each pattern acquired by the area sound pickup means. (3) selecting area picked-up sound components of one or a plurality of partial areas from the area picked-up sound components of the partial areas calculated by the partial area component calculating means, and calculating the selected area picked-up sound components; (4) the microphone array unit includes N microphone array units arranged at the positions of the vertices of an N-sided polygon (N is an integer equal to or greater than 3); , or N sets of microphone arrays arranged so as to be at the positions of the vertices of the N-sided polygon .

A program for collecting sound according to a second aspect of the present invention provides a computer with: (1) a combination of two or more patterns of microphone arrays based on an input signal from a microphone array unit capable of forming a plurality of microphone arrays with different directivities; (2) two or more areas divided from the total area covering all of the sound pickup areas acquired by the area sound pickup means; Based on the area sound pickup components of the sound pickup areas of each pattern acquired by the area sound pickup means, for each of the area sound pickup components of the partial areas where the sound areas overlap and the partial areas where the sound areas do not overlap. (3) selecting area picked-up sound components of one or a plurality of partial areas from the area picked-up sound components of the partial areas calculated by the partial area component calculating means; (4) the microphone array section is arranged so as to be positioned at each vertex of an N-sided polygon (N is an integer equal to or greater than 3); It is characterized by having N microphones or N sets of microphone arrays arranged so as to correspond to the positions of the vertices of an N-sided polygon .

A third aspect of the present invention is a sound collection method performed by a sound collection device, comprising: (1) area sound collection means, partial area component calculation means, and partial area selection means; acquiring area sound components of a plurality of sound pickup areas based on a combination of two or more patterns of the microphone arrays, based on an input signal from a microphone array unit capable of forming microphone arrays with different directivities; The partial area component calculation means calculates a partial area in which two or more of the sound collection areas overlap, divided from the entire area covering all of the sound collection areas acquired by the area sound collection means, and a partial area in which the sound collection areas overlap each other. (4) the partial area selection means acquires area sound pickup components of each non-overlapping partial area based on the area sound pickup components of the sound pickup areas of each pattern acquired by the area sound pickup means; Selecting area picked-up sound components of one or more partial areas from the area picked-up sound components of the partial areas calculated by the partial area component calculation means, obtaining sound pickup results based on the selected area picked-up sound components , (5 ) The microphone array section includes N microphones arranged at the positions of the vertices of an N-sided polygon (N is an integer equal to or greater than 3), or arranged at the positions of the vertexes of the N-sided polygon. It is characterized by having N sets of microphone arrays .

According to the present invention, it is possible to provide a sound collection device that efficiently and stably performs area sound collection.

2 is a block diagram showing the configuration of each device (including the functional configuration of a sound pickup unit (sound pickup device) according to the embodiment) according to the first embodiment; FIG. 1 is a diagram (perspective view) showing a usage state of a handset according to the first embodiment; FIG. 2 is an enlarged view of the mouthpiece portion of the handset according to the first embodiment; FIG. FIG. 3 is an explanatory diagram (image diagram) showing a configuration example of a microphone array formed by three microphones; FIG. 4 is an explanatory diagram (image diagram) showing area sound pickup processing corresponding to each combination (combination pattern) of microphone arrays formed by three microphones; FIG. 10 is a diagram showing a distribution of area sound pickup sensitivities (distribution of calculated sensitivities) when the directivities of two microphone arrays are crossed; FIG. 11 is a block diagram showing a configuration related to a subtractive BF when the number of microphones is two; FIG. 4 is a diagram showing directivity characteristics formed by a subtractive BF using two microphones; 8 is a block diagram showing the configuration of each device related to the second embodiment; FIG. FIG. 10 is a diagram showing an arrangement of six microphones in a microphone array section and a configuration example of the microphone array according to the second embodiment; FIG. 11 is an explanatory diagram showing the distribution of sound pickup areas where the target area sound extraction unit according to the second embodiment performs area sound pickup. FIG. 11 is an explanatory diagram showing independent areas that do not overlap with a plurality of sound pickup areas and overlapping areas that overlap with a plurality of sound pickup areas in the sound pickup areas according to the second embodiment; FIG. 11 is an explanatory diagram showing, in the form of a bar graph, a composition image (power of each component) of each sound pickup area according to the second embodiment; FIG. 11 is an explanatory diagram showing a procedure of processing (part 1: processing for acquiring area sound pickup components of an independent area) by an independent area component calculation unit according to the second embodiment; FIG. 11 is an explanatory diagram showing a procedure of processing (part 2: processing for acquiring area sound pickup components of overlapping areas) by the independent area component calculation unit according to the second embodiment; FIG. 11 is a block diagram showing the configuration of each device related to the third embodiment; FIG. FIG. 11 is a diagram showing a plan view of a communication device (smartphone) according to a third embodiment; FIG. 11 is an explanatory diagram showing an image of three sound pickup areas according to the third embodiment; FIG. 11 is an explanatory diagram showing an exploded image of combination patterns (first to third combination patterns) in three sound pickup areas according to the third embodiment; FIG. 11 is an explanatory diagram showing an image of independent portions generated in areas A and D according to the third embodiment; FIG. 11 is an explanatory diagram showing an image of independent portions occurring in areas B and E according to the third embodiment; FIG. 11 is an explanatory diagram showing an image of an independent portion generated in areas C and F according to the third embodiment; FIG. 11 is an explanatory diagram showing an image of overlapping portions of three areas according to the third embodiment; FIG. 11 is an explanatory diagram showing a configuration (configuration of a modification according to the embodiment) when the number of microphones in the microphone array section according to the embodiment is four; FIG. 10 is an explanatory diagram showing a configuration example of a conventional sound collecting device in which directivity of beamformers (BF) of two microphone arrays is directed from different directions to a target area.

(A) First Embodiment Hereinafter, a first embodiment of the sound collecting device, program and method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound collecting device, program, and method of the present invention are applied to a sound collecting unit will be described.

First, the basic principle of area sound pickup processing using a microphone array in this embodiment will be described with reference to FIGS. 4 to 6. FIG.

By arranging a microphone at each vertex of the polygon, it is possible to construct multiple area sound pickups toward the center of the polygon.

For example, when considering an area sound pickup configuration using three microphones, as shown in Fig. 4, a maximum of three microphone arrays (three microphone arrays with different directivity directions) can be used depending on the combination of microphones. can be set. As shown in FIG. 4, three microphones ch1 to ch3 include a microphone array MA301 paired with microphones ch1 and ch2, a microphone array MA302 paired with microphones ch2 and ch3, and a microphone array MA302 paired with microphones ch3 and ch1. Array MA303 can be configured.

Furthermore, in the configuration of three microphones ch1 to ch3, as shown in FIG. 5, it is possible to pick up sound in an area corresponding to the combination (three combination patterns) of the three microphone arrays MA301, MA302, and MA303. .

In FIG. 5(a), the directivity of the microphone array MA301 is indicated by a one-dot chain line, and the directivity of the microphone array MA302 is indicated by a two-dot chain line. In FIG. 5(b), the directivity of the microphone array MA302 is indicated by a one-dot chain line, and the directivity of the microphone array MA303 is indicated by a two-dot chain line. Furthermore, in FIG. 5(c), the directivity of the microphone array MA301 is indicated by a one-dot chain line, and the directivity of the microphone array MA303 is indicated by a two-dot chain line. Furthermore, in FIG. 5A, the sound pickup area A301 corresponding to the combination (pattern) of the microphone arrays MA301 and MA302 is hatched (slanted lines). Also, in FIG. 5B, the sound pickup area A302 corresponding to the combination (pattern) of the microphone arrays MA302 and MA303 is hatched (slanted lines). Furthermore, in FIG. 5(c), the sound pickup area A303 corresponding to the combination (pattern) of the microphone arrays MA301 and MA303 is hatched (slanted lines).

As shown in FIG. 5, in the configuration of three microphones ch1 to ch3, any microphone array has an angle between the microphone arrays (line segments connecting the positions of two microphones constituting the microphone array). , by crossing each other's directivities, different area sound pickup (area sound pickup in different regions) can be realized for each combination.

On the other hand, the sound pickup area of area sound pickup using a microphone array has the property of expanding in front of the microphone array (farther from the microphone array). The properties will be described below with reference to FIG.

FIG. 6 is a diagram showing a sensitivity distribution (calculated sensitivity distribution) for area sound pickup when the directivities of the two microphone arrays MA400 and MA500 are crossed so as to form a right angle. In other words, FIG. 6 illustrates the area sound pick-up sensitivity in and around the area where the directivities of the two microphone arrays MA400 and MA500 intersect. In FIG. 6, the microphone arrays MA400 and MA500 each have two microphones ch1 and ch2. In addition, in FIG. 6, the sensitivity of area sound pickup is divided into five stages (0 to -5 dB, -5 to -10 dB, -10 to -15 dB, -15 to -20 dB, -20 to -25 dB), and each stage have different patterns. As shown in FIG. 6, it can be seen that the high-sensitivity region extends farther from the microphone arrays MA400 and MA500 (that is, toward the lower right).

Therefore, the combination of FIG. 5(a) (the combination of microphone arrays MA301 and MA302), the combination of FIG. 5(b) (the combination of microphone arrays MA302 and MA303), the combination of FIG. The area sound pickup area (sensitivity distribution of area sound pickup) differs for each combination of microphone arrays. will occur.

That is, as shown in FIG. 5, in the configuration of three microphones ch1 to ch3, area sound pickup is performed by combining two or three different microphone arrays, and if the respective sound pickup results are added, one microphone It is possible to pick up sound in a wider range than the sound pickup area achieved by combining arrays.

Therefore, in this embodiment, a combination (combination pattern), and the result of adding or averaging the area sound collection results (output of area sound collection) is treated as the final sound collection result of the target area. As a result, in the area sound pickup processing of this embodiment, area sound pickup is more robust (more stable) against the difference in the position of the speaker's mouth (the position of the speaker's mouth as seen from the transmitter). area sound pickup) can be performed.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing the configuration of each device related to this embodiment.

FIG. 1 illustrates a communication device 100 including a sound pickup unit 120 and a communication device 200 according to this embodiment. In addition, in FIG. 1, the communication apparatuses 100 and 200 are configured to be able to communicate with each other through a communication path P. As shown in FIG.

The communication device 100 picks up the voice (sound) uttered by the first user U1, transmits voice data of the picked-up voice to the communication device 200 via the communication path P, and receives the voice data from the communication device 200. It is a device that phonetically outputs voice (speech uttered by the second user U2) based on voice data. Further, the communication device 200 collects voice (sound) uttered by the second user U2, transmits audio data of the collected voice to the communication device 100 via the communication path P, It is a device that phonetically outputs voice (speech uttered by the first user U1) based on the received voice data.

The first user U1 is, for example, a crew member appearing in an emergency vehicle such as an ambulance or a fire engine. This applies to the commanding officer, etc.

The communication path P is not limited to wired or wireless, and various connection means and connection configurations (network configurations) can be applied.

Next, an overview of the configuration of the communication device 100 will be described with reference to FIG.

Communication device 100 includes handset 110 , sound pickup section 120 , communication section 130 and output section 140 .

The handset 110 has a microphone array section 111 and a speaker 112, which are composed of three microphones MC1 to MC3 (3ch microphones).

The communication unit 130 is a communication interface for communicating with the communication device 200 via the communication path P. FIG.

The sound pickup unit 120 picks up the voice (sound) uttered by the first user U1 based on the acoustic signal captured by the microphone array unit 111 . Then, the communication unit 130 transmits audio data of the sound collected by the sound pickup unit 120 to the communication device 200 side.

The output unit 140 acquires voice data (speech data of voice uttered by the second user U2) from the communication device 200 via the communication unit 130, and supplies an acoustic signal based on the voice data to the speaker 112. 112 to phonetically output the acoustic signal.

Although the hardware configuration of the communication device 100 is not limited, in this embodiment, as shown in FIG. It is assumed that Note that the communication device 100 does not necessarily include the handset 110, and the entire housing (chassis), like a smartphone, has a configuration that substantially functions as a handset (for example, transmission to a part of the housing of the smartphone). A configuration in which a mouth is set) may be used.

Next, an overview of the configuration of the communication device 200 will be described using FIG.

The communication device 200 has a speaker 210 , a microphone 220 , a communication section 230 , an output section 240 and a sound pickup section 250 . The hardware configuration of the communication device 200 is also not limited, but for example, various telephone devices (eg, speakerphones, etc.) can be applied.

The communication unit 230 is a communication interface for communicating with the communication device 200 via the communication path P. FIG.

The sound pickup unit 250 picks up the voice (sound) uttered by the second user U2 based on the acoustic signal captured by the microphone 220 . Then, the communication unit 230 transmits audio data of the sound collected by the sound pickup unit 250 to the communication device 100 side.

The output unit 240 acquires voice data (speech data of voice uttered by the first user U1) from the communication device 100 via the communication unit 230, and supplies an acoustic signal based on the voice data to the speaker 210. 210 to output the acoustic signal phonetically.

Next, a detailed configuration of the sound pickup unit 120 will be described with reference to FIG.

The sound pickup unit 120 has a signal input unit 121 , a frequency conversion unit 122 , a directivity formation unit 123 , a target area sound extraction unit 124 and an area sound addition unit 125 .

For example, the sound pickup unit 120 may cause a computer having a processor, memory, etc. to execute a program (including the sound pickup program according to the embodiment). It can be shown as in FIG. Details of the processing of each component of the sound pickup unit 120 will be described later.

Next, the configuration of handset 110 as a handset will be described with reference to FIGS. 2 and 3. FIG.

FIG. 2 is a perspective view showing a state in which handset 110 is held by hand U1a of first user U1.

As shown in FIG. 2, the handset 110 includes a bar-shaped handle portion 115 to be held by the first user U1 (hand U1a), and a mouthpiece 113 (transmitter) provided at one end of the handle portion 115. and an earpiece 114 (receiver) provided at the other end of the handle portion 115 .

FIG. 3 is an enlarged view of the mouthpiece 113 portion of the handset 110. As shown in FIG.

As shown in FIG. 2, a speaker 112 is arranged in the earpiece 114 . Also, as shown in FIGS. 2 and 3, a microphone array section 111 (microphones MC1 to MC3) is arranged in a mouthpiece 113 having a circular surface.

Next, the configuration of the microphone array section 111 will be described with reference to FIGS. 2 and 3. FIG.

In the example of this embodiment, the microphone array section 111 is configured to have three microphones MC1 to MC3.

As shown in FIG. 2, when the first user U1 holds the communication device 100 with the hand U1a and presses the speaker SP against the ear, the circumference of the mouthpiece 113 where the mouth of the first user U1 is located ( Three microphones MC1 to MC3 are arranged around the part closest to the mouth of the first user U1.

In the handset 110 shown in FIGS. 2 and 3, each position (center position of each microphone) of the three microphones MC1 to MC3 constituting the microphone array section 111 is the same as the configuration shown in FIGS. , are arranged on the periphery of the mouthpiece 113 so as to form vertices of an equilateral triangle. In FIGS. 2 and 3, each side of the triangle formed by the microphones MC1 to MC3 has the same distance (the triangle formed by the microphones MC1 to MC3 is an equilateral triangle) in order to expand the sound pickup area in an isotropic direction. and angles of each angle may not be the same.

As shown in FIG. 3, in the microphone array unit 111, the microphone array paired with the microphones MC1 and MC2 is MA1, the microphone array paired with the microphones MC2 and MC3 is MA2, and the microphones MC3 and MC1 are paired. A microphone array with the following is called MA3.

(A-2) Operation of First Embodiment Next, the operation of this embodiment having the above configuration (sound pickup method according to the embodiment) will be described.

In the communication device 100, the sound pickup unit 120 uses acoustic signals supplied from the microphones MC1 to MC3 of the microphone array unit 111 to perform target area sound pickup processing for picking up the target area sound of the target area.

The operation inside the sound pickup unit 120 constituting the communication device 100 will be mainly described below.

The signal input unit 121 converts the acoustic signals picked up by the microphones MC1 to MC3 from analog signals to digital signals, and supplies the digital signals to the frequency conversion unit 122 . After that, the frequency transform unit 122 transforms the microphone signal from the time domain to the frequency domain using, for example, fast Fourier transform. The directivity forming unit 123 forms directivity by BF.

Here, directivity formation by BF will be described with reference to FIGS. 7 and 8. FIG.

BF is a technique for forming the directivity of sound pickup using the time difference between signals reaching each microphone in a microphone array (see Non-Patent Document 1). BFs are broadly classified into two types: addition type and subtraction type. Here, a subtraction type BF capable of forming directivity with a small number of microphones will be described.

FIG. 7 is a block diagram showing the configuration of the subtractive BF 600 when the number of microphones is two (MC1, MC2).

FIG. 8 is a diagram showing directivity characteristics formed by a subtractive BF 600 using two microphones MC1 and MC2.

The subtraction type BF 600 first calculates the time difference between the signals of the sound existing in the target direction (hereinafter referred to as the “target sound”) arriving at each of the microphones MC1 and MC2 using the delay device 610, and adds a delay to the target sound. Match the phase of the sound. The time difference is calculated by equation (1). Here, d is the distance between the microphones MC1 and MC2, c is the speed of sound, and _τi is the amount of delay. θ _L indicates the angle from the direction perpendicular to the straight line connecting the positions of the microphones MC1 and M2 to the target direction.

Here, when a blind spot exists in the direction of the microphone MC1 with respect to the center of the microphone MC1 and the microphone MC2, the delay device 610 performs delay processing on the input signal x ₁ (t) of the microphone MC1. After that, the subtractor 620 performs subtraction processing according to equation (2). In the subtractor 620, this subtraction process can be performed in the frequency domain as well, in which case equation (2) is changed to equation (3).

Here, when θ _L =±π/2, the formed directivity is cardioid unidirectional as shown in FIG. 8(a ₎ . Figure 8 bi-directivity as shown in (b) is obtained. In addition, the subtractor 620 can also form strong directivity in bi-directional blind spots by using Spectral Subtraction processing (hereinafter also simply referred to as “SS”). Directivity by SS is formed in all frequencies or in a specified frequency band according to the equation (4). Although the input signal X1 of the microphone MC1 is used in the equation _{( 4} ), the same effect can be obtained with the input signal X2 of the microphone MC2. Here, n indicates a frame number, and β indicates a coefficient for adjusting the strength of SS. In the subtractor 620, if the value becomes negative during subtraction, flooring processing may be performed in which the value is replaced with 0 or a smaller value than the original value. In this method, sound existing in directions other than the target direction (hereinafter referred to as "non-target sound") is extracted due to bidirectional characteristics, and the amplitude spectrum of the extracted non-target sound is subtracted from the amplitude spectrum of the input signal. can emphasize the target sound.

By the way, when it is desired to pick up only the target area sound existing in a certain target area, only using the subtractive BF will not detect the sound source existing on the line in the same direction as the area (hereinafter referred to as "non-target area sound"). call) will also be picked up.

Therefore, in the directivity forming unit 123, the area sound pickup processing proposed in Patent Document 1 (using a plurality of microphone arrays, directing the directivity from each different direction to the target area, and making the directivity cross at the target area Processing for collecting the target area sound) will be described. Specifically, the directivity forming unit 123 may perform area sound pickup processing by the following processing.

The directivity forming unit 123 forms directivity by BF toward the inside of the triangle (the triangle formed by the microphones MC1 to MC3) for each of the microphone arrays MA1 to MA3. Then, directivity forming section 123 supplies BF outputs Y ₁ (n), Y ₂ (n), and Y ₃ (n) of microphone arrays MA 1 , MA 2 , and MA 3 to target area sound extracting section 124 .

The target area sound extraction unit 124 extracts the area sound using the BF outputs Y ₁ (n), Y ₂ (n), and Y ₃ (n) of the microphone arrays MA1, MA2, and MA3 formed by the directivity formation unit 123. do. As described above, each BF output (Y ₁ (n), Y ₂ (n), Y ₃ (n)) is generated from each side of the triangle (the triangle formed by microphones MC1-MC3) to the center (the inner side of the triangle). direction). Therefore, in each BF output, the two directivities intersect near the center of the triangle in any two combinations (combination patterns). can extract the sound in the area where the directivities intersect with each other. Here, as representatives, the case where the BF output Y ₁ (n) of the microphone array MA1 and the BF output Y ₂ (n) of the microphone array MA2 are used will be described. The target area sound extraction unit 124 performs SS on Y ₁ (n) and Y ₂ (n) according to formula (5) or (6), and extracts non-target area sounds N _1-1 (n) existing in the direction of the target area. , N _1-2 (n). Here, α ₁ and α ₂ are correction coefficients for correcting signal level differences caused by differences in distance between the target area and each microphone array, and should be calculated one by one by a predetermined process. Although described in Reference 1, here, for simplicity, the distances from the target area to each microphone array are assumed to be the same (α ₁ (n)=α ₂ (n)=1), and (5), (6) ) is replaced with equations (7) and (8).

After that, the target area sound extraction unit 124 extracts the target area sound by SSing the non-target area sound from each BF output according to formulas (9) and (10). Here, γ ₁ (n) and γ ₂ (n) are coefficients for changing the intensity during SS.

In the target area sound extraction unit 124, any one of the emphasized sounds Z _1-1 (n) and Z _1-2 (n) may be output _. It is used as the area sound pickup output Z ₁ (n) by the combination (combination pattern) of the microphone array MA2.

Similarly, the target area sound extraction unit 124 obtains an area picked-up sound output Z ₂ (n) by the combination of the microphone array MA2 and the microphone array MA3, and an area picked-up sound output Z ₃ (n) by the combination of the microphone array MA3 and the microphone array MA1. ) is extracted and supplied to the area sound addition unit 125 .

As shown in FIG. 2, the microphones MC1 to MC3 are all mounted in a narrow range of several centimeters in diameter in the mouthpiece 113 of the handset 110. FIG. Therefore, the microphone arrays MA1, MA2, and MA3 are arranged very closely (densely), and their sound pickup areas are limited to a narrow range in front of the mouthpiece 113. FIG. However, as shown in FIG. 6 described above, it is known that the sound pickup area by area sound pickup has the characteristic of expanding in the far direction of the two microphone arrays. Therefore, if the sound pickup areas (sound pickup areas corresponding to Z ₁ (n), Z ₂ (n), and Z ₃ (n)) extending in three different directions are overlapped, a single sound pickup area ( It is possible to pick up sound in a wider range than the sound pickup area corresponding to any one of Z ₁ (n), Z ₂ (n), and Z ₃ (n).

Therefore, the area sound adder 125 adds or averages the outputs Z ₁ (n), Z ₂ (n), and Z ₃ (n) of the three area sound pickups to obtain a final output (sound pickup result) W( n) is generated and output as the sound pickup result of the sound pickup unit 120 . The area sound addition unit 125 considers that areas overlap each other in the addition process, and adds the outputs of the three area sound pickups (Z ₁ (n) + Z ₂ (n), +Z ₃ ( n)) may be averaged or multiplied by a gain adjustment coefficient α as shown in equation (11). Note that the area sound addition unit 125 adds (or averages) only two or more of the three area sound outputs (Z ₁ (n), Z ₂ (n), Z ₃ (n)). ) may be performed. For example, the area sound addition unit 125 may perform processing of adding (or averaging) only two outputs among the three area sound outputs.

As described above, the sound pickup unit 120 outputs the final output W(n) as the target sound picked up from the enlarged area. At this time, the sound pickup unit 120 may output W(n) as sound data obtained by frequency-time conversion.

The communication unit 130 then transmits the audio data based on the final output W(n) to the communication device 200 via the communication path P. FIG.

Then, the communication unit 230 of the communication device 200 supplies the audio data (the audio data based on W(n)) received from the communication device 100 to the output unit 140 . The output unit 140 supplies an acoustic signal based on the received audio data to the speaker 210 to produce a phonetic output (a phonetic output toward the second user U2).

(A-3) Effects of First Embodiment According to this embodiment, the following effects can be obtained.

The sound pickup unit 120 of this embodiment performs area sound pickup from different directions, and adds them together to obtain a wider and isotropic area sound pickup than the conventional one-pair (two) microphone array. It is possible to form a sound pickup area (enlarged sound pickup area) with a certain property. As a result, in the sound pickup unit 120, when performing area sound pickup using the microphones MC1 to MC3 attached to the mouthpiece 113 of the handset 110, the mouth of the speaker (first user U1) and the mouthpiece Even if the position relative to 113 is displaced, stable sound pickup is possible.

(B) Second Embodiment Hereinafter, a second embodiment of the sound collecting device, program and method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound collecting device, program, and method of the present invention are applied to a sound collecting unit will be described.

As described above, the sound pickup unit 120 of the first embodiment picks up area sounds from different directions and superimposes (adds) them, thereby using one set (two) of conventional microphone arrays. It forms an isotropic sound pickup area (enlarged sound pickup area) that is wider than the normal sound pickup area.

However, an attempt to expand the sound pickup area like the sound pickup unit 120 of the first embodiment, on the other hand, suppresses surrounding unwanted sounds by picking up sound only in a specific area, and suppresses the target sound. There is a risk that the original effect of area sound pickup, which is to emphasize, will be diminished.

Therefore, in the sound pickup unit 120A of the second embodiment, the deterioration of the target sound enhancement performance is suppressed while expanding the sound pickup possible area in order to solve the above-described problem in the first embodiment. ing.

(B-1) Configuration of Second Embodiment FIG. 9 is a block diagram showing the configuration of each device related to the second embodiment. In FIG. 9, the same reference numerals or corresponding reference numerals are assigned to the same or corresponding portions as those in FIG.

In the second embodiment, the communication device 100 is replaced with a communication device 100A. Also, in the communication device 100A of the second embodiment, the microphone array section 111 and the sound pickup section 120 are replaced with a microphone array section 111A and a sound pickup section 120A.

Next, the internal configuration of the sound pickup section 120A in the second embodiment will be described.

Next, the internal configuration of the sound pickup section 120A will be described with reference to FIG.

The sound pickup unit 120A differs from the first embodiment in that the target area sound extraction unit 124 is replaced with a target area sound extraction unit 124A and the area sound addition unit 125 is omitted. Further, the sound pickup unit 120A differs from the first embodiment in that a partial area component calculation unit 126 and a partial area selection unit 127 are added.

Next, the configuration of the microphone array section 111A of the second embodiment will be described.

As shown in FIG. 9, in the second embodiment, the microphone array section 111A has six microphones MC1 to MC6.

FIG. 10 is a diagram showing an arrangement of the six microphones MC1 to MC6 in the microphone array section 111A and a configuration example of the microphone array.

As shown in FIG. 10, the six microphones MC1 to MC6 that constitute the microphone array section 111A are divided into three microphone arrays MA1 (microphone array paired with microphones MC1 and MC2) and MA2 (microphone arrays paired with two microphones each). A microphone array paired with microphones MC3 and MC4) and MA3 (a microphone array paired with microphones MC5 and MC6).

From the viewpoint of increasing the number of divisions and extracting the target sound from pinpoint areas, it is desirable to have a configuration of three or more areas. An example of area sound pickup in two areas will be described. A configuration of three overlapping areas will be described in a third embodiment, which will be described later.

(B-2) Operation of Second Embodiment Next, the operation of the second embodiment having the above configuration (sound pickup method according to the embodiment) will be described.

The signal input unit 121 converts the acoustic signals picked up by the six microphones MC1 to MC6 from analog signals to digital signals x ₁ to x ₆ , respectively, and supplies the digital signals x 1 to x 6 to the frequency conversion unit 122 .

The frequency transform unit 122 transforms the microphone signals x ₁ to x ₆ from the time domain to signals X ₁ to X ₆ in the frequency domain using, for example, fast Fourier transform.

Directivity forming section 123 forms directivity by BF using the input signal of each microphone that has been time-frequency converted by frequency converting section 122 . In the second embodiment, the BF output from the microphone array MA1 is Y ₁ , the BF output from the microphone array MA2 is Y ₂ , and the BF output from the microphone array MA3 is Y ₃ . The directivity of the BF outputs Y ₁ , Y ₂ and Y ₃ are as shown in FIG. In the second embodiment, as shown in FIG. 10, the microphone arrays MA1 to MA3 are arranged at the positions of the vertices of the triangle, and the directivity of the BF outputs Y ₁ , Y ₂ and Y ₃ (the directivity of the microphone arrays MA1 to MA3 directivity) are each oriented inward of the triangle.

The target area sound extraction unit 124A uses the BF output generated by the directivity formation unit 123 to perform area sound pickup processing. In the area sound pickup, the directivity of the BF is directed from different directions, and the components (area sound) of the areas where the directivities intersect are separated and extracted. Area sound pickup can be realized from each of the combination of BF outputs Y ₁ and Y ₂ and the combination of BF outputs Y ₁ and Y ₃ .

FIG. 11 is an explanatory diagram showing the distribution of sound pickup areas where the target area sound extraction unit 124A performs area sound pickup.

As shown in FIG. 6, area sound pickup has the characteristic that the sound pickup area spreads in the direction far from the microphone array. Therefore, an area sound pickup area by the microphone arrays MA1-MA2 (referred to as "area 1" or "sound pickup area 1" in the second embodiment) and an area sound pickup area by the microphone arrays MA2-MA3 (second In the embodiment, the area is called "area 2" or "sound pickup area 2") has an image as shown in FIG. In the second embodiment, the area picked-up sound component (area picked-up sound output) of the sound pickup area 1 is Z ₁ , and the area picked-up sound component (area picked-up sound output) of the area 2 is Z ₂ .

Each sound pickup area is divided into a portion where two sound pickup areas overlap and an independent portion where two sound pickup areas do not overlap, as shown in FIG.

In FIG. 12, the area where areas 1 and 2 overlap is defined as overlapping area 1∧2. In addition, in FIG. 12, an independent area (an area that does not overlap with other sound pickup areas) excluding the overlapping area 1∧2 in the area 1 is defined as an independent area A. In FIG. Furthermore, in FIG. 12, an independent area B is defined as an independent area within the area 2 except for the overlapping area 1∧2. An independent area (independent part) derived from one sound collecting area may be divided into a plurality of areas as shown in FIG. Areas are collectively indicated by one code. For example, in FIG. 12, the independent area A is divided (divided) into two areas by the overlapping area 1∧2.

As described above, area 1 consists of overlapping area 1 ∧ 2 and independent area A (area 1 minus overlapping area 1 ∧ 2), and area 2 consists of overlapping area 1 ∧ 2 and independent area B (overlapping from area 2). area 1 ∧ 2). By overlapping (adding) the area sound pickup output Z1 of area ₁ and the area sound pickup output Z2 of area ₂ , sound can be picked up from a wide range of areas, but the components of overlapping area 1 ∧ 2 are doubled. As a result, uniform sound pickup characteristics cannot be obtained for the entire sound pickup area. Therefore, if it is possible to separate and extract the sound sources of the overlapping area 1 ∧ 2 and the independent areas A and B individually, the areas 1 and 2 can be unified over the entire range by integrating the respective areas without overlapping. Sound pickup characteristics are obtained.

The partial area component calculation unit 126 calculates the picked-up sound component of the overlapping area 1^2 and the independent area area sound components (here, area sound components of independent areas A and B) are separated.

FIG. 13 is an explanatory diagram showing a composition image (power for each component) of each area shown in FIG. 12 in the form of a bar graph.

13(a) shows a composition image of area picked-up sound component Z1 of area 1, and FIG. 13(b) shows _a composition image of area picked-up sound component Z2 of area _2. FIG. FIG. 13(c) shows the composition image of the area sound pickup component Z1 shown in FIG. 13(a), with the components of the overlapping area _1∧2 hatched (diagonal line pattern). Further, FIG. 13(d) shows the composition image of the area sound pickup output Z2 shown in _FIG . 13(b), with the components of the overlapping area 1∧2 hatched (diagonal line pattern).

The overlapped area 1∧2 of area ₁ and area ₂ is literally overlapped and common, so it is included in Z1 and Z2 as the same component. Therefore, the target area sound extraction unit 124A separates each component by using the spectral subtraction method (SS) based on the same principle as for area sound pickup.

The partial area component calculator 126 SSs the area picked _- up sound output Z1 to the area picked _- up sound output Z2. The partial area component calculator 126 clips to 0 the component that becomes negative in SS. By doing so, in the target area sound extraction unit 124A, the area picked-up sound component of the overlapping area _1∧2 is removed from the area picked-up sound output Z1, and the area picked-up sound component of the independent area A (" V _A ”) are isolated. Similarly, the partial area component calculation unit 126 calculates the area picked-up sound component (“V _B ” in the first embodiment) of the independent area B by SSing the area picked-up sound output Z ₁ from the area picked-up sound output Z ₂ . ) can be separated.

FIG. 14 is an explanatory diagram showing a procedure of processing for calculating sound pickup components (V _A , V _B ) of the areas of the independent area (independent areas A, B) by the partial area component calculation unit 126 .

14(a) to 14(c), the partial area component calculation unit 126 SSs the area picked _- up sound output Z2 of the area ₂ from the area picked-up sound output Z1 of the area 1, and the independent area A The process of extracting the area picked-up sound component V _A (the process corresponding to the following equation (21)) is shown, and the individual graphs in FIGS. _A composition image of Z ₁ , area picked-up sound component Z ₂ of area 2, and area picked-up sound component VA of independent area A is shown.

14(d) to 14(f), the partial area component calculator 126 SSs the area picked-up sound component Z1 of the area ₁ from the area picked-up sound component Z2 of the area ₂ to make it independent. The process of extracting the area picked-up component V _B of the area B (the process corresponding to the following formula (22)) is shown, and the individual graphs in FIGS. A composition image of the collected sound component Z ₂ , the area collected sound component Z ₁ of the area 1, and the area collected sound component V _B of the independent area B is shown.

In each composition image shown in FIG. 14, the area picked-up sound component of overlapping area 1 ∧ 2, the area picked-up sound component V _A of independent area A, and the area picked-up sound component V _B of independent area B are different. It is illustrated with a pattern attached.

In the partial area component calculation unit 126, based on the area picked-up sound component V A of the independent area _A or the area picked-up sound component V _B of the independent area B, the area picked-up sound component of the overlapping area 1 2 (hereinafter referred to as "V _{1 ∧ 2} ”) can be obtained. For example, the partial area component calculator 126 SSs the area picked-up sound component _VA of the independent area _A from the area picked-up sound output Z1. The partial area component calculator 126 clips to 0 the component that becomes negative in SS. By doing so, the target area sound extraction unit 124A removes the area picked-up sound component V _A of the independent area _A from the area picked-up sound output _Z1 , and separates the area picked-up sound component V1^2 of the overlapping area 1^2. be done. Similarly, the partial area component calculation unit 126 separates the area picked-up sound component V1^2 of the overlapping area 1^2 by _SSing the area picked-up sound component VB of the independent area _B from the area picked _- up sound output Z2. can do.

FIG. 15 is an explanatory diagram showing a procedure of processing for the partial area component calculator ₁₂₆ to calculate the area picked-up sound component V 1 2 of the overlapping area 1 2 .

15(a), 15(b), and 15(e), the partial area component calculator 126 SSs the area picked-up sound component V _A of the independent area A from the area picked-up output Z ₁ to obtain the overlapping area. The process of extracting the area picked-up sound component V _1̂2 of 1̂2 (the process corresponding to the following formula (23)) is shown. 15(c), 15(d), and 15(e), the partial area component calculator 126 SSs the area picked-up sound component V _B of the independent area B from the area picked-up output Z ₂ to obtain the overlapping area. It shows the processing (processing corresponding to the following equation (24)) for extracting the area picked-up sound component V _1̂2 of 1̂2. As described above, in the partial area component calculation unit 126, the area picked-up sound component V _A of the independent area A, the area picked-up sound component V _B of the independent area B, and the area picked-up sound component V 1 ^2 of the overlapping area 1 _^2 can be separated and extracted.

The partial area selection unit 127 selects and outputs any of the area picked-up sound components V _1-2 , V _A , and V _B of each divided partial area in addition to the area picked-up sound components Z ₁ and Z ₂ ( output as the final sound pickup result W). Note that the selection processing method by the partial area selection unit 127 is not limited.

As described above, the sound pickup unit 120A outputs the area picked-up sound component selected by the partial area selection unit 127 as the final output W(n).

The communication unit 130 then transmits the audio data based on the final output W(n) to the communication device 200 via the communication path P. FIG.

Then, the communication unit 230 of the communication device 200 supplies the audio data (the audio data based on W(n)) received from the communication device 100A to the output unit 140 . The output unit 140 supplies an acoustic signal based on the received audio data to the speaker 210 to produce a phonetic output (a phonetic output toward the second user U2).

(B-3) Effects of Second Embodiment According to the second embodiment, the following effects can be obtained as compared with the first embodiment.

In the sound pickup unit 120A of the second embodiment, for two area sound pickup outputs having overlapping areas, the overlap between the areas is utilized, and the area components of each of the overlapping areas and the independent areas that do not overlap are separated and extracted. divides the whole area into a plurality of smaller areas. Then, in the sound pickup unit 120A of the second embodiment, by selecting an area most suitable as the target sound pickup area from among the divided small areas, the range of the entire area is covered by the multi-area sound pickup. , it becomes possible to extract the emphasized speech from the pinpoint area containing the target sound.

(C) Third Embodiment Hereinafter, a third embodiment of the sound collecting device, program and method according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound collecting device, program, and method of the present invention are applied to a sound collecting unit will be described.

(C-1) Configuration of Third Embodiment FIG. 16 is a block diagram showing the configuration of each device related to the third embodiment.

In FIG. 16, the same reference numerals or corresponding reference numerals are given to the same or corresponding portions as Y1 described above. The third embodiment will be described below, focusing on differences from the second embodiment.

The third embodiment differs from the second embodiment in that the communication device 100A is replaced with a communication device 100B. Further, the communication device 100B of the third embodiment differs from the second embodiment in that the microphone array section 111A is replaced with the microphone array section 111B. Furthermore, the communication device 100B of the third embodiment differs from the second embodiment in that the sound pickup unit 120A is replaced with the sound pickup unit 120B.

Next, the internal configuration of the sound pickup section 120B according to the third embodiment will be described.

In the sound pickup unit 120B of the third embodiment, a target area sound extraction unit 124A, a partial area component calculation unit 126, and a partial area selection unit 127 are respectively composed of a target area sound extraction unit 124B, a partial area component calculation unit 126B, and a partial area sound extraction unit 126B. It differs from the second embodiment in that it is replaced with an area selection section 127B.

Next, the configuration of the microphone array section 111B will be described using FIG.

In the example of this embodiment, as shown in FIG. 17, the communication device 100B has a hardware configuration of a smart phone (a smart phone owned by the speaker U1). Also, in the example of the third embodiment, the microphone array section 1B is configured to have three microphones MC1 to MC3.

In addition, as shown in FIG. 17, in the example of this embodiment, the communication device 100 has the configuration of a smart phone, so these three microphones MC1 to MC3 are the parts (speaker SP is placed around the edge opposite to the part where the In other words, in the communication device 100, the three microphones MC1 to MC3 can be arranged around the portion facing the mouth of the speaker U1 (the portion closest to the mouth of the speaker U1) when the communication device 100 is used. desirable. In FIG. 17, when the speaker U1 holds the communication device 100 by hand and presses the speaker SP against the ear, the part where the mouth of the speaker U1 is located (the lower part when viewed from the direction of FIG. 17) is shown. Three microphones MC1 to MC3 are arranged around (around the part closest to the mouth of speaker U1).

In the communication device 100 (microphone array unit 1) shown in FIG. 17, each position (center position of each microphone) of the three microphones MC1 to MC3 is arranged so as to be a vertex of an equilateral triangle. In this embodiment, three microphone arrays MA1 to MA3 are configured by combining three microphones MC1 to MC3. Hereinafter, as shown in FIG. 17, a microphone array paired with microphones MC1 and MC2 is called MA1, a microphone array paired with microphones MC2 and MC3 is called MA2, and a microphone array paired with microphones MC3 and MC1 is called MA3. shall be

In this embodiment, the microphones MC1 to MC3 are arranged in an equilateral triangle in order to expand the area in the same direction, but the arrangement is not necessarily limited to an equilateral triangle. In other words, the distances of the sides of the triangle formed by the microphones MC1 to MC3 and the angles of the corners may not all be the same.

As described above, in the third embodiment, as shown in FIG. 18, three microphone arrays (MA1 to MA3) are configured from three microphones (MC1 to MC3). Area sound pickup shall be performed.

(C-2) Operation of the Third Embodiment Next, the operation of the third embodiment (sound pickup method according to the embodiment) having the configuration as described above will be described.

The signal input unit 121 converts the acoustic signals picked up by the three microphones MC1 to MC3 from analog signals to digital signals x ₁ to x 3 and supplies the digital signals x 1 to x ₃ to the frequency conversion unit 122 .

The frequency transform unit 122 transforms the microphone signals from the time domain to signals X ₁ to X ₃ in the frequency domain using, for example, fast Fourier transform.

Directivity forming section 123 forms directivity by BF using the input signal of each microphone that has been time-frequency converted by frequency converting section 122 . In the _third embodiment, the BF output from the microphone array MA1 is Y1 _, the BF output from the microphone array MA2 is Y2, and the BF _output from the microphone array MA3 is Y3.

The target area sound extraction unit 124 uses the BF outputs Y ₁ , Y ₂ , and Y ₃ formed by the directivity formation unit 123, and the combinations of Y ₁ -Y ₂ , Y ₂ -Y ₃ , and Y ₃ -Y ₁ , perform area sound pickup processing respectively.

In the third embodiment, the area (sound pickup area) of the combination of Y ₁ -Y ₂ is "1", the area (sound pickup area) of the combination of Y ₂ -Y ₃ is 2, and the combination of Y ₃ -Y ₁ The area (sound pickup area) due to is called "3".

As shown in FIG. 6, area sound pickup has the characteristic that the sound pickup area spreads in the direction far from the microphone array. Therefore, area 1 by BF outputs Y ₁ -Y ₂ (microphone arrays MA1-MA2), area 2 by BF outputs Y ₂ -Y ₃ (microphone arrays MA2-MA3), BF outputs Y ₃ -Y ₁ (microphone arrays MA3- The distribution of the sound pickup area by MA1) is imaged as shown in FIG. In the third embodiment, area picked-up sound components (area picked-up sound outputs) of areas ₁ , ₂ , and ₃ are Z1, Z2, and Z3, respectively.

The partial area component calculator 126B uses the area picked-up outputs Z ₁ , Z ₂ , and Z ₃ to calculate a portion where the three areas overlap, a portion where the two areas overlap, and an independent portion without overlap. do. In the second embodiment, two sound pickup areas overlap, but in the third embodiment, there are three sound pickup areas, so the overlapping pattern is more complicated than in the second embodiment. Become. The same method (calculation method) as in the second embodiment is used to calculate each portion of the overlapping area of the three sound pickup areas by decomposing into a combination of two area sound pickup components whose area sound pickup components are known. It becomes possible.

Specifically, the partial area component calculation unit 126B calculates each combination of two sound pickup areas (a combination of areas 1 and 2, a combination of areas 2 and 3, an area 3 and 1) (hereinafter referred to as "combination pattern"), the same method as in the second embodiment can be used. That is, in the partial area component calculation unit 126B, the process of separating the area sound pickup components of the two sound pickup areas having the overlapping area into the overlapping area portion and the independent area portion is the same as in the second embodiment. .

Hereinafter, the combination pattern of areas 1 and 2 will be referred to as a "first combination pattern", the combination pattern of areas 2 and 3 will be referred to as a "second combination pattern", and the combination pattern of areas 3 and 1 will be referred to as a "third combination pattern". shall be referred to as the "combination pattern of

FIG. 19 is an explanatory diagram (image diagram) showing an exploded image of combination patterns (first to third combination patterns) of two sound pickup areas for three areas 1 to 3. FIG.

FIG. 19(a) is a diagram showing three areas 1 to 3 superimposed. FIGS. 19(b) to 19(d) are explanatory diagrams showing images decomposed into first to third combination patterns, respectively.

First, of the three combination patterns shown in FIGS. 19B to 19D, the first combination pattern (combination pattern of areas 1 and 2) shown in FIG. 19B will be described as a representative example.

The partial area component calculation unit 126B obtains an independent part of area 1 with respect to area ₂ (referred to as "area A" in this embodiment) by SSing area picked-up sound output Z1 to area picked _- up sound output Z2. ( _refer to FIG. 19(b)) is obtained. In addition, the partial area component calculation unit 126B SSs the area picked _- up sound output Z2 from the area picked _- up sound output Z1, so that an independent portion of the area 2 with respect to the area 1 ("area B" in this embodiment) (refer to FIG. 19(b)) (referred to as “V _B ” in the third embodiment) can be obtained. Similar to the second embodiment, the partial area component calculator 126B can obtain the area picked-up sound components V _A and V _B using the above equations (21) and (22).

In the partial area component calculation unit 126B, similarly for the second combination pattern (combination pattern of areas 2 and 3), a portion of area 2 independent of area 3 (referred to as "area C" in this embodiment) 19(c)) (referred to as “V _C ” in the third embodiment) and a portion of area 3 independent of area 2 (in the third embodiment, , “area D”; see FIG. 19(c)) can be obtained (referred to as “V _D ” in the third embodiment). Similarly, in the partial area component calculation unit 126B, for the third combination pattern (combination pattern of areas 3 and 1), a portion of area 3 independent of area 1 ("area E" in this embodiment) (see FIG. 19(d)) and the area sound pickup component (referred to as “V _E ” in the third embodiment) and the portion of area 1 independent of area 3 (in this embodiment, , “area _F ”; see FIG. 19(d)) can be obtained.

The partial area component calculator 126B can obtain the area picked-up sound components V _C , V _D , V _E and V _F according to the following equations (31) to (34).

In the partial area component calculation unit 126B, when the area picked-up sound component V _A of the area A or the area picked-up sound component V _{B of the area B} is known, the portion where the areas 1 and 2 overlap (in this embodiment, "overlapping area 1 2”; see FIG. _19B ) can be obtained (referred to as “V 1 2 ” in this embodiment). Specifically, the partial area component calculation unit _126B , as shown in the following equation (35), calculates the area ₁ 2 area sound components can be obtained. In addition, the partial area component calculation unit _126B also calculates the area collected sound component VB of area _B from the collected area sound output Z2 as shown in the following equation (36). sound component can be obtained.

Similarly, in the partial area component calculation unit 126B, based on the area picked-up sound component V C of the area _C or the area picked-up sound component V _D of the area D, as shown in the following formulas (37) and (38), The area sound pickup component (referred to as "V _2∧3 " in this embodiment) of the area where areas 2 and 3 overlap (referred to as "overlapping area 2∧3" in this embodiment; see Fig. 19(c)) is Obtainable. Further, the partial area component calculation unit _{126B calculates} an area Obtain an area pickup component (referred to as “V _{3 1} ” in this embodiment) of an area where 3 and 1 overlap (referred to as “overlapping area 3 1 ” in this embodiment; see FIG. 19(d)) be able to.

In the partial area component calculation unit 126B, area picked-up sound components (V _A , V _B , V _C , V _D , V _E , V _F ) and overlapped portions (V _{1 ∧ 2} , V _{2 ∧ 3} , V _{3 ∧ 1} ) are all calculated. can calculate the area pickup component of each partial area.

20 to 23 are explanatory diagrams showing each partial area when three areas, area 1, area 2, and area 3, are overlapped at the same time. For example, in the above calculation, the area picked-up sound component V _A of area A and the area picked-up sound component V _D of area D are already known, so the partial area component calculation unit 126B uses the following equation (41): As shown, by SSing the area picked-up sound component V _A of the area A to the area picked-up sound component V _D of the area D by the same calculation as before, the independent part of the area A (hereinafter referred to as "area Ad" 20) can be obtained (hereinafter referred to as “V _Ad ”). Further, the partial area component calculation unit 126B calculates the area picked-up sound component V _{D of the area A from the area picked-up sound component V D of} the area D as shown in the following equation (42) by the same calculation as before. By doing so, it is possible to obtain an area pickup component (hereinafter referred to as "V _Da ") of an independent portion of area D (hereinafter referred to as "area Da"; see FIG. 20).

After obtaining the area picked-up sound component VAd of the area _Ad or the area picked-up sound component VDa of the area _Da , the partial area component calculation unit 126B obtains the overlapping portion of the area A and the area D (hereinafter referred to as “area A∧D ”; _see FIG. 20) can be calculated.

Specifically, as shown in the following equation (43), the partial area component calculation unit 126B calculates the area picked-up sound component V _Ad of the area Ad from the area picked-up sound output V _D of the area D in the same manner as before. By performing SS according to the calculation method, it is possible to calculate the area picked-up sound component V _A∧D of the area A∧D. In addition, the partial area component calculation unit _126B also calculates the area _A ∧ D can be _calculated .

Similarly, in the partial area component calculation unit 126B, as shown in the following equations (45) and (47), the independent part of the area B (hereinafter referred to as “area Be”; see FIG. 21) area sound pickup component (hereinafter referred to as “V _Be ”) and an area pickup component (hereinafter referred to as “V _Eb ”) of an independent portion of area E (hereinafter referred to as “area Eb”; see FIG. 21) can be done. Then, in the partial area component calculation unit 126B, based on the area collected sound component V Be of the area _Be or the area collected sound component V _Eb of the area Eb, the area It is possible to calculate an area picked-up component (hereinafter referred to as "V _B∧E ") of the overlapping portion of B and area E (hereinafter referred to as "area B∧E"; see Fig. 21).

Further, in the partial area component calculation unit 126B, as shown in the following formulas (49) and (51), the area pickup component of the independent part of the area C (hereinafter referred to as "area Cf"; see FIG. 22) (hereinafter referred to as “V _Cf ”) and an area pickup component (hereinafter referred to as “V _Fc ”) of an independent portion of area F (hereinafter referred to as “area Fc”; see FIG. 22). can. Then, in the partial area component calculation unit 126B, based on the area picked-up sound component V Cf of the area _Cf or the area picked-up sound component V _Fc of the area Fc, the area It is possible to calculate an area sound pickup component (hereinafter referred to as "VCF") of the overlapping portion of C and area F (hereinafter referred to as "area _C∧F "; see FIG. 22).

Then, in the partial area component calculation unit 126B, when the area picked-up sound component V _{1 ∧ 2} and the area picked-up sound component V _{C ∧ F} are known, the area ( Hereafter, it is possible to obtain an area pickup component (hereinafter referred to as "V _{1 2 3} ") of the area 1 2 3; see FIG. 23). Specifically, in the partial area component calculation unit 126B, as shown in the following equation (53), the area picked-up sound component V _C^ F is calculated from the area picked-up sound component V _1^2 by the same calculation method as before. By performing SS, it is possible to obtain the area _picked -up sound component V 1 2 3 of the area 1 2 3 .

Similarly, in the partial area component calculation unit 126B, when the area picked-up sound component V2 _^3 and the area picked-up sound component _VB^E are known, area 1^2^3 is obtained by the following equation (54). V _{1 2 3} can be obtained. Furthermore, in the partial area component calculation unit 126B, when the area picked-up sound component V _{3 ∧ 1} and the area picked-up sound component _{VA ∧ D} are known, the area 1 ∧ 2 ∧ 3's V _{1 ∧ 2∧3} can be obtained.

Through the above processing, the partial area component calculation unit 126B _obtains the area picked-up sound component (V 1 2 3 ) of the portion where the three areas overlap, and Area sound components ( _VA∧D , _VB∧E , _VC∧F ) in the portion where the two areas overlap, and area sound in the independent portion without overlapping with other areas among the three areas Components (V _Ad , V _Da , V _Be , V _Eb , V _Cf , V _Fc ) can be separated and extracted.

The partial area selection unit 127B picks up the area pickup sound component of the area that is estimated to contain the most target sound component from among the area pickup sound components of the areas decomposed into each part in this way. Select (acquire) as result W. Hereinafter, each area divided from all areas (all areas covered by areas 1, 2, and 3) in partial area selection section 127B will be referred to as a "partial area". For example, the partial area selection unit 127B selects areas 1, 2, 3, A, B, C, D, E, F, 1^2, 2^3, 3^1, 1^2^3, Some or all of Ad, Da, A^D, Be, Eb, B^E, Cf, Fc, and C^F may be set.

The method by which the partial area selection unit 127B selects one of the partial areas (area-collected sound components) is not limited. In addition, below, the partial area selected by the partial area selection unit 127B is also referred to as a "selected area".

For example, the partial area selection unit 127B may select a partial area with the largest power (for example, an area with the largest average power spectrum obtained by averaging each frequency component constituting the area sound pickup component of the partial area). good. Further, at that time, the partial area selection unit 127B may select one of the partial areas based on the evaluation value obtained by normalizing the power of each partial area by area. For example, the partial area selection unit 127B calculates an evaluation value for each partial area so that an area with the same power but a smaller area has a higher evaluation, and selects the partial area with the highest evaluation (evaluation value). can be

Moreover, when selecting a partial area, the partial area selection unit 127B does not have to select all the partial areas. For example, the partial area selection unit 127B excludes an area whose area (area) is smaller than the other partial areas from the selection targets (for example, an area that is less than one-third the area of the largest partial area). A partial area may be excluded from selection targets), and selection may be made from the remaining partial areas (partial areas not excluded).

Further, the partial area selection unit 127B may integrate areas adjacent to the area selected by the above method (areas bordering on each other) and select them as one partial area. For example, the partial area selection unit 127B may select an area with the highest power and an area adjacent to that area.

Furthermore, the partial areas to be selected by partial area selection section 127B do not necessarily have to be independent of each other. For example, the partial area selection unit 127B may select partial areas having overlapping areas, such as area D and area A shown in FIG. 20, as selection targets. In other words, the partial area selection unit 127B selects an area that is smaller than the entire area (the entire area covered by areas 1, 2, and 3) and does not include areas unnecessary for extracting the target sound. An area should be selected.

Then, the partial area selection unit 127B acquires the area picked-up sound component of the partial area selected by the above processing as the final picked-up sound result W(n). The method of selecting partial areas (area picked-up sound components of partial areas) in the partial area selection section 127B as described above may be applied to the partial area selection section 127A of the second embodiment.

As described above, the sound pickup unit 120B outputs the area picked-up sound component selected by the partial area selection unit 127B as the final output W(n).

The communication unit 130 then transmits the audio data based on the final output W(n) to the communication device 200 via the communication path P. FIG.

Then, the communication unit 230 of the communication device 200 supplies the audio data (the audio data based on W(n)) received from the communication device 100B to the output unit 140 . The output unit 140 supplies an acoustic signal based on the received audio data to the speaker 210 to produce a phonetic output (a phonetic output toward the second user U2).

(C-3) Effects of Third Embodiment According to the third embodiment, the following effects can be obtained.

In the communication device 100B (sound pickup unit 120B) according to the third embodiment, three overlapping sound pickup areas include an independent area that does not overlap with any area, an area that overlaps two areas, and three overlapping areas. Areas where all three areas overlap, calculating the components of each part. Then, in the communication apparatus 100B (sound pickup unit 120B) of the third embodiment, an area most suitable as the target sound pickup area is selected from among the divided areas, and the area sound pickup component of the selected area is finalized. It is output as a typical sound collection result. As a result, in the communication apparatus 100B (sound pickup unit 120B) of the third embodiment, the selected area is much smaller than the entire area while covering the entire range of the multiple area sound pickup results. , it is possible to extract the emphasized speech from the pinpoint area containing the target sound without including unnecessary area components.

(D) Other Embodiments The present invention is not limited to the above embodiments, and modified embodiments such as those exemplified below can be exemplified.

(D-1) In each of the above-described embodiments, the sound pickup unit is a part of the communication device, but it may be constructed as an independent device. Further, in each of the above-described embodiments, the sound pickup unit does not include the microphone array unit, but the sound pickup unit and the microphone array unit may be integrated as a device.

(D-2) In each of the above-described embodiments, an example in which the sound collecting device (sound collecting unit) of the present invention is applied to a device having a hand-held transmitter (transmitter/receiver) such as a handset has been described. The sound collecting device of the present invention is applied to a headset or a wearable device (for example, a head-mounted display with a microphone, a neckband type headphone with a microphone, etc.), and when worn by the first user U1, the first user U1 An area where the mouth is positioned may be set as a target area, microphones may be installed at each vertex of a polygon (N-sided polygon) around the area (mouthpiece), and area sound pickup processing may be performed in the same manner as in the above embodiment. .

(D-3) In the first and third embodiments, an example of area sound pickup using three microphones MC1 to MC3 was shown, but the number of microphones installed in the microphone array unit 111 (the number of microphones The number of sides (corners) of the polygon is not limited. For example, area sound pickup may be performed from three or four directions. For example, in the first and third embodiments, four microphones may be arranged at the corner vertices of a quadrangle.

FIG. 24 is an explanatory diagram showing a configuration when the number of microphones in the microphone array section 111 is four.

In FIG. 24, four microphones MC1 to MC4 are arranged at the corner vertices of a quadrangle (square). The four microphones MC1 to MC4 are combined with adjacent microphones to form a microphone array MA701 formed by a pair of microphones MC1 and MC2, a microphone array MA702 formed by a pair of microphones MC2 and MC3, a microphone MC3, A microphone array MA703 formed by a pair of MC4 and a microphone array MA704 formed by a pair of microphones MC4 and MC1 are formed. Further, these microarrays are combined with adjacent microphone arrays (combination of microphone arrays sharing some microphones) to enable sound pickup in four areas. For example, when the configuration of four microphones MC1 to MC4 is applied to the microphone array unit 111, the sound pickup unit 120B performs area sound pickup by the combination of the microphone arrays MA701 and MA702 and area pickup by the combination of the microphone arrays MA702 and MA703. It is possible to acquire each output of area sound pickup by combination of sound, microphone arrays MA703 and MA704, and area sound pickup output by combination of microphone arrays MA704 and MA701 (four area sound pickup outputs). Then, the sound pickup unit 120B can acquire a sound pickup result based on the outputs of the above four area sound pickups.

(D-4) In the sound pickup unit 120B (partial area selection unit 127B) of the third embodiment, one area containing the most target sound is selected from a plurality of areas. You may make it In this case, the sound pickup unit 120B (partial area selection unit 127B) of the third embodiment may integrate (add) area sound pickup components of a plurality of selected areas to obtain a sound pickup result W. good. However, in this case, in the sound pickup unit 120B (partial area selection unit 127B) of the third embodiment, the selected multiple areas are selected so as not to have common portions (overlapping portions). The area should be considered in advance.

DESCRIPTION OF SYMBOLS 100, 100A, 100B... Communication apparatus 110... Handset 111... Microphone array part MC1-MC6... Microphone 112... Speaker 113... Mouthpiece 114... Earpiece 115... Handle part 120, 120A, 120B Sound pickup unit 121 Signal input unit 122 Frequency conversion unit 123 Directivity formation unit 124, 124A, 124B Target area sound extraction unit 125 Area sound addition unit 126, 126B Partial area components Calculation unit 127, 127B partial area selection unit 130 communication unit 140 output unit 200 communication device 210 speaker 220 microphone 230 communication unit 240 output unit 250 sound pickup unit , U1... first user, U1a... listener's hand, U2... second user, P... channel.

Claims (6)

Area sound pickup for acquiring area sound components of a plurality of sound pickup areas based on a combination of two or more patterns of the microphone arrays based on an input signal from a microphone array section capable of forming a plurality of microphone arrays with different directivities. means and
A partial area where two or more of the sound pickup areas that are divided from the total area covering all of the sound pickup areas acquired by the area sound pickup means overlaps, and a partial area that does not overlap between the sound pickup areas. Partial area component calculation means for acquiring sound pickup components based on the area sound pickup components of the sound pickup areas of each pattern acquired by the area sound pickup means;
A partial area that selects area sound pickup components of one or more partial areas from the area sound pickup components of the partial areas calculated by the partial area component calculation means, and obtains a sound pickup result based on the selected area sound pickup components. a selection means ;
The microphone array unit includes N microphones arranged at positions of vertices of N polygons (N is an integer of 3 or more), or N microphones arranged at positions of vertices of N polygons. Having a pair of microphone arrays
A sound collecting device characterized by 2. The partial area selection means according to claim 1, wherein said partial area selection means selects the area picked-up sound component of one or a plurality of partial areas based on the result of comparing the power of the area picked-up sound component of each partial area. sound collection device. 3. The sound collecting device according to claim 2, wherein said partial area selection means selects an area picked-up sound component having the strongest power from the area picked-up sound components of the partial areas calculated by said partial area component calculation means. . 3. The partial area selection means, when selecting area picked-up sound components of a plurality of partial areas, acquires a result of adding the selected plurality of area picked-up sound components as a picked-up result. The sound collecting device described in . the computer,
Area sound pickup for acquiring area sound components of a plurality of sound pickup areas based on a combination of two or more patterns of the microphone arrays based on an input signal from a microphone array section capable of forming a plurality of microphone arrays with different directivities. means and
A partial area where two or more of the sound pickup areas that are divided from the total area covering all of the sound pickup areas acquired by the area sound pickup means overlaps, and a partial area that does not overlap between the sound pickup areas. Partial area component calculation means for acquiring sound pickup components based on the area sound pickup components of the sound pickup areas of each pattern acquired by the area sound pickup means;
A partial area that selects area sound pickup components of one or more partial areas from the area sound pickup components of the partial areas calculated by the partial area component calculation means, and obtains a sound pickup result based on the selected area sound pickup components. act as a means of selection,
The microphone array unit includes N microphones arranged at positions of vertices of N polygons (N is an integer of 3 or more), or N microphones arranged at positions of vertices of N polygons. Having a pair of microphone arrays
Sound collection program characterized by. In the sound collection method performed by the sound collection device,
Area sound collection means, partial area component calculation means, and partial area selection means,
The area sound pickup means picks up sound in a plurality of sound pickup areas based on a combination of two or more patterns of the microphone arrays based on an input signal from a microphone array unit capable of forming a plurality of microphone arrays with different directivities. get the ingredients,
The partial area component calculation means calculates a partial area in which two or more of the sound collection areas overlap, divided from the entire area covering all of the sound collection areas acquired by the area sound collection means, and a partial area in which the sound collection areas overlap each other. Acquiring area sound components of each non-overlapping partial area based on the area sound components of the sound collecting areas of each pattern acquired by the area sound collecting means;
The partial area selecting means selects area picked-up sound components of one or a plurality of partial areas from the area picked-up sound components of the partial areas calculated by the partial area component calculating means, and picks up sound based on the selected area picked-up sound components. get the sound result,
The microphone array unit includes N microphones arranged at positions of vertices of N polygons (N is an integer of 3 or more), or N microphones arranged at positions of vertices of N polygons. Having a pair of microphone arrays
A sound collection method characterized by:

Images