JP7207159B2 - Sound collection device, sound collection program, sound collection method, and sound collection system - Google Patents

Description

TECHNICAL FIELD The present invention relates to a sound collecting device, a sound collecting program, a sound collecting method, and a sound collecting system. can be applied to a system that suppresses

Currently, in the systems that make up call centers that handle phone calls with customers, the content of conversations between operators and customers is recorded (recorded) in order to improve the operator's response skills, comply with compliance, prevent troubles, and take countermeasures. The introduction of recording systems (sound pickup systems) is progressing. In addition, in companies that conduct BtoC (Business to Consumer) business, there are usually various sales office counters that directly interact with customers face-to-face instead of answering telephones as points of contact with customers other than call centers. A typical example of this is a financial institution consultation desk that provides consultation on housing loans, inheritance, asset management, and the like. At present, there is an increasing demand to record the contents of conversations between the respondent and the consulter (customer) at such sales counters for the same purpose as the call center. In the following, this system will be referred to as a "face-to-face recording system" as opposed to the "call recording system" described above.

In a conventional call recording system, the operator and the customer are separated from each other in terms of the place of speech and the sound receiving devices such as headsets and receivers. It is possible. On the other hand, in conventional face-to-face recording systems, the voices of the respondent and the consulter cannot be distinguished because they are usually recorded by a single microphone (monaural or stereo) placed on the desk between the respondent and the consulter. It will be recorded without mixing. This is a major obstacle to the subsequent use of data, such as confirming the content of dialogue using recorded data, converting the content of dialogue into text using speech recognition technology, and utilizing it for knowledge management (e.g., text mining). becomes. In order to apply speech recognition to recorded speech, it is necessary to separate the voices of the respondent and the consulter, as well as various ambient sounds in the real environment (conversation sounds in adjacent booths, back office conversations, equipment noise, waiting room noise, etc.). It is necessary to prevent contamination of conversations, call announcements that tell the turn, etc.).

By the way, when using a speech recording system or a speech recognition application system in a noisy environment, ambient noise mixed in at the same time as the necessary target speech impedes the understanding of the conversation content and reduces the speech recognition rate of the recorded speech. It is a nuisance that brings degradation. Conventionally, in an environment where multiple sound sources exist, a beamformer using a microphone array has been used as a technology that separates and collects only the sound from a specific direction to avoid mixing in unwanted sounds and obtain the necessary target sound. (Beam Former; hereinafter also referred to as "BF"). BF is a technique of forming directivity using the time difference between signals arriving at each microphone (see Non-Patent Document 1). BFs are broadly classified into two types: addition type and subtraction type. In particular, the subtractive BF has an advantage over the additive BF in that directivity can be formed with a smaller number of microphones.

FIG. 12 is a block diagram showing the configuration of the subtraction type BF 400 when the number of microphones is two (M1, M2).

The subtraction type BF 400 has a delay device 410 and a subtractor 420 .

The subtraction-type BF 400 calculates the time difference between the signals of sound existing in the target direction (hereinafter referred to as “target sound”) arriving at each microphone, and delays the target sound by the delay device 410 to adjust the phase of the target sound. match. The time difference at this time is calculated by the following formula (1). Here, d is the distance between the microphones, c is the speed of sound, _τi is the delay amount, and _θL is the angle from the vertical direction to the straight line connecting the microphones to the target direction.

Further, here, the delay device 410 delays the input signal x ₁ (t) of the microphone M1 when a blind spot exists in the direction of the microphone M1 with respect to the center of the microphones M1 and M2. After that, the subtractor 420 performs the subtraction process according to the formula (2).

Subtractor 420 can perform subtraction processing in the same way in the frequency domain, in which case equation (2) is changed to equation (3) below.

Here, when θ _L =±π/2, the formed directivity is cardioid unidirectional as shown in FIG. 13(a ₎ . Figure 8 bi-directivity as shown in (b) is obtained. Here, a filter that forms unidirectionality from an input signal is called a "unidirectional filter", and a filter that forms bidirectionality is called a "bidirectional filter".

The subtractor 420 can also form strong directivity in bidirectional blind spots by using spectral subtraction (hereinafter also referred to as “SS”). Directivity by SS is formed in all frequencies or in a specified frequency band according to the equation (4). Although the input signal X1 of the microphone M1 is used in the equation _{( 4} ), the same effect can be obtained with the input signal X2 of the microphone M2. Here, β is a coefficient for adjusting the intensity of SS. When the SS is processed according to the formula (4), if the value becomes negative at the time of subtraction, flooring processing is performed to replace it with 0 or a smaller value than the original value. This method uses a bidirectional filter to extract sounds that exist in directions other than the target direction (hereafter referred to as "non-target sounds"), and subtracts the amplitude spectrum of the extracted non-target sounds from the amplitude spectrum of the input signal. , can emphasize the target sound direction.

When it is desired to pick up only sounds that exist within a certain specific area (hereinafter referred to as “target area sounds”), only using a subtractive BF will not collect sound sources that exist around that area (hereinafter referred to as “non-target area sounds”). (called "sound") may also be picked up. Therefore, in the technique described in Patent Document 1, a method of collecting target area sound by using a plurality of microphone arrays, directing the directivity from each different direction to the target area, and intersecting the directivity at the target area (area recording) is proposed.

FIG. 14 is an explanatory diagram showing processing for picking up target area sound from a sound source in the target area using two microphone arrays MA1 and MA2.

FIG. 14A is an explanatory diagram showing a configuration example of each microphone array. FIGS. 14(b) and 14(c) are graphs (image diagrams) showing in the frequency domain the BF outputs of the microphone arrays MA1 and MA2 shown in FIG. 14(a), respectively. In the area sound pickup, as shown in FIG. 14(a), the directivity of the microphone arrays MA1 and MA2 is crossed in areas (target areas) where the sound is to be picked up from different directions to pick up the sound. In the state of FIG. 14(a), the directivity of each of the microphone arrays MA1 and MA2 includes not only the sound existing in the target area (target area sound) but also the noise in the direction of the target area (non-target area sound). there is However, as shown in FIGS. 14(b) and 14(c), when the directivity of the microphone arrays MA1 and MA2 is compared in the frequency domain, the target area sound component is included in both outputs, but the non-target area The sound component will be different for each microphone array. In the conventional area sound pickup technique, by using such characteristics, only the target area sound can be extracted by suppressing components other than those commonly included in the BF outputs of the two microphone arrays MA1 and MA2.

When conventional area sound pickup processing is performed in an environment such as that shown in FIG. Two microphone arrays will be used. At this time, the correction coefficient of the target area sound amplitude spectrum is calculated by the "formulas (5) and (6)" or the "formulas (7) and (8)". where Y _1k (n), Y _2k (n) are the amplitude spectra of the BF outputs of the microphone arrays MA1, MA2, N is the total number of frequency pins, k is the frequency, α ₁ (n), α ₂ (n) is the amplitude spectrum correction factor for each BF output. Also, here, mode indicates the most frequent value, and median indicates the median value.

After that, by correcting each BF output with a correction coefficient and performing SS, non-target area sounds existing in the direction of the target area can be extracted. Furthermore, the target area sound can be extracted by SSing the extracted non-target area sound from the output of each BF. For example, to extract the non-target area sound N ₁ (n) existing in the direction of the target area viewed from the microphone array MA1, the BF output Y ₁ (n) of the microphone array MA1 is converted to the microphone SS the BF output Y ₂ (n) of the array MA2 multiplied by the amplitude spectrum correction factor. Similarly, according to the equation (10), the non-target area sound N ₂ (n) existing in the direction of the target area viewed from the microphone array MA2 is extracted.

After that, according to equations (11) and (12), the target area sound can be extracted by SSing the non-target area sound from each BF output. Expression (11) indicates extraction of the target area sound using the microphone array MA1 as a reference, and expression (12) using the microphone array MA2 as a reference. In equations (11) and (12), γ ₁ (n) and γ ₂ (n) are coefficients for changing the intensity during SS.

As described above, in the method of Patent Document 1, the directivities of a plurality of microphone arrays are crossed from different directions toward the target area, and components commonly included in the directional outputs of the microphone arrays are extracted. As a result, with the technique of Patent Document 1, even in an environment where there are various unwanted sound sources in the surroundings, it is possible to emphasize and pick up the sound of a specific area such as the desk for the respondent and the desk for the counselor.

JP 2014-072708 A

Futoshi Asano, "Acoustic Technology Series 16 Sound Array Signal Processing - Localization, Tracking and Separation of Sound Sources -", Edited by Acoustical Society of Japan, Corona Publishing, February 25, 2011

By the way, in a face-to-face consultation desk, it is necessary to separately emphasize and record two areas, the receptionist's seat and the consultant's seat facing each other across the counter. Further, as shown in FIG. 15, in an actual consultation counter, there is usually only one person who responds, but there is not always one person who consults. For example, it is not uncommon for clients to visit with family members (for example, parents and siblings, spouse, etc.). For this reason, at general consultation counters, two or more seats are provided for the consultation side. The position at which the consulter speaks also changes depending on whether there is one consulter or two consulters. For example, as shown in FIG. 15(a), when there is only one person to consult, there are many cases in which the respondent sits in the center of the front. However, as shown in FIG. 15(c) and FIG. In some cases, you can put your luggage on the seat and sit on either the left or right side. Also, if two consulters visit, one person sits at each of the two seats to have a conversation, as shown in FIG. 15(b). Considering these cases, it is necessary to be able to set the sound pickup area by dividing the area of the client side to be recorded into a plurality of areas.

In view of the above problems, the sound pickup area on the first side (for example, the seat on the side of the respondent at the consultation desk) and the sound pickup area on the second side (for example, the seat on the side of the counselor at the consultation desk) When performing area pick-up on the second side, consider the number and position of speakers in the pick-up area on the second side, and select the optimum area pick-up process for area-pickup of each speaker's voice. A sound collecting device, a sound collecting program, a sound collecting method, and a sound collecting system capable of

A first sound collecting device of the present invention comprises: (1) a first speaker area in which one or more speakers are located; and one or more speakers who interact with the speakers in the first speaker area. based on input signals from a plurality of microphone array units comprising a plurality of microphone arrays comprising a plurality of microphones capable of directivity toward a plurality of sub-areas within the second speaker area where the respective (2 ) area sound detection means for performing area sound detection processing for detecting whether or not sub-area sounds are generated for each sub-area; Accordingly, an area sound determination process for determining the content of the sound pickup result of the second speaker area is performed, and a signal corresponding to the area sound determination process is acquired as the sound pickup result of the second speaker area. sound processing means; and (4) an output for outputting a result of area sound collection processing of the first speaker area and a result of area sound collection processing of the second speaker area acquired by the area sound processing means. and means.

A sound collecting program of a second aspect of the present invention comprises: (1) a first speaker area in which one or more speakers are located; obtaining beamformer outputs of each of the microphone arrays based on input signals from a plurality of microphone arrays capable of directivity toward a plurality of sub-areas within a second speaker area in which the speakers of area sound extraction means for performing area sound pickup processing on the first speaker area and each sub-area using the acquired beamformer output; and (2) sub-area sound for each sub-area. (3) area sound detection means for performing area sound detection processing for detecting whether or not the second speaker is occurring; (4) area sound processing means for performing area sound determination processing for determining the content of the sound pickup result of the area, and acquiring a signal corresponding to the area sound determination processing as the sound pickup result for the second speaker area; It is characterized by functioning as output means for outputting a result of area sound pickup processing of the first speaker area and a result of area sound pickup processing of the second speaker area acquired by the area sound processing means. .

A third aspect of the present invention is a sound collection method performed by a sound collection device, comprising: (1) area sound extraction means, area sound detection means, area sound processing means, and output means; , a first speaker area in which one or more speakers are located; obtaining a beamformer output of each of the microphone arrays based on input signals from a plurality of microphone arrays capable of directing directivity to a sub-area; using the obtained beamformer output; (3) the area sound detecting means performs area sound detection processing for detecting whether or not a sub-area sound is generated in each sub-area; (4) the area sound processing means determines the content of the sound pickup result of the second speaker area according to the combination pattern of the determination results of the area sound detection processing for each sub-area; (5) the output means performs area sound pickup processing for the first speaker area; A result and a result of the area sound pickup processing of the second speaker area acquired by the area sound processing means are output.

A fourth aspect of the present invention comprises a first speaker area in which one or more speakers are located, and a second speaker area in which one or more speakers who interact with the speakers in the first speaker area are located. a plurality of microphone array units capable of directing directivity to a plurality of sub-areas within a speaker area; The sound collecting system is characterized in that the sound collecting device of the first aspect of the present invention is applied as the sound collecting device.

According to the present invention, when performing area sound collection of the sound collection area on the first side and the sound collection area on the second side, the number and positions of speakers present in the sound collection area on the second side are determined. It is possible to select the optimum area sound pickup process for area sound pickup of each speaker's voice.

1 is a block diagram showing the functional configuration of a sound collecting system (including a sound collecting device) according to a first embodiment; FIG. FIG. 4 is an explanatory diagram (image diagram) showing a sound pickup area set in the first embodiment; FIG. 4 is a diagram showing an example of a layout image of the microphone array unit according to the first embodiment; 4 is a diagram showing the configuration of each microphone array unit according to the first embodiment; FIG. 4 is a diagram showing the configuration of a microphone array formed by each microphone array unit according to the first embodiment; FIG. 2 is a block diagram showing an example of the hardware configuration of the sound collecting device according to the first embodiment; FIG. FIG. 7 is a block diagram showing the functional configuration of a sound collection system (including a sound collection device) according to a second embodiment; FIG. 11 is an explanatory diagram (image diagram) showing a sound pickup area set in the second embodiment; FIG. 11 is a diagram showing an example of an image of the arrangement of microphone array units according to the second embodiment; FIG. 10 is a diagram showing the configuration of each microphone array unit according to the second embodiment; FIG. 10 is a diagram showing the configuration of a microphone array formed by each microphone array unit according to the second embodiment; FIG. 3 is a block diagram showing the configuration of a conventional subtractive BF (when the number of microphones is two); FIG. 4 is an explanatory diagram showing an example of a directivity filter formed by a conventional subtractive BF (when the number of microphones is two); FIG. 10 is an explanatory diagram showing a configuration example of a conventional sound collection device in which the directivity of the beamformer (BF) of two microphone arrays is directed from different directions to a target area. It is explanatory drawing (image figure) which showed the variation of staff arrangement|positioning in a consultation window.

(A) First Embodiment Hereinafter, a first embodiment of a sound collecting device, a sound collecting program, a sound collecting method, and a sound collecting system according to the present invention will be described in detail with reference to the drawings. In this embodiment, an example in which the sound collection system of the present invention is applied to a face-to-face recording system will be described.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing the overall configuration of a face-to-face recording system 1 of the first embodiment.

The face-to-face recording system 1 is a device that distinguishes and records the voices of a respondent and a consulter at a consultation counter or the like, and has a microphone array section 100 , a sound collecting device 200 and a recording device 300 .

The microphone array unit 100 is means for capturing the voices of the respondent and the consulter at the consultation desk.

The sound pickup device 200 is a device that distinguishes and picks up the voices of the responder and the consulter based on the acoustic signals captured by the microphone array section 100 .

The recording device 300 is a device that records the voices (the voices of the respondent and the consulter) collected by the sound collection device 200 and the textual data of the content of the conversation based on the collected sounds.

Here, in the face-to-face recording system 1, a sound pickup area from which sound is picked up will be described with reference to FIG.

FIG. 2 is a diagram showing a sound pickup area from which sound is picked up and the surrounding environment in the face-to-face recording system 1. As shown in FIG.

FIG. 2 shows an example in which the responder and the consulter are positioned across a rectangular counter table CT. That is, FIG. 2 shows the case where the responder and the consulter are positioned to face each other with the counter table CT interposed therebetween.

As shown in FIG. 2, in this embodiment, a sound pickup area (hereinafter referred to as "responder area A1") where one responder is located, and a responder located in the responder area A1. It is assumed that a sound pickup area (hereinafter referred to as "consultant area A2") in which the consulter is located is set. In this embodiment, as shown in FIG. 2, the counselor's seat area A2 further includes a counselor's seat center area A2-1, a counselor's left seat area A2-2, and a counselor's right seat area A2-3. Assume that three sound collection areas (sub-areas) are set.

As shown in FIG. 15, in an actual consultation counter, there is usually only one respondent, but there are cases where there are multiple consulters. Considering the case shown in FIG. 14 at the consultation desk, the sound pickup area of the consulter to be recorded is, as shown in FIG. (Either left seat area A2-2 or right seat area A2-3), both left and right (both left seat area A2-2 and right seat area A2-3) It is desirable to be able to emphasize and pick up the sound of each of the three areas (center area A2-1 of the customer's seat, A2-2 of the left-side area of the customer's seat, and A2-3 of the right-side area of the customer's seat).

As shown in FIG. 2, in the first embodiment, the counselor's left seat area A2-2 and the counselor's right seat area A2-3 are arranged in the lateral direction (the counselor's seat center area A2-1). left and right direction as viewed from above). In FIG. 2, a line CL1 connecting the responder area A1 (for example, the center point of the responder area A1) and the center area A2-1 of the counselor's seat (for example, the center point of the center area A2-1 of the counselor's seat) , and a line CL2 perpendicular to the line CL1. In the example of this embodiment, as shown in FIG. 2, it is assumed that the midpoint of the line CL1 and the center point of the counter table CT coincide, and the line CL2 is a line passing through the center point of the line CL1.

Next, the internal configuration of the microphone array section 100 will be described.

The microphone array section 100 has two microphone array units MAU1 and MAU2.

FIG. 3 is a diagram showing an example of arrangement configuration of the microphone array units MAU1 and MAU2.

In FIG. 3, the area where the microphone array units MAU1 and MAU2 are arranged is viewed from above.

In the microphone array section 100 of the first embodiment, the microphone array units MAU1 and MAU2 are arranged separately on the left and right sides of the line CL1. In the example of FIG. 3, the microphone array units MAU1 and MAU2 are arranged at symmetrical positions on the line CL2 with the line CL1 interposed therebetween. In other words, in the example of FIG. 3, the microphone array unit MAU1 is arranged on the left side of the counter table CT when viewed from the center area A2-1 of the consultant's seat, and the microphone array unit MAU2 is arranged on the right side. .

Next, the configuration of each microphone array unit in the first embodiment will be explained using FIG.

As shown in FIG. 4(a), the microphone array unit MAU1 installed on the left side as viewed from the caller side has a line segment substantially orthogonal to the direction of the respondent area A1 as viewed from the microphone mc1, with the microphone mc1 as an end point. With the microphones mc2 and mc1 arranged above as end points, the microphone mc3 arranged on a line segment substantially perpendicular to the direction of the central area A2-1 of the person's seat as seen from the microphone mc1 and the consultation as seen from the microphone mc1. It has a microphone mc4 arranged on a line segment substantially perpendicular to the direction of the left seat area A2-2. In this embodiment, as shown in FIG. 3, the right seat area A2-3 of the counselor is located ahead of the center area A2-1 of the counselor's seat when viewed from the microphone array unit MAU1 (microphone mc1). Assume that the microphone array unit MAU1 is arranged so as to

Further, as shown in FIG. 4B, the microphone array unit MAU2 installed on the right side as viewed from the client side is substantially perpendicular to the direction of the respondent area A1 as viewed from the microphone mc5, with the microphone mc5 as an end point. Microphones mc6 and mc5 arranged on a line segment are arranged on a line segment substantially orthogonal to the direction of the center area A2-1 of the counselor's seat as viewed from the microphone mc5, with the microphones mc6 and mc5 as endpoints. A microphone mc8 is arranged on a line segment substantially orthogonal to the direction of the right side seat area A2-3 of the client. In this embodiment, as shown in FIG. 3, the left seat area A2-2 of the counselor is located beyond the center area A2-1 of the counselor's seat when viewed from the microphone array unit MAU2 (microphone mc5). Assume that the microphone array unit MAU2 (microphone mc5) is arranged as shown in FIG.

FIG. 5 is a diagram showing combinations of microphones used in the microphone array formed by each microphone array unit of the microphone array section 100. As shown in FIG.

In the microphone array section 100, a microphone array can be configured by combining the eight microphones mc1 to mc8 that configure the microphone array units MAU1 and MAU2. In this embodiment, as shown in FIG. 5A, microphone array MA1 is formed by microphones mc1 and mc2, microphone array MA2 is formed by microphones mc1 and mc3, and microphone array MA3 is formed by microphones mc1 and mc4. shall be In this embodiment, as shown in FIG. 5(b), the microphone array MA4 is formed by the microphones mc5 and mc6, the microphone array MA5 is formed by the microphones mc5 and mc7, and the microphone array MA5 is formed by the microphones mc5 and mc8. Assume that an array MA6 is formed.

Next, the internal configuration of the sound collecting device 200 will be described.

As shown in FIG. 1, the sound collection device 200 includes a signal input unit 201, a time/frequency conversion unit 202, a directivity formation unit 203, an area sound extraction unit 204, an area sound detection unit 206, an area sound selection unit 207, and a frequency/time converter 208 . Details of each element constituting the sound collecting device 200 will be described later.

For example, the sound collecting device 200 may cause a computer having a processor, memory, etc. to execute a program (including the sound collecting program according to the embodiment). It can be shown as in FIG. Details of the processing of each component of the sound collecting device 200 will be described later.

FIG. 6 is a block diagram showing an example of the hardware configuration of the sound collecting device 200. As shown in FIG. Note that the symbols in parentheses in FIG. 6 are symbols used in second to fourth embodiments described later.

FIG. 6 shows a configuration when configuring the sound collecting device 200 using software (computer).

A sound collecting device 200 shown in FIG. 6 has, as hardware components, at least a signal input unit 201 and a computer 500 in which a program (a program including the sound collecting program of the embodiment) is installed.

The signal input section 201 can be configured using, for example, an A/D converter. Note that if the computer 500 itself is equipped with an A/D converter, the signal input section 201 need not be provided separately.

The computer 500 performs processing for performing area sound pickup processing on the acoustic signal (digital acoustic signal) supplied from the signal input unit 201 and outputting the processed signal. In this embodiment, it is assumed that a program (software) including the sound pickup program of this embodiment is installed in the computer 500 .

The computer 500 may be a computer dedicated to the sound collection program, or may be configured to be shared with programs for other functions (for example, the recording device 300).

A computer 500 shown in FIG. 6 has a processor 501 , a primary storage unit 502 and a secondary storage unit 503 . The primary storage unit 502 is storage means that functions as a working memory (work memory) for the processor 501, and for example, a high-speed memory such as a DRAM (Dynamic Random Access Memory) is applied. The secondary storage unit 503 is storage means for recording various data such as an OS (Operating System) and program data (including sound pickup program data according to the embodiment). sensitive memory is applied. In the computer 500 of this embodiment, when the processor 501 is activated, the OS and programs (including the sound pickup program according to the embodiment) recorded in the secondary storage unit 503 are read, and expanded on the primary storage unit 502. Run.

Note that the specific configuration of the computer 500 is not limited to the configuration of FIG. 6, and various configurations can be applied. For example, if the primary storage unit 502 is a non-volatile memory (for example, FLASH memory), the secondary memory may be excluded.

Next, the functional configuration of the recording device 300 will be described.

The recording device 300 has a dialogue recording unit 301 and a dialogue storage unit 302 .

The recording device 300 may also be configured in software by causing a computer to execute a program, but even in that case, it can be functionally shown as in FIG. Details of the processing of each component of the sound collecting device 200 will be described later.

(A-2) Operation of First Embodiment Next, the operation of the face-to-face recording system 1 of the first embodiment having the configuration as described above will be described.

The operation of the face-to-face recording device according to the embodiment of the present invention will be described in detail below with reference to FIG.

The signal input unit 201 converts the acoustic signals picked up by the eight microphones mc1 to mc8 from analog signals to digital signals, and supplies the digital signals to the time/frequency conversion unit 202 . In the following, digital audio signals (hereinafter also referred to as “input signals”) picked up by the microphones mc1 to mc8 are denoted by x1 to x8, respectively.

A time/frequency converter 202 converts the microphone signal from the time domain to the frequency domain. In the following, signals obtained by transforming the input signals x1 to x8 into the frequency domain are expressed as X1 to X8, respectively.

Directivity forming section 203 forms directivity by BF according to formulas (3) and (4) using the frequency-converted input signals (X1 to X8) of each microphone. Below, the BF outputs from the microphone arrays MA1 to MA6 are expressed as Y1 to Y6, respectively.

As shown in FIG. 3, the directivity of the BF output Y1 from the microphone array MA1 is directed toward the respondent area A1, and the directivity of the BF output Y2 from the microphone array MA2 is directed toward the center area A2-1 of the counselor's seat and The directivity of the BF output Y3 from the microphone array MA3 is directed toward the left seat area A2-2 of the client. As shown in FIG. 3, the directivity of the BF output Y4 from the microphone array MA4 is directed toward the respondent area A1, and the directivity of the BF output Y5 from the microphone array MA5 is directed toward the center area A2- 1 and the client's left seat area A2-2, and the directivity of the BF output Y6 from the microphone array MA6 is directed to the client's right seat area A2-3.

Therefore, the directivities of BF output Y1 and BF output Y4 intersect in the respondent area A1, the directivities of BF output Y2 and BF output Y5 intersect in the center area A2-1 of the counselor's seat, and BF output Y2 and BF output Y6 are crossed. intersects in the client's right seat area A2-3, and the directivities of the BF output Y3 and BF output Y5 intersect in the client's left seat area A2-2.

The area sound extraction unit 204 emphasizes and extracts area sounds from the BF outputs that intersect with each other. That is, the area sound extracting unit 204 extracts the two combinations of the BF outputs Y1 to Y6 generated by the directivity forming unit 203, in which the directivities intersect with each other, using the formula (9) or (10) shown in the prior art. SS is performed in the same manner as the formula to extract non-target area sounds existing in the direction of the target area. Further, the area sound extracting unit 204 extracts the target area sound by performing SS on the extracted noise from the output of each BF by the same method as the formula (11) or (12).

Specifically, the area sound extraction unit 204 performs area sound collection processing with the responder area A1 as the target area based on the BF output Y1 and the BF output Y4 (hereinafter referred to as "responder area sound Z1"). ) is acquired, and the result of area sound collection processing with the center area A2-1 of the counselor's seat as the target area based on the BF output Y2 and BF output Y5 (hereinafter referred to as "center area sound Z2 of the counselor's seat") ) is obtained, and based on the BF output Y2 and BF output Y6, area sound collection processing is performed with the counselor's right seat area A2-3 as the target area (hereinafter referred to as "counselor's right seat area sound Z4" ) is obtained, and based on the BF output Y3 and BF output Y5, area sound collection processing is performed with the client's left seat area A2-2 as the target area (hereinafter referred to as "the client's left seat area sound Z3"). call).

The area sound extracting unit 204 extracts the BF output of each microphone array in the same manner as the "expressions (5) and (6)" or the "expressions (7) and (8)" when extracting the area sound. Although it is necessary to estimate the ratio of the amplitude spectrum of the target area sound contained in , and use it as a correction coefficient, it is omitted in this embodiment for the sake of simplicity of configuration and description.

By the way, the three areas (A2-1 to A2-3) on the consulter's side do not always contain the target sound (speech uttered by the consulter). As described above, when the consulter sits alone in the center, it is also possible that even one consulter sits to the left and right instead of sitting in the center. In this case, only one of the three sound collection areas (A2-1 to A2-3) on the consulter side becomes the target sound area. When two persons visit the place, they are seated separately on the left and right sides, so both the left side seat area A2-2 and the right side seat area A2-3 of the counselor serve as target sound areas.

The sound pickup device 200 determines whether or not there is a target sound in each of the three sound pickup areas (A2-1 to A2-3) from the area sound pickup results for the consulter, and Dialog speech to be extracted and recorded is appropriately selected according to the presence of target area sounds in the sound areas (A2-1 to A2-3).

In the area sound detection unit 206 of this embodiment, the area sound detection unit 206 detects the target area sound (voice spoken by the client) for each of the three sound pickup areas (A2-1 to A2-3) on the client side. It shall be determined whether there exists That is, the area sound detection unit 206 determines whether or not the target area sound (the voice uttered by the client) exists in each of the sound collection areas A2-1 to A2-3. Various methods can be applied in the area sound detection unit 206 as a method for determining the presence or absence of the target area sound in each sound pickup area. For example, in the area sound detection unit 206, a method using the amplitude spectrum ratio between the area sound pickup output and the input sound, or a method using the coherence between the BF outputs when performing area sound pickup can be applied. In the area sound detection unit 206 of the embodiment, a method using an amplitude spectrum ratio is applied.

Amplitude spectrum ratio calculation section 205 converts input signals (X1, X3, X4, and X5, X7, and X8) related to the three area sound extractions from time/frequency conversion section 202 to area sound extraction section 204. acquires the area sound outputs Z2, Z3, and Z4 on the side of the consulter, and calculates the amplitude spectrum ratio. For example, the amplitude spectrum ratio calculator 205 calculates the amplitude spectrum ratios of the area sound outputs Z2, Z3, Z4 and the input signal for each frequency using the following formulas (13), (14), and (15). . Further, amplitude spectrum ratio calculation section 205 adds the amplitude spectrum ratios of all frequencies using the following formulas (16), (17), and (18) to obtain amplitude spectrum ratio added values U2, U3, Ask for U4.

Here, in formulas (13), (14), and (15), Win ₂ is the amplitude spectrum of the input signal of the microphone array used to calculate the area sound output Z2, and Win ₃ is the calculation of the area sound output Z3. _Win4 is the amplitude spectrum of the input signal of the microphone array used for calculating the area sound output Z4. Therefore, amplitude spectrum ratio calculation section 205 calculates the amplitude spectra of microphone arrays MA2, MA3, and MA6 used as main (references) in the calculation of area sound outputs Z2, Z3, and Z4, respectively, as Win ₂ , Win ₃ , and Win ₄ . shall be filled. In the amplitude spectrum ratio calculation section 205, as the amplitude spectrum of each microphone array, the amplitude spectrum of the input signal of one of the two microphones constituting each microphone array may be applied, or the input signal of the two microphones may be used. may be applied.

U2 obtained by the processing using equation (16) is the amplitude obtained by adding the amplitude spectrum ratio R _2i (where i is the frequency) of each frequency of the area sound output Z2 in the band from the lower limit j to the upper limit k of the frequency. Spectral ratio addition value. U3, which is performed using the processing of equation (17), is the sum of the amplitude spectrum ratios _R3i (where i is the frequency) of each frequency of the area sound output Z3 in the band from the lower limit j to the upper limit k of the frequency. be. U4 obtained by processing using equation (18) is obtained by summing the amplitude spectrum ratio R _4i (where i is the frequency) of each frequency of the area sound output Z4 in the band from the lower limit j to the upper limit k of the frequency. be. Here, the frequency band to be calculated in amplitude spectrum ratio calculation section 205 may be restricted. For example, the amplitude spectrum ratio calculating section 205 may perform the above calculation by limiting the frequency to be calculated from 100 Hz to 6 kHz, which sufficiently contains speech information.

Then, area sound detection section 206 compares the amplitude spectrum ratio addition value calculated by amplitude spectrum ratio calculation section 205 with a preset threshold value, determines whether or not area sound exists, and uses the determination result as an area sound. It is supplied to the selection unit 207 . In the following description, the determination results of the target area sound detection for each of the sound collection areas A2-1 to A2-3 are referred to as statuses S1 to S3 (area sound presence statuses). S1 to S3 are each indicated by a value of either "True" indicating that there is a target area sound or "False" indicating that there is no target area sound (target area sound not detected).

The area sound selection unit 207 uses the statuses S2, S3, and S4 to output area sounds Z2, Z3, and Z4 in accordance with the following procedure, except for the responder side, which has only one sound pickup area and has no room for selection. appropriate selection of the dialog voice of the consulter to be recorded (target area sound of the consulter). Specifically, the area sound selection unit 207 selects the area sound outputs Z2, Z3, and Z4 according to the pattern (combination) of the statuses S2, S3, and S4. (also called “personal voice”). The area sound output pattern selected by the area sound selection unit 207 will be described below for each pattern of statuses S2, S3, and S4 (hereinafter referred to as "status pattern").

[First status pattern]
When a pattern in which all of S2 (middle), S3 (left), and S4 (right) are false (no area sound exists) (hereinafter referred to as “first status pattern”), the area sound selection unit 207 does not select any area sound output, and outputs silence as the counselor's voice. In the case of the first status pattern, the area sound selection unit 207 may perform processing not to output the voice signal of the consulter.

[Second status pattern]
When any one of S2 (middle), S3 (left), and S4 (right) is true (area sound exists) pattern (hereinafter referred to as "second status pattern") occurs, area sound selection The unit 207 recognizes that one consulter has spoken, and outputs the area sound output of the area where the area sound exists (the area whose status is true) as the consulter's voice.

[Third status pattern]
When a pattern in which both S3 (left) and S4 (right) are true (hereinafter referred to as a “third status pattern”) occurs, the area sound selection unit 207 selects the left and right areas (A2-2, A2- Since area sounds exist in each of 3), it is determined that there are two consulters on the consulter's side and that the two consulters spoke at the same time, and both area sound outputs Z3 and Z4 are the consulter's voices. Select and output as

[Fourth status pattern]
If a pattern in which both "S2 (middle), S3 (left)" or "S2 (middle), S4 (right)" are true (hereinafter referred to as "fourth status pattern") occurs, The center area A2-1 and the consulter's left seat area A2-2, and the consulter's seat central area A2-1 and the consulter's right seat area A2-3 are close to each other and overlap each other, so depending on the position of the consulter It is highly probable that the voice of one consulter was detected redundantly in two areas. When the fourth status pattern occurs, the area sound selection unit 207 evaluates the output level of any one or both of the two area sound outputs that are true, and selects the larger one as the counselor's voice. output as

However, as described above, since the sound pickup areas A2-1 to A2-3 originally have overlapping areas, even if the fourth status pattern occurs, there are two consulters on the consulter's side. It is also possible that the left or right caller is closer to the center and detected in the central area.

Therefore, when the fourth status pattern occurs, the area sound selection unit 207 selects two area sound outputs ("Z2, Z3" or “Z2, Z4”) is determined for similarity. If the similarity between two area sound outputs that are true at the same time is high, it can be estimated that the two area sound outputs indicate that the same sound has been redundantly detected in two areas. Therefore, when the similarity between two area sound outputs that are simultaneously true is high (for example, if the degree of similarity is equal to or greater than a predetermined threshold value), the area sound selection unit 207 follows the above rule and selects two area sound outputs that are simultaneously true. It is assumed that one of the two area sound outputs is selected. On the other hand, when the similarity between the two area sound outputs that are both true at the same time is low (for example, when the similarity is lower than a predetermined threshold), the area sound selection unit 207 detects the center area A2-1 of the counselor's seat. It can be assumed that the area sound that is in the center is the utterance of the second speaker who is closer to the center. Therefore, when the similarity between the two area sound outputs that are simultaneously set to true is low, the area sound selection unit 207 selects the two area sound outputs that are set to true at the same time as separate counselor voices (the two areas are individually to be selected as the consultant's voice).

The method of calculating the degree of similarity in the area sound selection unit 207 is not limited, and various calculation methods such as a method of calculating based on the degree of correlation between two signals and a method of calculating based on the coherence of two signals are applied. can do.

[Fifth status pattern]
When a pattern in which all of S2 (middle), S3 (left), and S4 (right) are true (hereinafter referred to as "fourth status pattern") occurs, the left and right sound collection areas (A2- 2, A2-3), one of the two speakers is closer to the center, and there are two sound pickup areas (A2-1, A2-2 combination area, or A2-1, A2-3) It can be estimated that there is a high possibility of duplicate detection in the combination area). Therefore, when the fourth status pattern occurs, the area sound selection unit 207 selects and selects two area sound outputs Z3 and Z4 as the counselor's voices in order to increase the degree of separation between the voices of the two counselors. shall be output.

As described above, the area sound selection unit 207 selects the dialogue voice of the consulter to be recorded from the area sound outputs Z2, Z3, and Z4 according to the status pattern. Then, the area sound selection unit 207 selects the dialogue voice of the consulter (area sound output or silence selected from area sound outputs Z2, Z3, Z4) and the dialogue voice of the responder (responder area sound Z1). , to the frequency/time converter 208 .

Each area sound output supplied from the area sound selection unit 207 as dialogue voice is converted into a time signal by the frequency/time conversion unit 208 and supplied to the recording device 300 .

Next, the operation of the recording device 300 will be described.

In the recorder 300 , when the dialogue voice (the area sound output of the responder and the consulter) is supplied from the sound collection device 200 , it is supplied to the dialogue recorder 301 .

The dialogue recorder 301 supplies the supplied dialogue voice data to the dialogue accumulator 302 for accumulation and storage.

As described above, in this embodiment, the dialogue voice acquired by the sound collecting device 200 is supplied to the recording device 300 and recorded/utilized.

(A-3) Effects of First Embodiment According to the first embodiment, the following effects can be obtained.

In the sound collecting device of the first embodiment, when collecting the voices spoken by the responder and the consultant, it is possible to determine whether there are one or two consultants, or where the consultant is seated. Appropriately separates the voice uttered by each consulter according to the pattern of area sound detection results (statuses S2, S3, S4) for the three sound collection areas A2-1 to A2-3, regardless of changes in the situation.・It can pick up sound.

(B) Second Embodiment A second embodiment of the sound collecting device, sound collecting program, sound collecting method, and sound collecting system according to the present invention will be described in detail below with reference to the drawings. In this embodiment, an example in which the sound collection system of the present invention is applied to a face-to-face recording system will be described.

(B-1) Configuration of Second Embodiment FIG. 10 is a block diagram showing the overall configuration of a face-to-face recording system 1A according to the second embodiment. In FIG. 10, the same or corresponding reference numerals are given to the same or corresponding portions as in FIG. 1 described above. The second embodiment will be described below, focusing on differences from the second embodiment.

A face-to-face recording system 1A of the second embodiment differs from the second embodiment in that the microphone array section 100 and the sound collection device 200 are replaced with a microphone array section 100A and a sound collection device 200A, respectively.

Also, the microphone array section 100A of the second embodiment differs from the second embodiment in that the microphone array units MAU1 and MAU2 are replaced with microphone array units MAU1A and MAU2A, respectively. The microphone array unit MAU1A has mc1 to mc4 as in the first embodiment. Also, the microphone array unit MAU2A has mc5 to mc8 as in the first embodiment. However, in the second embodiment, the placement positions of the microphones forming each microphone array unit are different.

Furthermore, in the sound collection device 200A of the second embodiment, the area sound selection unit 207 and the frequency/time conversion unit 208 are replaced with the area sound selection unit 207A and the frequency/time conversion unit 208A, and the area sound addition unit 209 is It differs from the first embodiment in that it is added.

Also, the hardware configuration of the sound collecting device A in the second embodiment can be shown using FIG. 6, as in the first embodiment. Symbols in parentheses in FIG. 6 are symbols used only in the second embodiment.

Here, in the face-to-face recording system 1A of the second embodiment, a sound collection area to be sound-collected will be described with reference to FIG.

FIG. 8 is a diagram showing a sound pickup area to be a sound pickup target and its surrounding environment in the face-to-face recording system 1A of the second embodiment.

In the first embodiment, sound pickup areas (A2-1 to A1-3) as three sub-areas are set in the consulter area A2, but in the second embodiment, the microphone array units MAU1A and MAU2A By adjusting the arrangement, as shown in FIG. 8, two sound pickup areas (sub-areas) are used to cover the case of one person seeking advice and the case of two persons seeking advice. In the second embodiment, in the counselor area A2, an area on the right side as viewed from the counselor side (hereinafter referred to as "counselor right seat area A2-4 ”) and an area on the left side as seen from the client side (hereinafter referred to as “the client’s left seat area A2-5”). In this embodiment, as shown in FIG. 8, there is an overlapping area around the line CL1 between the counselor's right seat area A2-4 and the counselor's left seat area A2-5.

Next, the internal configuration of the microphone array section 100A will be described.

FIG. 9 is a diagram showing an example of arrangement configuration of the microphone array units MAU1A and MAU2A.

FIG. 9 is a view when the area where the microphone array units MAU1A and MAU2A are arranged is viewed from above.

As shown in FIG. 9, in the microphone array section 100A of the second embodiment, the positions of arranging the microphone array units MAU1A and MAU2A are the same as in the first embodiment, so detailed description thereof will be omitted.

In the microphone array unit MAU1A and the microphone array unit MAU2A, the directivity Y2 and the directivity Y6 are directed slightly toward the center (in the direction of the line CL1) compared to the first embodiment (see FIG. 3 described above). It is possible to set two sound pickup areas (a client's right seat area A2-4 and a client's left seat area A2-5) having overlapping areas. Thus, in the second embodiment, the dialogue voice of the counselor seated in the right side seat area A2-4 and the voice dialogue of the counselor seated in the left side seat area A2-5 are collected separately. be able to. In addition, in the sound pickup device 200A of the second embodiment, there is only one person and the central position on the side of the person to be consulted (the area where the right side seat area A2-4 of the person to consult and the left side seat area A2-5 of the person to consult overlap) ), the conversation of the subject is either in one of the two overlapping sound pickup areas on the subject's side (the subject's right seating area A2-4, the subject's left seating area A2-5), or Sound can be picked up in both areas.

Next, the configuration of each of the microphone array units MAU1A and MAU2A in the second embodiment will be explained using FIG.

As shown in FIG. 10(a), the microphone array unit MAU1A installed on the left side as seen from the caller side is arranged on a line segment substantially orthogonal to the direction of the attendant's seat as seen from the microphone mc1 with the microphone mc1 as an end point. a microphone mc3 arranged on a line segment substantially perpendicular to the direction of the right side seat area A2-4 of the client viewed from the microphone mc1 with the microphone mc1 as an end point; and a microphone mc4 arranged on a line segment substantially perpendicular to the left seat area A2-5 of the client.

As shown in FIG. 10(b), the microphone array unit MAU2A installed on the right side as viewed from the caller side is arranged on a line segment substantially orthogonal to the direction of the respondent's seat as viewed from the microphone mc5 with the microphone mc5 as an end point. a microphone mc7 arranged on a line segment substantially orthogonal to the direction of the counselor's left seat area A2-5 as viewed from the microphone mc5 with the microphone mc5 as an end point; and a microphone mc8 arranged on a line segment substantially perpendicular to the direction of the seat area A2-4.

FIG. 11 is a diagram showing combinations of microphones used in the microphone array formed by each microphone array unit of the microphone array section 100A.

As shown in FIG. 11, the combination in which the eight microphones mc1 to mc8 constituting the microphone array unit MAU1A and the microphone array unit MAU2A form the microphone arrays MA1 to MA6 is the first embodiment (see FIG. 5 above). , so detailed description is omitted.

(B-2) Operation of Second Embodiment Next, the operation of the face-to-face recording system 1A of the second embodiment having the configuration as described above will be described.

The operation of the face-to-face recording system 1A will be described below, focusing on differences from the first embodiment.

The signal input unit 201 converts the acoustic signals picked up by the eight microphones mc1 to mc8 from analog signals to digital signals (x1 to x8) and sends them to the time/frequency conversion unit 202 . The frequency conversion unit 202 converts the microphone signal from the time domain to frequency domain signals (X1 to X8).

Directivity forming section 203 forms directivity by BF using the frequency-converted input signals (X1 to X8) of each microphone.

Here, the BF outputs from the microphone arrays MA1 to MA6 are respectively Y1, Y2, Y3, Y4, Y5, and Y6. In this embodiment, as shown in FIGS. 9 to 11, the directivity of Y1 is directed toward the respondent area A1, and the directivity of Y2 is directed toward the counselor's right seat area A2-4. The directivity of Y3 is directed to the client's left seat area A2-5. Further, in this embodiment, the directivity of Y4 is directed toward the respondent, the directivity of Y5 is directed toward the counselor's left seat area A2-5, and the directivity of Y6 is directed toward the counselor. It faces the right seat area A2-4.

The area sound extraction unit 204 extracts the target area sound from two combinations of the BF outputs Y1 to Y6 generated by the directivity forming unit 203, the directivity of which intersects with each other. In this embodiment, as shown in FIGS. 3 and 5, the BF outputs Y1 and Y4 intersect in the correspondent area A1, the BF outputs Y2 and Y6 intersect in the counselor's right seat area A2-4, and the BF output Y3 and Y5 intersect in the A2-5 area on the left side of the client. The area sound extraction unit 204 extracts the area sound pickup result of the correspondent area A1 (correspondent area sound Z1) and the area sound pickup result of the counselor's right seat area A2-4 (consultant's right seat area sound Z4), and the area sound pickup result of the counselor's left seat area A2-5 (consultant's left seat area sound Z3) are emphasized and extracted.

Amplitude spectrum ratio calculation section 205 obtains an amplitude spectrum ratio addition value for each of the two sound collection areas (A2-4, A2-5) on the client's side by the same method as in the first embodiment. In this embodiment, the amplitude spectrum ratio added value of the client's right seat area A2-4 is represented as "U2", and the amplitude spectrum ratio added value of the client's left seat area A2-5 is represented as "U3".

The area sound detection unit 206A compares the amplitude spectrum ratio added values U2 and U3 calculated by the amplitude spectrum ratio calculation unit 205 with preset threshold values, respectively, and determines two sound collection areas (A2-4, A2- For each of 5), it is determined whether an area sound exists or not. In this embodiment, the determination result for the client's right seat area A2-4 is represented as "S2", and the determination result for the client's left seat area A2-5 is represented as "S3".

Next, a method of selecting two area sound pickup results (Z2, Z3) on the client's side in the area sound selection unit 207A will be described.

The area sound selection unit 207A uses the statuses S2 and S3 to record from the area sound outputs Z2 and Z3 in accordance with the following procedure, except for the responder side who has only one sound pickup area and has no room for selection. To appropriately select dialogue voices on the side of a consulter (target area sounds on the side of the consulter). Specifically, the area sound selection unit 207A selects the dialogue voice of the counselor (counselor voice) to be recorded from the area sound outputs Z2 and Z3 according to the status patterns of the statuses S2 and S3. The area sound output pattern selected by the area sound selection unit 207A will be described below for each of the status patterns of statuses S2 and S3.

[Sixth status pattern]
When both S2 (right) and S3 (left) are false (there is no area sound) pattern (hereinafter referred to as “sixth status pattern”), the area sound selection unit 207A selects which area sound No output is selected, and silence is output as the counselor's voice. Note that, in the case of the sixth status pattern, the area sound selection unit 207A may perform a process of not outputting the counselor's voice.

[Seventh status pattern]
When any one of S2 (right) and S3 (left) is true (an area sound exists) pattern (hereinafter referred to as "seventh status pattern") occurs, the area sound selection unit 207A It recognizes that one person has spoken, and outputs the area sound output of the area where the area sound exists (the area whose status is true) as the counselor's side voice.

[Eighth status pattern]
When a pattern in which both S2 (right) and S3 (left) are true (hereinafter referred to as the “eighth status pattern”) occurs, two counselors sitting on the left and right (counselor left seat area A2-5 A case where the counselor seated in the 2nd floor and the counselor seated in the counselor's right seat area A2-4 speak at the same time, and the sound is picked up in each of the left and right areas (A2-4, A2-5) (hereinafter referred to as "first case”), and a case in which the utterances of a single person sitting in the center were picked up simultaneously in two areas (A2-4, A2-5) on the left and right (hereinafter referred to as the “second case”). ) can be considered.

When the eighth status pattern occurs, the area sound selection unit 207A determines the degree of similarity between the two area sound outputs Z2 and Z3 in order to determine which of the above two cases corresponds. When the similarity between the two area sound outputs Z2 and Z3 is low (for example, when the similarity is lower than a predetermined threshold value), the area sound selection unit 207 determines that the first case has occurred, and selects the two area sounds. When the similarity between the outputs Z2 and Z3 is high (for example, when the similarity is equal to or greater than a predetermined threshold), it is determined that the second case has occurred.

Note that the similarity calculation method in the area sound selection unit 207A is not limited, and the same processing as in the first embodiment can be applied.

Then, when determining that the first case has occurred, the area sound selection unit 207A supplies the two left and right area sound outputs Z2 and Z3 to the frequency/time conversion unit 208A for recording. On the other hand, when it is determined that the second case has occurred, the area sound selection unit 207A does not provide the central area as in the first embodiment, but provides two left and right areas (A2-4, A2-5). Then, it is considered that the utterance components of one client are dispersedly recorded in two areas (A2-4, A2-5), and the area sound outputs Z2 and Z3 of the two areas are output to the area sound addition unit 209. It instructs to supply the integrated result to the frequency/time conversion unit 208A. In this case, the area sound selection unit 207A does not supply the area sound output of the client side to the frequency/time conversion unit 208A. In this embodiment, in the second case, the area sound adder 209 adds the left and right area sound outputs Z2 and Z3 and supplies the result to the frequency/time converter 208A as the voice of the client.

In the second case, any one of Z2 and Z3 may be supplied to the frequency/time conversion unit 208A without integrating (adding) Z2 and Z3. Only one of Z2 and Z3 having a higher output level may be supplied to the frequency/time converter 208A.

(B-3) Effects of Second Embodiment According to the second embodiment, the following effects can be obtained.

In the sound collection device 200A of the second embodiment, when collecting the voices spoken by the responder and the consultant, it is possible to determine whether there are one or two consultants, or where the consultant is seated. Regardless of the situation change, according to the pattern of the area sound detection results (statuses S2, S3) for the two sound collection areas A2-4, A2-5, the voice uttered by each consulter is appropriately separated and separated. It can pick up sound.

(C) Other Embodiments The present invention is not limited to the above-described embodiments, and modified embodiments such as those illustrated below can be exemplified.

(C-1) In each of the above embodiments, two microphone array units are used to pick up sound in the responder area and the consulter area. can be separately picked up, the specific number of microphone array units and the configuration of each microphone array unit are not limited. For example, the number of microphone array units may be three or more.

(C-2) In each of the above embodiments, cases where the number of sub-areas in the consulter's area A2 are three and two are shown, but the number of sub-areas in the consulter's area A2 is not limited. For example, four sub-areas may be provided in the consulter area A2 so that sound can be picked up separately.

(C-3) In each of the above embodiments, only the sound pickup area on the side of the consulter is divided into a plurality of sub-areas to pick up the sound, but there are cases where the responder also consults with an expert in the field. Alternatively, the sound pickup area on the responder side may be divided into a plurality of sub-areas for sound pickup.

1, 1A... face-to-face recording system, 100, 100A... microphone array unit, MAU1, MAU1A, MAU2, MAU2A... microphone array unit, MA1 to MA6... microphone array, mc1 to mc8... microphone, 200, 200A... sound collection device, 201 Signal input unit 202 Time/frequency conversion unit 203 Directivity formation unit 204 Area sound extraction unit 205 Amplitude spectrum ratio calculation unit 206, 206A Area sound detection unit 207, 207A Area sound Selection unit 208, 208A Frequency/time conversion unit 209 Area sound addition unit 300 Recording device 301 Dialogue recording unit 302 Dialogue storage unit 500 Computer 501 Processor 502 Primary storage unit , 503 . . . secondary storage unit.

Claims (15)

A first speaker area in which one or more speakers are located, and a plurality of subs in a second speaker area in which one or more speakers interacting with the speakers in the first speaker area. Obtaining a beamformer output of each of the microphone arrays based on an input signal from a plurality of microphone array units including a plurality of microphone arrays having a plurality of microphones capable of directivity in an area, and obtained beams area sound extraction means for performing area sound pickup processing on the first speaker area and each sub-area using a format output;
area sound detection means for performing area sound detection processing for detecting whether or not a sub-area sound is generated for each sub-area;
Area sound determination processing for determining the contents of the sound pickup result of the second speaker area according to the combination pattern of the determination results of the area sound detection processing for each sub-area, and a signal corresponding to the area sound determination processing. as a sound pickup result of the second speaker area;
and output means for outputting a result of area sound pickup processing of the first speaker area and a result of area sound pickup process of the second speaker area acquired by the area sound processing means. sound pickup device. The area sound extracting means includes a first sub-area in the second speaker area and a left side of the first sub-area when the first speaker area is viewed from the first sub-area. and a third sub-area adjacent to the right side of the first sub-area when the first speaker area is viewed from the first sub-area. 2. The sound collecting device according to claim 1, wherein processing is performed. The area sound processing means determines that, as a result of area sound detection processing, no sub-area sound is generated in all of the first sub-area, the second sub-area, and the third sub-area. 3. The sound collecting device according to claim 2, wherein, when the second speaker area is in the second speaker area, a silent signal is output. The area sound processing means produces a sub-area sound for only one of the first sub-area, the second sub-area, and the third sub-area as a result of area sound detection processing. If so, only one sub-area in which the sub-area sound was generated is selected as a sound pickup target, and the result of the area sound pickup processing of the selected sub-area is used as the sound pickup result of the second speaker area. 4. The sound collecting device according to claim 2, which outputs. When the area sound detection processing results in that the sub-area sound is generated only in the second sub-area and the third sub-area, the area sound processing means detects two sub-areas in which the sub-area sound was generated. is selected as a sound pickup target, and a result of area sound pickup processing of the selected sub-area is output as the sound pickup result of the second speaker area. Acquisition device as described. The area sound processing means performs, as a result of area sound detection processing, sub-area sounds occurring only in two sub-areas of the first sub-area and the second sub-area, or When sub-area sounds are generated only in two sub-areas, i.e., the sub-area and the third sub-area, the degree of similarity is calculated for the results of the area sound collection processing of the two sub-areas in which the sub-area sounds were generated. If the degree of similarity is equal to or higher than a predetermined degree, either one of the two sub-areas in which the sub-area sound was generated or only the one with the higher output level is selected as a sound pickup target, and the degree of similarity is less than the predetermined degree. , both of the two sub-areas where the sub-area sound was generated are selected as sound collection targets, and the result of the area sound collection processing of the selected sub-area is used as the sound collection result of the second speaker area. 6. The sound collecting device according to any one of claims 2 to 5, characterized in that it outputs . The area sound processing means determines that, as a result of area sound detection processing, sub-area sounds are generated in all of the first sub-area, the second sub-area, and the third sub-area. In this case, only the second sub-area and the third sub-area are selected as sound pickup targets, and the result of the area sound pickup processing of the selected sub-area is used as the sound pickup result of the second speaker area. 7. The sound collecting device according to any one of claims 2 to 6, which outputs. The area sound extracting means includes a first sub-area in the second speaker area and a left side of the first sub-area when the first speaker area is viewed from the first sub-area. 2. The sound collecting device according to claim 1, wherein area sound collecting processing is performed on a second sub-area adjacent to the right side. When the result of the area sound detection processing is that the sub-area sound is not generated in both the first sub-area and the second sub-area, the area sound processing means performs the second process. 9. The sound collecting device according to claim 8, wherein a silent signal is output as a result of sound collection in the person's area. The area sound processing means, if the result of the area sound detection processing is that the sub-area sound is generated only in either the first sub-area or the second sub-area, the sub-area sound is generated. 10. The method according to claim 8, wherein only the selected sub-area is selected as a sound pickup target, and a result of area sound pickup processing of the selected sub-area is output as the sound pickup result of the second speaker area. Acquisition device as described. The area sound processing means performs area sound collection processing for two sub-areas when sub-area sounds are generated in both the first sub-area and the second sub-area as a result of the area sound detection processing. A degree of similarity is calculated for the result, and if the degree of similarity is equal to or higher than a predetermined value, a signal obtained by integrating the results of the area sound pickup processing of the first sub-area and the second sub-area, or the first sub-area. and any one of the results of the area sound pickup processing of the second sub-area or only the one with the higher output level, and if the degree of similarity is less than a predetermined value, the first sub-area 11. The sound collecting device according to any one of claims 8 to 10, wherein the output is for both the second sub-area and the second sub-area. the computer,
A first speaker area in which one or more speakers are located, and a plurality of subs in a second speaker area in which one or more speakers interacting with the speakers in the first speaker area. obtaining a beamformer output of each of the microphone arrays based on input signals from a plurality of microphone arrays capable of directivity toward an area; using the obtained beamformer output, the first speaker; area sound extraction means for performing area sound pickup processing on the area and each sub-area;
area sound detection means for performing area sound detection processing for detecting whether or not a sub-area sound is generated for each sub-area;
Area sound determination processing for determining the contents of the sound pickup result of the second speaker area according to the combination pattern of the determination results of the area sound detection processing for each sub-area, and a signal corresponding to the area sound determination processing. as a sound pickup result of the second speaker area;
functioning as output means for outputting a result of area sound pickup processing of the first speaker area and a result of area sound pickup processing of the second speaker area acquired by the area sound processing means; Featured sound collection program. In the sound collection method performed by the sound collection device,
having area sound extraction means, area sound detection means, area sound processing means and output means,
The area sound extraction means comprises a first speaker area where one or more speakers are located and a second speaker area where one or more speakers who interact with the speakers in the first speaker area are located. obtaining a beamformer output of each of the microphone arrays based on input signals from a plurality of microphone arrays capable of directing directivity to a plurality of subareas within a subject area, and using the obtained beamformer output , performing area sound pickup processing on the first speaker area and each sub-area,
The area sound detection means performs area sound detection processing for detecting whether or not a sub-area sound is generated for each sub-area,
The area sound processing means performs area sound judgment processing for judging the content of the sound pickup result of the second speaker area according to a combination pattern of the judgment results of the area sound detection processing for each sub-area. acquiring a signal corresponding to the sound determination process as a sound pickup result of the second speaker area;
The output means outputs the result of the area sound pickup process of the first speaker area and the result of the area sound pickup process of the second speaker area acquired by the area sound processing means. sound pickup method. A first speaker area in which one or more speakers are located, and a plurality of subs in a second speaker area in which one or more speakers interacting with the speakers in the first speaker area. A sound collection system comprising a plurality of microphone array units capable of directing directivity to an area, and a sound collection device that performs area sound collection processing for the first speaker area and the second speaker area, A sound collecting system, wherein the sound collecting device according to any one of claims 1 to 11 is applied as a sound collecting device. 15. The sound collection system according to claim 14, further comprising a recording device for recording a result of area sound collection processing by said sound collection device.

Images