JP2010026361A - Speech collection method, system and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately collect speech of only a specified speaker such as a sales person in counter selling or the like. <P>SOLUTION: A speech collection system 10 extracts and collects target speech which is a target in a plurality of pieces of speech in which coming directions are different from each other. The system includes a microphone array 11 including at least first and second microphones 11a and 11b, in which the first and second microphones are arranged by separating them with a predetermined distance. Discrete Fourier transform is performed on each signal of speech received by the first and second microphones, and a plurality of cross spectrum power (CSP) coefficients related to the coming direction of speech are calculated, and a plurality of speech signals are detected from the plurality of CSP coefficients. Then, a speech direction index defined according to an angle between a line for connecting the first and second microphones and the coming direction, is detected from the plurality of calculated CSP coefficients, and the signal of the target speech is extracted from the plurality of speech signals, which are detected from the detected speech direction index. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a sound collection method, system, and program for collecting specific sound. In particular, the present invention relates to a voice collection method, system, and program for collecting only a salesperson's voice in face-to-face sales.

  In recent years, corporations may lose the trust (credit) of consumers or business partners due to illegal or anti-social acts. In addition, it may have a significant impact on business continuity. For this reason, establishment of a so-called compliance system has become an urgent issue for companies. For example, in the financial services industry, sales activities of sales staff are monitored as part of efforts to strengthen compliance. For example, sales activities by telephone are handled by telephone sales (contents of calls). Is stored on a server, etc., and a system for checking at random is incorporated. There is also an attempt to automatically detect inappropriate responses of salespersons by using both speech recognition technology and natural language processing technology.

  On the other hand, in the so-called face-to-face sales where products are sold at the counter, there is no mechanism for collecting salespersons' customer response records, as in the case of sales by telephone, so the development of the monitoring system is delayed compared to sales by telephone. . At present, a method of reporting sales activities carried out by salespersons in writing (reports) or the like is employed, but not only it takes time to create a report, but also proper reporting may not be performed.

  In the conventional technology, as a measure for face-to-face sales, a method in which a salesperson wearing a close-up microphone records a conversation with a customer is being studied. Are also recorded, so many customers are reluctant to record conversations, which is not always an appropriate technique. For this reason, it is possible to install a (single) directional microphone in a location that is not visible to the customer and collect the salesperson's voice, but the standard microphone has low directivity and the customer's voice is also recorded. Will end up. In order to improve the directivity, when using a super-directive gun microphone or the like, it is not suitable to use a gun microphone for face-to-face sales, considering that the gun microphone is generally expensive and its size is large. .

Therefore, in the prior art, as an attempt to use the audio signal processing technology in combination, two omnidirectional microphones are arranged in a straight line toward the direction of the speaker, and output depends on the sound pressure level to one of the microphones. There is known a microphone device that has means for switching signals and thereby exhibits strong directivity (see Patent Document 1). Further, in the prior art, a technique is known in which a microphone array having a plurality of microphone elements is used to detect a speech section and extract a speech signal (see Patent Document 2).
JP-A-9-149490 JP 2007-86554 A

  However, a microphone array including a method of switching output according to the sound pressure level determination result described in Patent Document 1 and a technique using the difference between the sound and noise components described in Patent Document 2 is used. The conventional technique of using software to create directionality collects the customer's voice when recording in the microphone arrangement, and it is difficult to collect only the salesperson's voice except for the customer's voice in face-to-face sales. .

  The present invention provides a method for installing a microphone array that separates voices of a salesperson and a customer in face-to-face sales, a voice enhancement method for improving voice recognition performance for separated voices, and speaker direction indexing of dialog voices using the same. Provided are a voice collection method, system, and program for accurately collecting voices of only salespersons in face-to-face sales. Furthermore, the present invention provides a method, a system, and a program that do not leave an utterance record of an unacknowledged customer, but reliably leave only a salesperson's voice that needs to be recorded.

  In view of the above problems, the present invention includes the following solutions.

(Use of audio arrival time difference)
The present invention uses a microphone array having two microphone elements arranged at a predetermined distance, and uses a time difference, that is, a time delay, in which sound reaches these microphone elements from a specific sound source. Furthermore, in the present invention, the line segment connecting the two microphone elements included in the microphone array is arranged so as to be substantially parallel to the line segment connecting the customer and the salesperson. For example, as viewed from above, according to the present invention, the microphone array is arranged on a straight line connecting a customer and a salesperson. With such an arrangement, the difference in the arrival time of the voice uttered by the customer or salesperson to each of the two microphone elements can approach the maximum. Therefore, in the present invention, in a situation where a plurality of face-to-face sales booths are lined up, etc., an arrangement that can effectively cut voices from adjacent booths where the difference in arrival time to the microphone element is not necessarily the maximum, and can use the arrival time difference Within this range, changes in attitude and position of salespersons and customers can be allowed. Furthermore, in general, in a microphone array, there is a problem that the sound from the direction that arrives in the same phase (same time delay) cannot be distinguished (mirror image position problem). In the present invention, this problem is caused by the arrangement of the microphone elements. It is possible to avoid it.

(Use of CSP coefficient)
Further, according to the present invention, by detecting a target speaker utterance section based on a CSP (Cross power-Spectrum Phase, whitening cross-correlation) coefficient, it is possible to distinguish between a customer and a salesperson and perform voice recognition individually. At the same time, by using the speaker direction index by the CSP method and the time stamp of the speech recognition result, the recording of the target speaker voice can be simplified and the recording location can be selectively designated. In other words, the present invention is characterized by having an interface for designating a recording speaker and a recording location from the direction index and the speech recognition result.

(Speech enhancement)
Furthermore, the present invention realizes high speech recognition performance by performing gain adjustment, that is, speech enhancement based on the CSP coefficient. In the present invention, gain adjustment processing based on the CSP coefficient is linked to a processing procedure that combines spectral subtraction (abbreviated as SS) processing and flooring processing, which are typical noise removal techniques. Specifically, gain adjustment is performed between the SS process and the flooring process. Through this series of processing, speech enhancement is performed simultaneously with speech separation, and practical speech recognition performance as software processing is realized at low cost.

  A computer device, a digital signal processing device, a digital recording device, or the like having a function of voice signal processing can be used as the means for implementing the voice collecting method according to the present invention. The computer apparatus or the like is arbitrarily capable of performing various steps for the voice collecting method according to the present invention, such as recording a voice signal based on the voices of salespeople and customers, and calculating a CSP coefficient for the recorded voice signal. Can be used.

  The present invention is combined with existing technologies such as a speech collection technology that collects only voiced sections, and a speech signal processing technology that adjusts frequency characteristics or gain of signal processing to improve speech intelligibility and ease of hearing. Such combined techniques are also within the scope of the present invention. Similarly, a voice collecting device including the technique of the present invention, a voice collecting function incorporated in a portable computer device including the technique of the present invention, a voice collecting system for cooperating a plurality of devices including the technique of the present invention, etc. It is included in the technical scope of the present invention. In addition, the technique of the present invention provides the steps for voice collection, FPGA (field programmable gate array), ASIC (application specific integrated circuit), equivalent hardware logic elements, programmable integration. It may be provided as a program form that can be stored in the circuit or a combination thereof, that is, as a program product. Specifically, the salesperson voice collection device according to the present invention can be provided as a form of a custom LSI (large scale integrated circuit) having a data input / output, a data bus, a memory bus, a system bus, and the like. The form of the program product stored in the circuit is also included in the technical scope of the present invention.

  According to the present invention, an audio signal received by the first and second microphones using the microphone array including at least the first and second microphones and the first and second microphones arranged at a predetermined distance. Are respectively subjected to discrete Fourier transform to obtain a plurality of CSP coefficients related to the direction of arrival of the voice, and after detecting a plurality of voice signals from the plurality of CSP coefficients, the first and second signals are obtained from the obtained plurality of voice signals. A speech direction index defined according to an angle formed by a line connecting the two microphones and the direction of arrival is detected, and a target speech signal is extracted from a plurality of detected speech signals based on the detected speech direction index. Therefore, there is an effect that only the target voice can be reliably extracted and collected. Further, the present invention has an effect that it is possible to reliably leave only the voice of the salesperson who needs to be recorded without leaving the utterance record of the customer who does not obtain the consent. Simultaneously with speech separation, subsequent speech recognition performance is enhanced by performing speech enhancement processing including a series of steps of SS processing, gain adjustment processing using CSP coefficients, and flooring processing.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Audio collection system]
FIG. 1 is a diagram schematically illustrating an example of a voice collection system according to an embodiment of the present invention. In FIG. 1, a voice collection system 10 includes a microphone array 11, a target voice extraction device 12, and a customer dialogue recording server 13. The microphone array 11 includes two microphones 11a and 11b, which are commercially available, for example. A possible integral type or a set of stereo microphones may be used. Details of the target speech extraction device 12 will be described later with reference to FIG.

  In the example of FIG. 1, the customer 21, the salesperson 22, the table 14 and the like are viewed from above. The microphone array 11 is arranged so as to be substantially located on a straight line connecting the customer 21 and the salesperson 22 when viewed from above. That is, the microphone array 11 is arranged so that the line segment connecting the microphones 11a and 11b and the line segment connecting the customer 21 and the salesperson 22 are substantially parallel. Thereby, the difference in the arrival time of the voices uttered by the customer or the salesperson to each of the two microphone elements can be maximized. By arranging in this way, in the present invention, in a situation where a plurality of face-to-face sales booths are arranged side by side, it is possible to effectively cut sound from adjacent booths where the difference in arrival time to the microphone element is not necessarily the maximum.

  In the illustrated example, the target speaker utterance section is detected based on the CSP coefficient to distinguish the utterances of the customer and the salesperson. Specifically, CSP coefficients are calculated for the speech signals received by the two microphones, and the target speech signal is extracted by regarding the section where the CSP coefficient is large as the speech section of the target speaker.

  Further, the extracted speech signal is subjected to speech enhancement processing by performing gain adjustment using a CSP coefficient between the SS processing and the flooring processing. This voice enhancement process is a process for improving the voice recognition performance. AFE (ASR Front-end for speech Enhancement, ASR) means automatic voice recognition by combining the target speaker voice extraction by the CSP coefficient and the voice enhancement process. It is referred to as “Automatic Speech Recognition”. In the present embodiment, speech recognition is performed individually for speech signals that have been separated and emphasized using AFE, and, as will be described later, by using the speaker direction index by the CSP method and the time stamp of the speech recognition result, Simplify recording of the speaker's voice signal and selectively specify the recording location.

  As shown in FIG. 1, the microphone array 11 may be arranged so that the microphones 11 a and 11 b are substantially located on a straight line connecting the customer 21 and the salesperson 22 when viewed from above. The microphone array 11 may be placed in the approximate center of the table 14 or may be embedded in the approximate center of the table 14.

  FIG. 2 is a diagram showing the voice arrival direction with respect to the microphone. In FIG. 2, assuming that the microphones 11a and 11b are spaced apart by a distance d, the angle θ formed by the straight line connecting the microphones 11a and 11b and the voice arrival direction is expressed by the following equation (1).

ここで、ｃは音速であり、τはマイクロホン１１ａ及び１１ｂに音声が到来する時間差（到来時間差）を表す。好適には、マイクロホン１１ａ及び１１ｂを結ぶ直線は、マイクロホン１１ａから１１ｂへの方向ベクトルであり、上式においてθ＝０°及びθ＝１８０°は、音声の到達方向と当該方向ベクトルとがそれぞれ平行及び逆平行の状態にあるものとして区別され得る。

Here, c is the speed of sound, and τ represents the time difference (arrival time difference) at which sound arrives at the microphones 11a and 11b. Preferably, the straight line connecting the microphones 11a and 11b is a direction vector from the microphones 11a to 11b. In the above equation, θ = 0 ° and θ = 180 ° indicate that the direction of speech arrival and the direction vector are parallel. And can be distinguished as being in an antiparallel state.

[Speech enhancement processing (AFE)]
Next, in the voice collection system or the like according to the present invention, the CSP coefficient can be calculated and the voice enhancement process can be performed using the CSP coefficient. Specifically, in the speech enhancement process, gain adjustment is performed using the CSP coefficient in the SS process and the flooring process, so that the performance of identifying the salesperson's voice and the performance of speech recognition can be improved. Hereinafter, specific components of the audio processing means and their relation will be exemplified.

  FIG. 3 is a diagram showing a configuration of the target speech extraction device 12 according to an embodiment of the present invention. The target speech extraction device 12 receives speech signals received by the microphones 11a and 11b included in the microphone array 11, and receives discrete Fourier transform processing units 105 and 106, a CSP coefficient calculation unit 110, a group delay array processing unit 120, and noise estimation. Unit 130, SS processing unit 140, gain adjustment processing unit 150, flooring processing unit 160, and the like. The processing of the discrete Fourier transform processing units 105 and 106 is performed in digital audio signal processing, such as appropriately amplifying signals from the two microphones 11a and 11b, dividing the frames into frames having a predetermined time width, and appropriately limiting the frequency band. A complex discrete spectrum can be output from the input signal, including known techniques.

  In the CSP coefficient calculation unit 110 shown in FIG. 3, CSP coefficients are calculated from the complex discrete spectrum. Here, the CSP coefficient is a cross-correlation coefficient between two channel signals calculated in the frequency domain, and is calculated by the following equation (2).

式中、φ（ｉ，Ｔ）は、１番目と２番目のマイクロホン１１ａ及び１１ｂで受けた音声信号から求まるＣＳＰ係数、ｉは音声到来方向（話者方向インデックス）、Ｔはフレーム番号、ｓ_１（ｔ）とｓ_２（ｔ）はそれぞれ時刻ｔに受音した１番目と２番目のマイクロホン１１ａ及び１１ｂの信号である。また，ＤＦＴは離散フーリエ変換を表し、ＩＤＦＴは逆離散フーリエ変換を表している。また、＊は共役複素数を表す。

In the equation, φ (i, T) is a CSP coefficient obtained from the voice signals received by the first and second microphones 11a and 11b, i is the voice arrival direction (speaker direction index), T is the frame number, and s ₁ (T) and s ₂ (t) are the signals of the first and second microphones 11a and 11b received at time t, respectively. DFT represents discrete Fourier transform, and IDFT represents inverse discrete Fourier transform. * Represents a conjugate complex number.

  Next, in the group delay array processing unit 120, signals arriving from the θ direction are received by at least two microphones, in-phased and added, thereby enhancing the signal arriving from the θ direction. Therefore, signals coming from other than the θ direction are not emphasized because they are not in-phase. Therefore, it is possible to form directivity that sensitivity is high in the θ direction and sensitivity is low in other directions.

  Instead of the group delay array processing unit 120, a blind spot can be formed in the direction of noise or reverberation by adaptive array processing. Furthermore, other array processing may be substituted. Further, these array processes can be omitted, that is, passed through, and one of the audio signals received by the two microphones can be used as it is.

ここで、Ｘω（Ｔ）はＳＳ処理前のパワースペクトル，Ｙω（Ｔ）はＳＳ処理後のパワースペクトルすなわち減算後パワースペクトル，Ｕωは雑音のパワースペクトルである。このＵωについては、雑音区間すなわち目的話者の非発話区間で推定されるものであって、事前に推定して固定的に使ってもよく、又は入力された音声信号と同時に逐次推定（更新）してもよく、あるいは一定時間間隔で推定（更新）してもよい。 Next, speech enhancement processing is performed using the CSP coefficient calculated as described above. Specifically, in the speech enhancement process, gain adjustment is performed using the CSP coefficient in the SS process and the flooring process. Typically, SS processing is subtraction processing expressed by the following equation.

Here, Xω (T) is a power spectrum before SS processing, Yω (T) is a power spectrum after SS processing, that is, a power spectrum after subtraction, and Uω is a noise power spectrum. This Uω is estimated in the noise interval, that is, in the non-speech interval of the target speaker, and may be estimated in advance and used in a fixed manner, or sequentially estimated (updated) simultaneously with the input speech signal. Alternatively, it may be estimated (updated) at regular time intervals.

  That is, for example, a signal integrated by array processing for both of the two input signals received by the microphones 11a and 11b, or Xω (T) that is one of the two input signals is input to the noise estimation unit 130. The noise power spectrum Uω is estimated as appropriate. α is a subtraction constant, and can take an arbitrary value such as a value close to 1 (for example, 0.90).

式中、Ｄω（Ｔ）は利得調整後のパワースペクトルである。目的話者が発話していないときはＣＳＰ係数が小さくなるので、到来方向以外からの音声信号のパワースペクトルはこの処理により抑圧されることになる。この式が示すように「利得調整」を行うことができれば、本発明の技術的思想は、何もＣＳＰ係数を利用したものだけに限定されるものではないことが理解できる。 Next, gain adjustment can be appropriately performed as in the following equation. That is, the gain adjustment is performed by multiplying the subtracted spectrum Yω (T) after the SS process by the CSP coefficient.

In the equation, Dω (T) is a power spectrum after gain adjustment. Since the CSP coefficient is small when the target speaker is not speaking, the power spectrum of the voice signal from other than the direction of arrival is suppressed by this processing. If the “gain adjustment” can be performed as shown by this equation, it can be understood that the technical idea of the present invention is not limited to the one using the CSP coefficient.

式中、Ｚω（Ｔ）はＦｌｏｏｒｉｎｇ処理後のパワースペクトル、Ｕωは雑音のパワースペクトルであって、Ｕωとしては、数式３で用いるものと同様のもの、又は雑音推定部１３０の出力等を適宜利用できるが、他の方法で推定した異なったものを利用してもよい。数式５が示すように、Ｕωは条件判断のためだけに用いられることもある。フロアリング係数（Ｆｌｏｏｒｉｎｇ係数）βは定数であり、例えば０（ゼロ）に近い値（例えば、０．１０）等の、当技術分野において好都合な任意の値をとることができる。 Further, a flooring process is performed as in the following equation. That is, the flooring process refers to replacing a small value included in actual data with an appropriate numerical value without using it as it is.

In the equation, Zω (T) is the power spectrum after the flooring process, Uω is the noise power spectrum, and Uω is the same as that used in Equation 3, or the output of the noise estimation unit 130 is used as appropriate. It is possible to use a different one estimated by other methods. As Equation 5 shows, Uω may be used only for condition determination. The flooring coefficient (flooring coefficient) β is a constant, and can take any value convenient in the art, such as a value close to 0 (eg, 0.10).

  Normally, the SS process and the flooring process are used in compliance with this procedure, but it is one point of the present invention that a gain adjustment by a CSP coefficient is introduced between the two processes. The output Zω (T) obtained as described above can be used as a salesperson's voice signal to be stored in a server device or the like, or input to voice recognition means. FIG. 3 shows an example in which one of the sound signals that can be observed using the two microphones 11a and 11b is used for output. However, the present invention is not limited to this, and the sound collection method according to the present invention uses FIG. As will be described later, it is possible to obtain an output for recording or voice recognition or the like for each voice signal received with respect to two voices having different directions to reach the microphone array 11. The output for recording or voice recognition or the like can be used for voice recognition or the like, as will be described later with reference to FIG.

[Speaker Direction Index]
FIG. 4 is a diagram showing an example of the speaker direction index with respect to the position of the microphone. Assuming a direction vector connecting the microphones 11 a and 11 b included in the microphone array 11, the direction in which the voice from the speaker arrives can be distinguished as a range of azimuth angles with respect to the direction vector centering on the microphone array 11. For example, the voice arriving along the direction from the microphone 11a to the microphone 11b is substantially parallel to the direction vector, and the value of the cosine of the azimuth is close to +1 (the region where the speaker direction index shown in FIG. 4 is +7). . Further, for example, the voice arriving along the direction of the microphone 11a from the microphone 11b is nearly antiparallel to the direction vector, and the cosine value of the azimuth is close to −1 (the speaker direction index shown in FIG. 4 is −7). Area). As shown in Equation 1, when the microphone interval d and the sound speed c are given, the arrival time difference τ depends on the angle θ, so the speaker direction index shown in FIG. 4 includes information on the arrival time difference τ.

  There is no difference in arrival time between voices arriving at the microphones 11a and 11b from a direction perpendicular to the microphone array 11, and the speaker direction index in this direction is represented as 0 here. That is, as described above, the angle θ is expressed by Equation 1, and when the number of incoming samples is x and the sampling frequency is f, it is expressed by τ = x / f, so that the sampling frequency is 22050 Hz and the distance between the microphones. If d = 12.5 cm, then x = 0, that is, if the speaker direction index = 0, then if the sound speed is 340 m / s, the angle θ = 90 °.

  In FIG. 4, the speaker direction index +1 (or -1) represents a range in which the sound reaching the microphones 11a and 11b is shifted by one sample (that is, X = 1). Is an angle θ = 82.9 °.

  Similarly, the speaker direction indexes +2 to +7 (or −2 to −7) represent ranges in which sounds reaching the microphones 11a and 11b are shifted by 1 to 7 samples, respectively. In the AFE, the target sound is extracted using a CSP coefficient that takes into account the difference in arrival times of the sounds input to the microphones 11a and 11b. Here, the angle θ = 30.3 ° at x = + 7, and the angle θ = 149.7 ° at x = −7. Therefore, a range of about 30 ° can be allowed as the same voice arrival direction in the linear direction connecting the microphones 11a and 11b. As described above, the present invention is characterized in that changes in attitudes and positions of salespersons and customers can be allowed within the range of arrangements in which the arrival time difference can be used.

  Assuming that there is a target speaker in the direction of the speaker direction index = 0 (for example, the right side), when the target speaker in the speaker direction index = 0 speaks, as described above, the microphones 11a and 11b. There is no time delay in the audio signal received at, and the correlation between both audio signals is high. For this reason, the CSP coefficient φ (0, T) increases.

  On the other hand, for example, when the voice comes from the direction of the speaker direction index = + 4 (for example, the right side in the figure), the voice arrives at the microphone 11b with a delay of 4 samples from the microphone 11a. For this reason, φ (0, T) becomes small (in this case, φ (4, T) becomes large).

  Therefore, when it is desired to extract only the voice coming from the direction of the speaker direction index = 0, the value of φ (0, T) is tracked and the section where φ (0, T) increases is extracted. It will be. However, in AFE, the direction of arrival in the microphones 11a and 11b with the same time difference, that is, the voice coming from the target direction with respect to the axis connecting the microphones 11a and 11b is also received.

  For example, when attention is paid to the speaker direction index = + 4, it is not possible to distinguish between voices coming from the right speaker direction index = + 4 and voices coming from the left speaker direction index = + 4 in the figure. . Therefore, it is necessary to arrange the microphones 11a and 11b so as not to be affected by the mirror image position problem.

  By the way, when a speaker (that is, customer 21 and salesperson 22 here) sits facing each other across the table 14, there is a possibility that the speaker will be displaced laterally (that is, in a wide range in the lateral direction), In addition, the sitting position and posture often change during the conversation. For this reason, it is necessary to be able to pick up a certain range of sounds with respect to the target speaker direction.

  Superdirective microphones are highly effective from the viewpoint of recording only the target speaker's voice signal, but are generally expensive and difficult to deal with variations in speaker position, depending on the seating position. Sound collection performance will change drastically. In addition, the super-directional microphone is large in size and has a sharp directivity in the direction opposite to the target direction. For this reason, the layout relationship between the booth layout and the microphone becomes extremely difficult.

  On the other hand, when a unidirectional microphone is used, the directivity accuracy is not so high, so that ambient ambient sounds and conversations in the adjacent booth are also recorded. Unidirectional microphones are also relatively expensive.

  FIG. 5 is a diagram showing classification by microphone directivity. The omnidirectional microphone shown in FIG. 5 (a) has the same sensitivity in all directions of 360 degrees, and the bidirectional design shown in FIG. 5 (b). The sensitive microphone is sensitive to the front and the opposite side. In addition, the unidirectional microphone shown in FIG. 5C is sensitive to sound only in the front direction. The sharp directional microphone shown in FIG. 5D and the super directional microphone shown in FIG. 5E each have sharper directional characteristics than unidirectional.

  When AFE is used, as shown in FIG. 4, a relatively wide lobe is formed with respect to the axial direction (+7, -7) of the microphone array 11. For example, the salesperson 22 has a speaker direction index = + 7. When the customer 21 is positioned at the customer direction index = −7, the lobes are wide in the axial direction (+7, −7), so that the postures and positions of the customer 21 and the salesperson 22 may be slightly shifted. It is possible to effectively cut the voice that reaches from outside the range.

  When AFE is used, the directivity / omnidirectionality of the microphone is irrelevant, and any directivity microphone can be used. As a result, the cost required for the microphone can be kept low.

[Arrangement of microphone array]
FIG. 6 shows an example of the arrangement of microphone arrays according to an embodiment of the present invention. As described above, since there is a problem of the mirror image position when using AFE, it is necessary to consider the position of the microphone. For example, the position indicated by symbol A in FIG. 6 (the threshold 17 with the adjacent booth 16 or the like). When the microphone array 11 is arranged at the same time, the sound of the adjacent booth 16 may be extracted in the same way.

  For this reason, in this embodiment, the microphone array 11 is installed at a position (for example, on the table 15) indicated by symbol B in FIG. Regarding the installation of the microphone array 11 in the present embodiment, it is difficult to accurately detect the direction of the speaker, but there is no problem in that only the voice of the salesperson 22 is collected. Of course, in an environment where there is no incoming voice from an adjacent booth, for example, an embodiment in which a microphone is arranged at a position indicated by reference symbol A in FIG. 6 and only a portion related to voice enhancement by AFE of the present invention is applied can be assumed. .

[Target voice extraction device]
FIG. 7 is a block diagram showing in detail the target speech extraction device 12 shown in FIG. In FIG. 7, it is assumed that the salesperson 22 and the customer 21 are having a one-to-one dialogue. The target speech extraction device 12 includes a speech segment index detection processing unit 31, a first speech recognition unit 32, a second speech recognition unit 33, an integration selection unit 34, and a recording range extraction unit 35. Each of the audio signals received from the microphones 11a and 11b is input to the index detection processing unit 31.

In FIG. 7, it is assumed that the microphone 11a is located on the salesperson 22 side and the microphone 11b is located on the customer 21 side, and the audio signal S ₁ (t) received by the microphone 11a (L-ch) and the microphone Assume that the audio signal S ₂ (t) received at 11b (R-ch) is input. Here, the input from any of the microphones is sampled at a predetermined sampling frequency by an A / D conversion unit (not shown) and provided as a digital signal to the speech segment index detection processing unit 31. Details of the operation of the speech section index detection processing unit 31 will be described later with reference to FIG.

  Next, the target speech extraction device 12 according to the present invention uses the speech recognition units 32 and 33, and outputs the speech segment index detection processing unit 31 as the separated speech signal and the customer speech and customer speech. A speech recognition operation is appropriately performed on each of the signals, and a recognition result and a time stamp are obtained. Here, the time stamp is time information output by the voice recognition units 32 and 33. The time stamp can be time-series information when integrating the recognition results in a subsequent stage.

  Next, the target speech extraction apparatus 12 according to the present invention can integrate the results of speech recognition using the integration selection unit 34. Specifically, data in which speaker discrimination, speech recognition results, time stamps, and the like are associated with each other can be generated.

  Next, the target speech extraction device 12 according to the present invention uses the recording range extraction unit 35 to generate a speech signal included in a predetermined or specified time region based on information such as a speaker direction index, a speech recognition result, and a time stamp. Can be cut out and stored in a server device or the like as appropriate. In the present invention, by performing voice recognition individually for each salesperson or customer, when the recording portion is designated, the contents of the dialogue between the two can be confirmed. It is also possible to avoid recording unnecessary parts, and resources such as server devices can be used efficiently.

[Processing of Speech Section Index Detection Processing Unit 31]
FIG. 8 is a flowchart for explaining processing in the utterance section index detection processing unit 31. The speech section index detection processing unit 31 acquires a voice signal (step S1) and determines whether the voice signal is an input from the microphone 11a (step S2). If it is an input from the microphone 11a (first microphone), the salesperson digital voice input signal is subjected to, for example, a windowing process using a Hanning window or a Hamming window to obtain a salesperson windowed signal (step). S3). Subsequently, the salesperson windowed signal is converted into a frequency domain signal by a discrete Fourier transform process to be a salesperson frequency domain signal (step S4), and the process proceeds to a process indicated by a broken line in the figure. Similarly, when it is determined in step S2 that the input is from the microphone 11b (second microphone), the windowing process (step S5) and the discrete Fourier transform process (step S5) are similarly performed for the customer digital voice input signal. S6) is performed to obtain a customer frequency domain signal.

  As described above, the speech section index detection processing unit 31 detects the speaker direction index, and based on the salesperson frequency domain signal, the customer frequency domain signal, and the speaker direction index, that is, based on the CSP, A coefficient is calculated (step S7).

  Subsequently, the salesperson side delay sum array processing is performed on the salesperson frequency domain signal and the customer frequency domain signal (step S8), and the salesperson's voice signal is emphasized to obtain a salesperson enhancement signal. Similarly, the customer side delay sum array processing is performed on the salesperson frequency domain signal and the customer frequency domain signal (step S9), and the customer's voice signal is emphasized to obtain a customer emphasized signal.

  Next, the salesperson emphasis signal is subjected to spectrum floor subtraction processing (step S10), noise is removed, and after performing gain adjustment processing (step S11) using the CSP coefficient, flooring processing (step S12) is performed as appropriate. To obtain a salesperson's voice signal.

  Similarly, the customer-enhanced signal is subjected to a flooring process (step S15) after the noise is removed in the spectral subtraction process (step S13) and the gain adjustment process (step S14) is further performed using the CSP coefficient. To obtain the customer's voice signal.

  Further, the utterance section index detection processing unit 31 performs the utterance section detection processing based on the CSP coefficient expressed by the above-described formula 1, and the salesperson side audio signal and the customer side audio signal obtained as described above are used. Are temporarily stored as independent channels (in the speech section detection process, the algorithm based on the target sound extraction method described above is used). Here, as described above, the speaker direction index is detected together with the separation of the target sound, and the separated speech signal and the speaker direction index are associated with each other.

  The utterance section index detection processing unit 31 provides the sales person side speech signal and the speaker direction index of the speech signal to the first speech recognition unit 32 and also to the recording range extraction unit 35. Further, the utterance section index detection processing unit 31 gives the customer-side voice signal and the speaker direction index of the voice signal to the second voice recognition unit 33 and to the recording range extraction unit 35.

  The first voice recognition unit 32 performs voice recognition on the salesperson side voice signal to obtain a recognition result and a time stamp (a salesperson voice recognition result and a salesperson time stamp are obtained). The second voice recognition unit 33 performs voice recognition on the customer-side voice signal to obtain a recognition result and a time stamp (a customer voice recognition result and a customer time stamp are obtained). Here, the time stamp is time information output from the first speech recognition unit 32 and the second speech recognition unit 33, and is used as time-series information when integrating the recognition results.

  The salesperson voice recognition result, salesperson time stamp, customer voice recognition result, and customer time stamp described above are provided to the integration selection unit 34, where these voice recognition results are integrated into the dialogue table shown in Table 1. (Note that this dialog table may be presented to the user in HTML format, for example).

  When a desired voice signal portion is selected as a recording unit from the dialogue table, the integrated selection unit 34 generates a target speaker recording range (that is, a range delimited by a time stamp) and sends it to the recording range extraction unit 35. The recording range extraction unit 35 extracts a voice signal of a corresponding section (range) based on the speaker direction index and the target speaker recording range, and stores it in the customer dialogue recording server 13 as salesperson voice.

  In the present embodiment, as described above, the recording interval is determined using the speaker direction index, the speech recognition result, and the time stamp, and by performing speech recognition for each speaker individually, When specifying the recording part, the recording part can be specified while confirming the content of the dialogue between the two.

  Further, in the present embodiment, recording of unnecessary portions can be avoided. As a result, the disk capacity in the customer interaction recording server 13 can be reduced, which is efficient.

  Here, the types of microphones and AFE were compared from the viewpoint of reducing the customer's voice signal (an evaluation test was performed). In the evaluation experiment, voice signals collected in a simulated face-to-face sales format were used. In the evaluation test, it is assumed that one person who speaks as a salesperson and one customer role sits on both sides of a vertical (salesperson-customer) 100cm table and talks about the contents of the investment trust. .

  The dialogue consists of three cases: a standard position, a position slightly deviated from the standard position to the left and right, and a position that is extremely close to the table. Three sets of each were recorded. A microphone array was configured using two Sony (registered trademark) omnidirectional microphones (Sony ECM-55B), and the microphone array was placed in the center of the salesperson role and the customer role.

  For comparison, a unidirectional microphone (AKG400) was installed in the direction of each speaker, and the voices of both speakers were collected. The distance between the microphones was 12.5 cm for both directivity and non-directivity. In this evaluation test, AFE was performed with an audio signal received by an omnidirectional microphone.

  Here, in order to extract only the sales person's voice signal and not leave the customer's voice signal as a record, the customer's voice signal is regarded as noise and is evaluated by a noise reduction rate (NRR). Went. At this time, the effect was compared based on the degree of reduction of the voice signal of the customer from the reference, based on the voice pressure level of the customer collected by the omnidirectional microphone near the salesperson.

  However, in order to absorb the difference in recording level due to the difference in recording devices, normalization was performed on the computer so that the power of the salesperson's audio signal was the same in each case. The definition of NRR used in this evaluation experiment is as follows.

  Noise Reduction Rate (NRR:%) = Customer utterance sound pressure level [dB] by omnidirectional microphone (reference microphone) −Customer utterance sound pressure level [dB] of directional microphone (or after AFE)

  Normally, the NRR is calculated based on the input / output SNR, but in this evaluation experiment, the power of the audio signal is normalized, so it is formulated as a noise-only difference as defined above. Table 2 shows the experimental results.

  From the experimental results, it can be seen that the omnidirectional microphone picks up all of the voice regardless of the voice arrival direction, and therefore shows a high sound pressure level for the voice of the customer. In addition, it can be seen that the unidirectional microphone has directivity with respect to the front direction, but the directivity is dull so that the customer's voice cannot be cut off so much. This means that it is completely useless for the purpose of recording only the salesperson's voice on the server.

  On the other hand, in the voice collection system according to the present embodiment (use of an omnidirectional microphone), it can be seen that the customer voice is remarkably reduced and the customer voice is effectively suppressed. Note that the sound collection system according to the present embodiment shows a sound pressure level of 19.6 dB. This is because the AFE performs a flooring process shown in Formula 5 for voice recognition, and adds a small amount of noise. Note that this voice has no information that can identify the phoneme (what it is talking about). In the voice collecting system according to the present embodiment, the salesperson's voice is completely detected.

  In the above-described embodiment, the voice is collected from the microphone, and only the salesperson's voice is stored in the customer interaction recording server by the microphone array target voice extraction device, but the customer's voice is stored in the server as necessary. It is also possible to do. In addition, if necessary, three or more microphones may be arranged according to the speaker direction index shown in FIG. 4 to extract the voice of only a desired speaker.

  In the above-described embodiment, the cross-correlation coefficient is used, but other methods for obtaining the correlation coefficient may be used. Then, even if a program that realizes the operation of the above-described voice collection system is operated on a computer, the voice of only a desired speaker can be similarly extracted.

[Example of speech enhancement performance based on the sequence of speech processing steps]
In the voice collection according to the present invention, the target voices are collected in the order of SS processing → gain adjustment by CSP → flooring processing, as shown in the steps of voice processing and their order using FIG. 8 described above. Voice enhancement processing is performed. This order is an important point in speech enhancement for the speech collection method according to the present invention, and the difference in speech enhancement performance due to the difference in processing order will be exemplified below.

  Speech for testing the difference in performance of speech enhancement is collected through the microphone array 11 and processed under the conditions of a sampling frequency of 22 kHz, a frame size of 23 ms, a frame shift of 15 ms, and an FFT size of 512 points, and then used for speech enhancement. The target speech enhancement signal was used. A speech recognition process was further appropriately performed on the obtained target speech enhancement signal.

First, an example in which the speech recognition rate is improved by using speech enhancement according to the present invention will be described. Table 3 shows the command recognition rate when the speech recognition processing is performed only with the SS processing according to the prior art in the speech recording of 50 types of speech commands by four speakers, and the predetermined number according to the present invention. A comparison of command recognition rates when performing speech enhancement based on order, that is, SS processing → gain adjustment by CSP → flooring processing is shown. The command recognition rate can be treated as a speech recognition rate. Therefore, as shown in Table 3, the speech recognition rate can be increased by the speech enhancement according to the present invention.

Next, an example is shown in which the order of the steps of speech enhancement according to the present invention affects the result of speech recognition rate. Table 4 shows the result of comparing the command recognition rates when the speech enhancement processing procedure is changed as a table added to Table 3. The speaker, voice collection conditions, and the like are the same as those in the example shown in Table 3 above. As “Processing Procedure Replacement 1”, voice enhancement is performed in the procedure of SS processing → flooring processing → gain adjustment by CSP, and “ As “Processing Procedure Change 2”, speech enhancement was performed by gain adjustment by CSP → SS processing → Flooring processing. Comparing the speech recognition rates shown as the command recognition rates in Table 4, a significantly high performance was obtained when processing in the order of SS processing → gain adjustment by CSP → flooring processing as a speech enhancement procedure according to the present invention. . Therefore, it is understood that the procedure of processing in this order is important.

  FIG. 9 shows an example of a speech signal in a noise section at various stages of speech enhancement processing according to the present invention. As a reason why the speech enhancement processing procedure according to the present invention skips and exhibits high performance, explanations using schematic diagrams as shown in FIGS. 9A, 9B, 9C, and 9D can be considered. All examples (200) of the noise section (non-speech section of the target speaker) are expressed as amplitude frequency characteristics. FIG. 9A is a schematic diagram showing the power spectrum Xω (T) before performing the spectrum subtraction (SS) process. FIG. 9B is a schematic diagram showing the subtracted power spectrum Yω (T) subjected to the SS process, and noise is reduced by the SS process. FIG. 9C is a schematic diagram showing the power spectrum Dω (T) after gain adjustment by the CSP coefficient, and noise is further reduced by the gain adjustment by the CSP coefficient. FIG. 9D is a schematic diagram showing a recognition power spectrum Zω (T) after performing the flooring process, and the spectrum of the lumpy noise becomes gentle.

  The effects of CSP and Flooring appear in the noise section (non-speaking section of the target speaker). The spectrum of the noise section is flattened by SS processing, and the peaks that pop out in some places are further crushed by applying the CSP coefficient, and further, valleys are filled and smoothed by applying Flooring (as a metaphor A gentle spectral envelope (like snow). As a result, noise is not mistaken as the target speaker's voice. In the speech recognition method according to the prior art, although the target speaker is not speaking, there is a problem that the surrounding noise is mistaken for the target speaker's voice and erroneous recognition is caused. It is considered that the error can be reduced by processing according to the processing procedure of processing → gain adjustment (by CSP coefficient) → flooring processing.

[Example of operation status of portable salesperson voice collection device]
FIG. 10 illustrates an operation state of the portable salesperson voice collection device 60 according to an embodiment of the present invention. The portable salesperson voice collecting apparatus 60 includes microphones 60a and 60b, which constitute a microphone array in the above-described voice collecting method implementing apparatus according to the present invention with reference to FIGS. Furthermore, the portable salesperson voice collection device 60 includes digital signal processing means capable of performing the steps of the voice collection method according to the present invention, and appropriately includes storage means, voice reproduction means, and the like.

  Typically, the portable salesperson voice collection device 60 is fixed to the chest of the salesperson 22 or the like, and when the salesperson 22 faces the customer 21, the portable salesperson voice collection device 60 from the mouth of the salesperson 22. The direction of the voice arrival direction 1 (70) toward the mobile phone 2 and the direction of voice arrival 2 (72) from the mouth of the customer 21 to the portable salesperson voice collecting device 60 are different from the direction vector connecting the microphone 60a and the microphone 60b. Are arranged to have For example, the direction vector is directed from the top of the salesperson 22 toward the feet and is directed in a direction substantially parallel to the body axis (the two microphones 60a and 60b appear to be arranged one above the other as viewed from the customer 21. ), Voice arrival direction 1 (70) may be a direction substantially parallel to the direction vector, and voice arrival direction 2 (71) may be a direction substantially perpendicular to the direction vector. The portable salesperson voice collecting device 60 is not limited to this, and the direction vector connecting the microphone 60a and the microphone 60b makes different angles with respect to the voice arrival direction 1 (70) and the voice arrival direction 2 (71). The size, shape, etc. of the portable salesperson voice collecting device 60 can be designed as appropriate.

  In this way, the portable salesperson voice collecting device 60 is arranged, the microphone 60a and the microphone 60b are used as the microphone array in the voice collecting method according to the present invention, and the above-described method for extracting the target voice is performed. By extracting the voice that reaches the microphone array with a time difference, the voice of the salesperson 22 can be selectively collected. In the present invention, it is possible to realize an implementation means for selectively collecting a salesperson's voice using a portable salesperson voice collection device 60 having a form similar to a commercially available voice recorder or the like.

[Hardware configuration of salesperson voice collection device]
FIG. 11 is a diagram showing a hardware configuration of a salesperson voice collection device according to an embodiment of the present invention. In FIG. 11, the salesperson voice collection device is the information processing device 1000, and the hardware configuration thereof is illustrated. In the following, an overall configuration of an information processing apparatus typified by a computer will be described, but it goes without saying that the minimum required configuration can be selected according to the environment.

  The information processing apparatus 1000 includes a CPU (Central Processing Unit) 1010, a bus line 1005, a communication I / F 1040, a main memory 1050, a BIOS (Basic Input Output System) 1060, a parallel port 1080, a USB port 1090, a graphic controller 1020, and a VRAM 1024. , An audio processor 1030, an I / O controller 1070, and input means such as a keyboard and mouse adapter 1100. Storage means such as a flexible disk (FD) drive 1072, a hard disk 1074, an optical disk drive 1076, and a semiconductor memory 1078 can be connected to the I / O controller 1070.

  Microphones 1036 and 1037, an amplifier circuit 1032, and a speaker 1034 are connected to the audio processor 1030. A display device 1022 is connected to the graphic controller 1020.

The BIOS 1060 stores a boot program executed by the CPU 1010 when the information processing apparatus 1000 is activated, a program depending on the hardware of the information processing apparatus 1000, and the like. An FD (flexible disk) drive 1072 reads a program or data from the flexible disk 1071 and provides it to the main memory 1050 or the hard disk 1074 via the I / O controller 1070.
FIG. 5 shows an example in which the information processing apparatus 1000 includes a hard disk 1074. However, an external device connection interface (not shown) is connected to the bus line 1005 or the I / O controller 1070, and the information processing apparatus A hard disk may be connected or added to the outside of 1000.

  As the optical disk drive 1076, for example, a DVD-ROM drive, a CD-ROM drive, a DVD-RAM drive, or a CD-RAM drive can be used. In this case, it is necessary to use the optical disk 1077 corresponding to each drive. The optical disk drive 1076 can also read a program or data from the optical disk 1077 and provide it to the main memory 1050 or the hard disk 1074 via the I / O controller 1070.

  The computer program provided to the information processing apparatus 1000 is stored in a recording medium such as the flexible disk 1071, the optical disk 1077, or a memory card and provided by the user. The computer program is read from the recording medium via the I / O controller 1070 or downloaded via the communication I / F 1040 to be installed and executed in the information processing apparatus 1000. The operation that the computer program causes the information processing apparatus to perform is the same as the operation in the apparatus that has already been described.

  The aforementioned computer program may be stored in an external storage medium. As the storage medium, in addition to the flexible disk 1071, the optical disk 1077, or the memory card, a magneto-optical recording medium such as an MD or a tape medium can be used. Further, a storage device such as a hard disk or an optical disk library provided in a server system connected to a dedicated communication line or the Internet may be used as a recording medium, and a computer program may be provided to the information processing apparatus 1000 via the communication line. .

  In the above example, the information processing apparatus 1000 has been mainly described. However, the information described above is obtained by installing a program having the function described in the information processing apparatus in a computer and causing the computer to operate as the information processing apparatus. Functions similar to those of the processing device can be realized.

  This apparatus can be realized as hardware, software, or a combination of hardware and software. A typical example of implementation using a combination of hardware and software is implementation on a computer system having a predetermined program. In such a case, the predetermined program is loaded into the computer system and executed, whereby the program causes the computer system to execute the processing according to the present invention. This program is composed of a group of instructions that can be expressed in any language, code, or notation. Such instructions can be either or both of the following: (1) conversion to another language, code, or notation; (2) replication to other media; Can be executed after the Of course, the present invention includes not only such a program itself but also a program product including a medium on which the program is recorded. The program for executing the functions of the present invention can be stored in any computer-readable medium such as a flexible disk, MO, CD-ROM, DVD, hard disk device, ROM, MRAM, and RAM. Such a program can be downloaded from another computer system connected via a communication line or copied from another medium for storage in a computer-readable medium. Further, such a program can be compressed or divided into a plurality of parts and stored in a single or a plurality of recording media.

1 is a block diagram schematically showing an example of a sound collection system according to an embodiment of the present invention. It is a figure which shows the audio | voice arrival direction with respect to a microphone. It is a figure which shows the structure of the target audio | voice extraction apparatus 12 based on one Embodiment of this invention. It is a figure which shows an example of the speaker direction index with respect to the position of a microphone. It is a figure which shows the classification | category by the directivity of a microphone. It is a figure which shows an example of the place which arrange | positions the microphone array by embodiment of this invention. It is a block diagram which shows the target audio | voice extraction apparatus 12 shown in FIG. 1 in detail. It is a flowchart for demonstrating the process in the utterance area index detection process part 31 shown in FIG. It is a figure which shows the example of the audio | voice signal of the noise area in the various steps of the audio | voice emphasis treatment based on this invention. It is a figure which illustrates the operation condition of portable salesperson voice collection device 60 concerning one embodiment of the present invention. It is a figure which shows the hardware constitutions of the salesperson audio | voice collection apparatus based on one Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Voice collection system 11 Microphone array 12 Target voice extraction device 13 Customer dialogue recording server 31 Speech section index detection processing unit 32, 33 Speech recognition unit 34 Integrated selection unit 35 Recording range extraction unit 60 Portable salesperson voice collection device 105, 106 Discrete Fourier transform processing unit 110 CSP coefficient calculation unit 120 group delay array processing unit,
130 Noise Estimation Unit 140 SS Processing Unit 150 Gain Adjustment Processing Unit 160 Flooring Processing Unit

Claims (16)

A voice collection method using a microphone array in which at least a first microphone and a second microphone are arranged at a predetermined distance in order to extract and collect a target voice of interest from among a plurality of voices having different directions of arrival. There,
The speech signals received by the first microphone and the second microphone are discrete Fourier transformed to obtain a plurality of CSP coefficients related to the direction of arrival of the speech, and the plurality of speech signals are obtained from the plurality of CSP coefficients. Detecting a signal;
Detecting a speech direction index defined according to an angle formed by a line segment connecting the first microphone and the second microphone and the arrival direction from the obtained CSP coefficients;
Extracting the target speech signal from the detected plurality of speech signals according to the detected speech direction index;
The voice collecting method including:   The plurality of sounds are a first sound and a second sound, and each of the first sound generation source and the second sound generation source includes the first microphone and the second microphone. The speech collection method according to claim 1, wherein the speech collection method is located within a predetermined angle range with a connecting line segment as a central axis.   The plurality of sounds are a first sound and a second sound, a line connecting the first sound generation source and the second sound generation source, and the first sound included in the microphone array. The voice collecting method according to claim 1, wherein the microphone and the line connecting the second microphone are arranged substantially parallel to each other.   The step of detecting the voice direction index compares the magnitude relation of the plurality of CSP coefficients, and the voice direction index is derived from a difference in time when one voice reaches the first microphone and the second microphone. The voice collection method according to claim 1, wherein:   The speech collection method according to claim 1, further comprising a step of performing an array process to emphasize the target speech based on the result of the discrete Fourier transform. And performing SS (spectral subtraction) processing using the estimated noise power spectrum (Uω) and the subtraction constant (α) based on the respective discrete Fourier transform results;
Performing gain adjustment from the output of the SS processing step and the CSP coefficient;
Performing a flooring process using a flooring coefficient (β) w for the output of the gain adjustment step;
The voice collecting method according to claim 1, comprising:   The plurality of sounds are a first sound and a second sound, and the step of detecting the plurality of sound signals further includes at least one of the first sound signal and the second sound signal based on the CSP coefficient. The speech collection method according to claim 1, wherein an utterance section is detected for one side.   8. The voice collection method according to claim 7, wherein the step of detecting the plurality of voice signals further separates at least one of the first voice signal and the second voice signal from the detected speech period. .   The step of detecting the plurality of sound signals further includes a sound direction index corresponding to each of the first sound signal and the second sound signal as a first sound direction index and a second sound direction index. The voice collecting method according to claim 7, which is associated. The step of extracting the target speech signal further comprises the step of extracting the first speech from the first speech signal, the second speech signal, the first speech direction index, and the second speech direction index. And the second voice signal are subjected to voice recognition processing to obtain the first voice recognition result and the second voice recognition result, and the time when the first voice and the second voice are spoken. A voice recognition step for obtaining first time information and second time information indicating:
An integration step of integrating the first speech recognition result and the second speech recognition result together with the first time information and the second time information;
When a location to be extracted is selected as a result of the integration, a cutout step of cutting out a speech signal of an utterance section according to the location;
The voice collecting method according to claim 9, comprising:   The integration step further includes the first voice recognition result and the second voice recognition result, the first time information and the second time information, the first voice direction index, and the second voice. The method of claim 10, comprising associating with a direction index.   The voice collection method according to claim 10, wherein the cutout step includes a step of cutting out a voice signal of an utterance section according to a voice direction index and time information corresponding to the selected location from the integrated information.   The voice collecting method according to claim 10, comprising a step of recording the cut out voice signal as a voice to be recorded.   The computer program for performing each step of the method of any one of Claim 1 to 13 using a computer. In a sound collection system using a microphone array in which at least a first microphone and a second microphone are arranged at a predetermined distance in order to extract and collect a target target sound from a plurality of sounds having different directions of arrival. There,
The speech signals received by the first microphone and the second microphone are discrete Fourier transformed to obtain a plurality of CSP coefficients related to the direction of arrival of the speech, and the plurality of speech signals are obtained from the plurality of CSP coefficients. Voice detection means for detecting a signal;
A voice direction index detecting means for detecting a voice direction index defined according to an angle formed by a line segment connecting the first microphone and the second microphone and the arrival direction from the plurality of CSP coefficients obtained;
Target speech extraction means for extracting the target speech signal from the plurality of speech signals detected by the detected speech direction index;
Including voice collection system. A microphone array in which at least a first microphone and a second microphone are arranged at a predetermined distance in order to extract and collect the first voice out of the first voice and the second voice having different directions of arrival. A voice collection system using
Each of the first sound generation source and the second sound generation source is located within a predetermined angle range with a line segment connecting the first microphone and the second microphone as a central axis. And
Means for performing array processing to emphasize the target sound based on the result of discrete Fourier transform of the sound signals received by the first microphone and the second microphone;
Means for performing SS (spectral subtraction) processing using a power spectrum (Uω) of an estimated noise and a subtraction constant (α) based on the respective discrete Fourier transform results;
Means for obtaining a CSP coefficient from the result of each discrete Fourier transform, and performing gain adjustment from the output of the means for performing the SS processing and the CSP coefficient;
Means for performing a flooring process using a flooring coefficient (β) for the output of the means for performing the gain adjustment;
Voice detection means for detecting the first voice signal and the second voice signal from the voice signal subjected to the flooring process;
The time required for one sound to reach the first microphone and the second microphone by comparing the magnitude relationship of the obtained CSP coefficients independently for each of the first sound and the second sound. A voice direction index detecting means for determining a voice direction index derived from the difference between
Target speech extraction means for extracting the first speech signal from the speech direction index;
Utterance period detecting means for detecting an utterance period of the signal of the first voice from the CSP coefficient;
Target speech separation means for separating the first speech signal from the detected speech section;
Including voice collection system.

Images