JP2019045576A - Target voice extraction method, target voice extraction device and target voice extraction program - Google Patents

Abstract

To precisely extract target voice in environment where various background noises are mixed.SOLUTION: A keyword detection section 11 detects that a keyword is spoken on the basis of an inputted sound signal, and calculates a keyword section being a time section when the keyword is spoken. A space covariance calculation section 12 calculates a target signal space covariance matrix under noise, which is a space covariance matrix on the basis of the sound signal of the keyword section, and calculates a noise space covariance matrix being the space covariance matrix on the basis of the sound signal of the prescribed time section except for the keyword section. A noise suppression section 13 calculates a noise suppression filter on the basis of the target signal space covariance matrix under noise and the noise space covariance matrix, and applies the noise suppression filter to the inputted sound signal to extract a target voice.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a target speech extraction method, a target speech extraction device, and a target speech extraction program.

  A method of extracting only a target voice from observation signals collected by a plurality of microphones in the situation where the voice generated by the target speaker and the background noise are mixed has been proposed conventionally (for example, non-patent literature) See 1).

  Here, a conventional target speech extraction device will be described using FIG. FIG. 10 is a diagram showing the configuration of a conventional target speech extraction apparatus. As shown in FIG. 10, the target speech extraction device 10a has a space covariance calculation unit 11a, a noise suppression unit 13a, and a storage unit 14a.

  First, the space covariance calculation unit 11a calculates, for each time frequency point, a time frequency mask representing which of the target voice and noise is dominant. Next, the space covariance calculation unit 11a uses the time frequency mask to calculate the feature of the acoustic signal at the time frequency point where the target voice is dominant, and includes both the target voice and noise based on the feature. Compute the noisy target signal space covariance matrix which is the space covariance matrix of the acoustic signal. In addition, the spatial covariance calculation unit 11a calculates a feature of an acoustic signal at a time frequency point where noise is dominant using a time frequency mask, and based on the feature, a spatial covariance of an acoustic signal containing only noise. Compute the noise space covariance matrix, which is a matrix.

  Then, the noise suppression unit 13a calculates the noise suppression filter based on the acoustic signal, the target signal space covariance matrix under noise, and the noise space covariance matrix, and applies the calculated noise suppression filter to the observation signal, Extract the target voice.

  As a method of mask calculation, a method based on spatial feature clustering of acoustic signals (see, for example, Non-Patent Document 1), a method based on a deep neural network (DNN) (see, for example, Non-patent document 2), etc. are known. It is done.

Takuya Higuchi, Nobutaka Ito, Takuya Yoshioka, Tomohiro Nakatani, “Robust MVDR beamforming using time-frequency masks for online / offline ASR in noise,” ICASSP 2016, pp. 5210-5214, 2016. Jahn Heymann, Lukas Drude, Reinhold Haeb-Umbach, “Neural network based spectral mask estimation for acoustic beamforming,” ICASSP 2016, pp. 196-200, 2016.

  However, the conventional target speech extraction method has a problem that the extraction accuracy of the target speech may be low in an environment where various background noises are mixed. For example, when a speaker other than the target speaker is included in the sound signal, or a sound such as a TV is included, in the conventional target speech extraction method, it is necessary to determine which is the target speech. It may not be possible. Also, for example, when processing an input acoustic signal sequentially, the estimation accuracy of the spatial covariance matrix may be low at the start of the target speech.

  Furthermore, in order to avoid the above-mentioned problems, a method has been proposed in which a previously learned noisy target signal space covariance matrix or noise space covariance matrix is used as an initial value of space covariance. However, in this method, it is necessary to fix the position where the speaker speaks to some extent in advance. Therefore, when the speaker speaks freely, it is difficult to extract a target voice.

  In order to solve the problems described above and achieve the purpose, the target speech extraction method of the present invention detects that a keyword has been spoken based on the input acoustic signal, and a time interval in which the keyword is spoken. Calculating a keyword segment that is a keyword segment, and calculating a noisy target signal space covariance matrix that is a spatial covariance matrix based on the acoustic signal of the keyword segment, and an acoustic signal of a predetermined time segment other than the keyword segment Calculating a noise covariance calculation process for calculating a noise space covariance matrix which is a space covariance matrix based on the noise suppression filter based on the noise target signal space covariance matrix and the noise space covariance matrix, And applying a noise suppression filter to the acoustic signal to extract a target speech.

  According to the present invention, it is possible to accurately extract the target voice in an environment where various background noises are mixed.

FIG. 1 is a diagram showing an example of the configuration of a target speech extraction apparatus according to the first embodiment. FIG. 2 is a flowchart showing the process flow of the target speech extraction device according to the first embodiment. FIG. 3 is a flowchart showing a process flow of the target speech extraction device according to the modification of the first embodiment. FIG. 4 is a diagram showing an example of the configuration of the target speech extraction device according to the second embodiment. FIG. 5 is a flowchart showing a process flow of the target speech extraction device according to the second embodiment. FIG. 6 is a diagram showing an example of the configuration of the target speech extraction device according to the third embodiment. FIG. 7 is a flowchart showing the process flow of the target speech extraction device according to the third embodiment. FIG. 8 is a flowchart showing the process flow of the target speech extraction apparatus according to the modification of the third embodiment. FIG. 9 is a diagram illustrating an example of a computer that executes a target voice extraction program. FIG. 10 shows a conventional target speech extraction apparatus.

  Hereinafter, embodiments of a target speech extraction method, a target speech extraction device, and a target speech extraction program according to the present application will be described in detail based on the drawings. In the embodiment, the target speech extraction method and the target speech extraction program are executed by the target speech extraction device.

  In the following description, in a situation where target voice and background noise are mixed, M (where M is an integer of 2 or more) acoustic signals recorded at different positions are input to the target voice extraction device. Do.

Hereinafter, in the embodiment, each acoustic signal will be described as being expressed as a short-time Fourier transformed signal x _{t, f} ^(m) (t: time number, f: frequency number, m: microphone number) . The target speech extraction device can handle other time-frequency expressed signals in the same manner, and even if an acoustic signal is given as a waveform, the time-frequency can be obtained by appropriately performing frequency analysis. It can be converted to a represented signal and used. The present invention is not limited by the embodiments described herein.

First Embodiment
Configuration of First Embodiment
The configuration of the target speech extraction device according to the first embodiment, the flow of processing, and effects will be described. First, the configuration of the first embodiment will be described using FIG. FIG. 1 is a diagram showing an example of the configuration of a target speech extraction apparatus according to the first embodiment. As shown in FIG. 1, the target speech extraction device 10 has a keyword detection unit 11, a space covariance calculation unit 12, and a noise suppression unit 13.

  The keyword detection unit 11 detects that a keyword has been spoken based on the input acoustic signal, and calculates a keyword section that is a time interval in which the keyword is spoken. The keyword detection unit 11 monitors whether or not a predetermined keyword is included in the input audio signal, and when a keyword is detected, determines a time interval in which the keyword is included and outputs it.

  The keyword detection unit 11 can obtain the start time and the end time of the keyword, and can set the time section from the start time to the end time as the keyword section. For example, the keyword detection unit 11 can detect the end time at which the keyword has been spoken, and calculate the keyword section based on the end time and a predetermined time for talking the keyword. Specifically, the keyword detection unit 11 can obtain the keyword end time, and trace back the time that is expected from the end time by the predetermined time length of the keyword, and can determine it as the keyword start time.

  In addition, the keyword detection unit 11 may define a keyword section including time sections before and after the obtained keyword section in consideration of an estimation error of the keyword section. Also, the keyword detection unit 11 may use any method as a keyword detection method as long as it can obtain a keyword section or a keyword end time. Conventionally, various methods have been proposed as keyword detection methods.

  The spatial covariance calculation unit 12 receives an acoustic signal and a keyword section. Then, the space covariance calculation unit 12 calculates a noisy target signal space covariance matrix which is a space covariance matrix based on the sound signal of the keyword section, and generates space covariance based on the sound signal of a predetermined time section other than the keyword section. Compute the noise space covariance matrix, which is the variance matrix. A noisy target signal space covariance matrix is a space covariance matrix of an acoustic signal that contains both speech and noise. Also, the noise space covariance matrix is a space covariance matrix of an acoustic signal containing only noise.

  Now, it is assumed that the acoustic signals from all the microphones are summarized for each time frequency point (t, f) and represented by one vector as shown in the following equation (1).

Here, 'represents non-conjugate transposition of a matrix or a vector. In addition, a set of time number that is included in the keyword section is denoted by T _in, and be referred to a set of time number that is included in the time interval of non-keyword section and T _out. Further, the number of elements T _in and T _out, respectively, and be referred to as N _in and N _out.

Here, T _out refers to a time interval in which only noise before and after the keyword interval is expected to be present. For example, T _out may be a predetermined time interval (eg, 3 seconds) immediately before the keyword interval, or may be a predetermined time interval (eg, 1 second) immediately after the keyword interval. Further, T _out may be a combination of a predetermined time section immediately before the keyword section and a predetermined time section immediately after the keyword section. Then, the space covariance calculation unit 12 obtains the noise target signal space covariance matrix _{s s} and the noise space covariance matrix _{n n} respectively as the following equations (2-1) and (2-2) be able to.

  However, H represents conjugate transposition of a matrix or a vector. Further, t∈T represents that t is an element of the set T.

  The noise suppression unit 13 receives the input acoustic signal, the noise space covariance matrix, and the noisy target signal space covariance matrix. Then, the noise suppression unit 13 calculates a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix, applies a noise suppression filter to the input acoustic signal, and extracts the target voice. .

  For example, the noise suppression unit 13 is described in Reference 1 (Nobutaka Ito, Shoko Araki, Marc Delcroix, Tomohiro Nakata, “PROBABILISTIC SPATIAL DICTIONARY BASED ONLINE ADAPTIVE BEAMFORMING FOR MEETING RECOGNITION IN NOISY AND REVERBERANT ENVIRONMENTS,” ICASSP 2017, pp. 681-685 , 2017.) can be used to calculate the noise suppression filter.

  When the noise suppression unit 13 uses the method described in reference 1, assuming that E () is a function for extracting an eigenvector corresponding to the maximum eigenvalue of the matrix, the following (3-1) formula and (3-2) The noise suppression filter can be calculated as in

Here, h _f = [h _f ⁽¹⁾ , h _f ⁽²⁾ ,..., H _f ^(M) ] corresponds to the acoustic transfer function (steering vector of the target voice) from the speaker to the microphone, and w _f = [w _f ⁽¹⁾ , w _f ⁽²⁾ ,..., w _f ^(M) ] corresponds to a distortionless minimum dispersion filter.

  Also, for example, the noise suppression unit 13 is described in Reference 2 (Shoko Araki, Hiroshi Sawada, Shoji Makino, “Blind speech separation in a meeting situation with maximum SNR beamformers,” ICASSP 2007, vol. I, pp. 41-44, The noise suppression filter can be calculated as the following equations (4-1) and (4-2) using the method described in 2007.).

Here, φ _x represented by the equation (5) corresponds to the space covariance matrix of the acoustic signal, and for example, according to the equations (4-1) and (4-2), It can be asked. Also, e _j = [0, ..., 0, 1, 0, ..., 0] 'means that only the j-th element corresponding to the microphone number for which speech is to be extracted is 1 and the other elements are It corresponds to an M-order column vector that is zero.

  In addition to the above, the noise suppression unit 13 can use various methods such as the method described in Non-Patent Document 1 or Non-Patent Document 2 as a calculation method of the noise suppression filter. Subsequently, the noise suppression unit 13 extracts the target voice by applying the calculated noise suppression filter to the acoustic signal, for example, as in the following equation (7).

  The sound signal received by the keyword detection unit 11, the sound signal received by the space covariance calculation unit 12, and the sound signal received by the noise suppression unit 13 may be the same or different from each other. Good.

  For example, the keyword detection unit 11 can calculate a keyword section based on the first acoustic signal. Also, the space covariance calculation unit 12 can calculate the noise target signal space covariance matrix and the noise space covariance matrix based on the first acoustic signal and the keyword section. At this time, the noise suppression unit 13 applies the noise suppression filter calculated based on the noisy target signal space covariance matrix and the noise space covariance matrix to the second acoustic signal different from the first acoustic signal. Speech can be extracted.

(Modification)
A modification of the first embodiment will be described. In the modification of the first embodiment, the space covariance calculation unit 12 further calculates, for each time frequency point, a time frequency mask indicating which of the target voice and noise is dominant for the sound signal in the keyword section, The time-frequency mask can be used to calculate the noisy target signal space covariance matrix and the noise space covariance matrix.

Here, the time frequency mask of the time frequency point (t, f) is expressed as M _{t, f,} and it is assumed that 0 <= M _{t, f} <= 1. Further, M _{t, f} = 1 represents that the target speech is dominant, and M _{t, f} = 0 represents that noise is predominant. Further, as a method of calculating a time frequency mask, a method described in Non-Patent Document 1 or 2 and a method described in Reference 1 are known.

In the modification of the first embodiment, the space covariance calculation unit 12 always sets M _{t, f} = 0 in a time interval other than the keyword interval indicated by T _out , and the keyword interval indicated by T _in. Then, assume that the time frequency mask is calculated by some time frequency mask calculation method. Then, based on the calculated time-frequency mask, the space covariance calculation unit 12 determines the noise target signal space covariance matrix and the noise space covariance matrix as the following equations (8-1) and (8-2), respectively. ) Calculate as equation.

Processing of the First Embodiment
Here, the flow of processing of the target speech extraction device according to the first embodiment will be described with reference to FIG. FIG. 2 is a flowchart showing the process flow of the target speech extraction device according to the first embodiment.

  First, as shown in FIG. 2, the keyword detection unit 11 monitors an acoustic signal until a keyword is detected (No in step S101). Here, when the keyword detection unit 11 detects a keyword (Yes in step S101), the keyword detection unit 11 calculates a keyword section (step S102).

  The space covariance calculation unit 12 calculates a noisy target signal space covariance matrix based on the acoustic signal in the keyword section (step S103). Next, the space covariance calculation unit 12 calculates a noise space covariance matrix on the basis of the acoustic signal of the section other than the keyword section (step S104).

  Then, the noise suppression unit 13 calculates a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix (step S105). Here, the noise suppression unit 13 extracts the target voice from the acoustic signal using the noise suppression filter (step S106).

  The process flow of the target speech extraction apparatus according to the modification of the first embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing a process flow of the target speech extraction device according to the modification of the first embodiment.

  First, as shown in FIG. 3, the keyword detection unit 11 monitors an acoustic signal until a keyword is detected (No in step S151). Here, when the keyword detection unit 11 detects a keyword (Yes in step S151), the keyword detection unit 11 calculates a keyword section (step S152).

  Here, the space covariance calculation unit 12 calculates a time frequency mask based on the acoustic signal of the keyword section (step S153). Then, the space covariance calculation unit 12 calculates a noisy target signal space covariance matrix based on the acoustic signal of the keyword section using the time frequency mask (step S154). Further, the space covariance calculation unit 12 calculates the noise space covariance matrix based on the sound signal of the keyword section and the section other than the keyword section using the time frequency mask (step S155).

  Then, the noise suppression unit 13 calculates a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix (step S156). Here, the noise suppression unit 13 extracts the target voice from the acoustic signal using the noise suppression filter (step S157).

[Effect of First Embodiment]
Here, the keyword section calculated by the keyword detection unit 11, each space covariance matrix calculated by the space covariance calculation unit 12, and the noise suppression filter and time frequency mask calculated by the noise suppression unit 13 are all estimated. It is a value. Therefore, in the present embodiment, the target speech extraction device 10 estimates each space covariance matrix, the noise suppression filter, and the noise suppression mask. Also, as the estimation accuracy of each estimation by the target speech extraction device 10 is improved, the extraction accuracy of the target speech by the target speech extraction device 10 is improved.

  In the first embodiment, the keyword detection unit 11 detects that a keyword has been spoken based on the input acoustic signal, and calculates a keyword section which is a time interval in which the keyword is spoken. Also, the space covariance calculation unit 12 calculates a noisy target signal space covariance matrix which is a space covariance matrix based on an acoustic signal of a keyword section, and generates a spatial covariance based on an acoustic signal of a predetermined time section other than the keyword section. Compute the noise space covariance matrix, which is the variance matrix. Also, the noise suppression unit 13 calculates a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix, applies a noise suppression filter to the input acoustic signal, and extracts the target voice. . As described above, according to the first embodiment, even in an environment in which various background noises are mixed, a noisy target signal space covariance matrix relating to the position of a speaker who emits a target voice by detecting a keyword. Because the noise space covariance matrix can be accurately estimated, it is possible to accurately extract the target speech emitted by the speaker.

  The keyword detection unit 11 can detect the end time at which the keyword has been spoken, and calculate the keyword section based on the end time and a predetermined time for talking the keyword. Thus, in the first embodiment, it is possible to calculate the keyword section even when the start time of the keyword can not be detected.

  The spatial covariance calculation unit 12 further calculates a time frequency mask indicating which of the target voice and noise is dominant for each time frequency point for the sound signal in the keyword section, and using the time frequency mask, the noise down purpose Signal space covariance matrices and noise space covariance matrices can be calculated. As described above, in the modification of the first embodiment, the time-frequency point where noise is dominant in the keyword section is included in the calculation of the noise space covariance matrix, and the same time frequency point is the noise target signal space covariance matrix. Since it can be excluded from the calculation, each space covariance matrix can be estimated with higher accuracy.

Second Embodiment
Configuration of Second Embodiment
The configuration of the target speech extraction apparatus according to the second embodiment, the flow of processing, and effects will be described. First, the configuration of the second embodiment will be described using FIG. FIG. 4 is a diagram showing an example of the configuration of the target speech extraction device according to the second embodiment. As shown in FIG. 4, the target speech extraction device 20 includes a keyword detection unit 21, a space covariance calculation unit 22, a noise suppression unit 23, and a storage unit 24.

  The keyword detection unit 21 performs the same process as the keyword detection unit 11 of the first embodiment. That is, the keyword detection unit 21 detects that the keyword has been spoken based on the input acoustic signal, and calculates a keyword section which is a time interval in which the keyword is spoken.

The space covariance calculation unit 22 further calculates a short time space covariance matrix which is a space covariance matrix based on each of short time sound signals obtained by dividing the sound signal into predetermined time intervals, and calculates the short time The space covariance matrix is stored in the storage unit 24. The space covariance calculation unit 22 calculates the short time space covariance matrix Ψ _d for each fixed short time interval as in equation (9) regardless of whether or not the keyword interval is calculated, and stores it in the storage unit 24. Store.

Here, d represents the number of the short time section, T _d represents a set of time numbers included in the short time section d, and N _d represents the number of time numbers included in the short time section d. In addition, the short time section corresponds to, for example, a length of several tens to several thousands milliseconds.

  Then, when the keyword detection unit 21 calculates the keyword interval, the space covariance calculation unit 22 determines, among the short time space covariance matrices stored in the storage unit 24, the short-time space covariance of the time interval including the keyword interval. A noisy target signal space covariance matrix is calculated based on the dispersion matrix, and among the short time space covariance matrices stored in the storage unit 24, based on the short time space covariance matrix of the time interval not including the keyword interval Calculate the noise space covariance matrix.

Here, a set of short time section numbers corresponding to a keyword section is denoted as D _in, and a set of short time section numbers corresponding to a short time section other than the keyword section is denoted as D _out . Further, the number of elements of D _in and D _out will be denoted as Q _in and Q _out , respectively.

Here, D _out refers to a set of short time intervals included in a time interval in which only noise before and after the keyword interval is expected to be present. For example, D _out may be a set of short time intervals corresponding to a predetermined time interval (for example, 3 seconds) immediately before the keyword interval, or D _out may be a predetermined time interval (for example, 1 second) immediately after the keyword interval. It may be a corresponding short time interval. Further, D _out may be a combination of a predetermined time section immediately before the keyword section and a predetermined time section immediately after the keyword section. Then, the space covariance calculation unit 22 obtains the noise target signal space covariance matrix _{s s} and the noise space covariance matrix _{n n} respectively as the following equations (10-1) and (10-2) be able to.

The space covariance calculation unit 22 can realize the calculation of the equation (9) by sequentially adding x _{t, f} x _{t, f} ^H at each time t and dividing by one N _d . For this reason, since x _{t, f} is only used for calculation at time t, the target voice extraction device 20 does not have to store x _{t, f} at times other than time t.

Also, in the calculation of the equations (8-1) and (8-2), it is necessary if there is only a short time space covariance Ψ _{d of} a short time interval corresponding to a keyword interval and a time interval of several seconds before and after that It is sufficient, and there is no need to store the past and future short-time spatial covariances from the short-time section. As a result, in the second embodiment, it is possible to reduce the storage area required for calculating the noise target signal space covariance matrix and the noise space covariance matrix.

  The noise suppression unit 23 performs the same processing as the noise suppression unit 13 of the first embodiment. That is, the noise suppression unit 23 receives the input acoustic signal, the noise space covariance matrix, and the noisy target signal space covariance matrix. Then, the noise suppression unit 13 calculates a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix, applies a noise suppression filter to the input acoustic signal, and extracts the target voice. .

Also, as described above, the space covariance calculation unit 22 can sequentially calculate the noise space covariance matrix and the noisy target signal space covariance matrix. For this reason, the noise suppression unit 23 sequentially receives the noise space covariance matrix and the noisy target signal space covariance matrix, and further performs noise suppression sequentially by the equations (3-1) and (3-2). Filters can be calculated. Also, at this time, the acoustic signal used for the calculation at time t is only x _{t, f} . As a result, in the second embodiment, the storage area required for the calculation of the noise suppression filter can be reduced.

[Process of Second Embodiment]
Here, the process flow of the target speech extraction device according to the second embodiment will be described with reference to FIG. FIG. 5 is a flowchart showing a process flow of the target speech extraction device according to the second embodiment.

  First, as shown in FIG. 5, the space covariance calculation unit 22 calculates a short time space covariance matrix based on the acoustic signal, and stores it in the storage unit 24 (step S201). The space covariance calculation unit 22 calculates a short time space covariance matrix even if it is a time interval in which the keyword is not detected by the keyword detection unit 21 and stores it in the storage unit 24.

  In addition, the keyword detection unit 21 monitors the sound signal until the keyword is detected (No in step S202). Here, when the keyword detection unit 21 detects a keyword (Yes at step S202), it calculates a keyword section (step S203).

  The space covariance calculation unit 22 reads the short time space covariance matrix from the storage unit 24 (step S204). Then, the space covariance calculation unit 22 calculates a noisy target signal space covariance matrix based on the short time space covariance matrix of the keyword section (step S205). Next, the space covariance calculation unit 22 calculates a noise space covariance matrix based on the short time space covariance matrix of the section other than the keyword section (step S206).

  Then, the noise suppression unit 23 calculates a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix (step S207). Here, the noise suppression unit 23 extracts the target voice from the acoustic signal using the noise suppression filter (step S208).

[Effect of Second Embodiment]
In the second embodiment, the space covariance calculation unit 22 further generates a short time space covariance matrix which is a space covariance matrix based on each of short time sound signals obtained by dividing the sound signal into predetermined time intervals. When the calculated short time space covariance matrix is stored in the storage unit and the keyword section is calculated by the keyword detection unit, a time including the keyword section in the short time space covariance matrix stored in the storage unit The noisy target signal space covariance matrix is calculated based on the short time space covariance matrix of the interval, and the short time spatial covariance of the time interval which does not include the keyword interval among the short time spatial covariance matrices stored in the storage unit. Calculate the noise space covariance matrix based on the variance matrix. Thus, in the second embodiment, it is not necessary to store the acoustic signal of the keyword section and the time section before that for the calculation of the space covariance matrix, and only the short-term space covariance matrix is stored. By doing so, it is possible to reduce the storage area required for the calculation.

Third Embodiment
Configuration of Third Embodiment
The configuration of the target speech extraction apparatus according to the third embodiment, the flow of processing, and effects will be described. First, the configuration of the third embodiment will be described using FIG. FIG. 6 is a diagram showing an example of the configuration of the target speech extraction device according to the third embodiment. As shown in FIG. 6, the target speech extraction device 30 has a keyword detection unit 31, a space covariance calculation unit 32, a noise suppression unit 33 and a speech section detection unit 35.

  The keyword detection unit 31 performs the same process as the keyword detection unit 11 of the first embodiment or the keyword detection unit 21 of the second embodiment. That is, the keyword detection unit 31 detects that the keyword has been spoken based on the input acoustic signal, and calculates a keyword section which is a time interval in which the keyword is spoken.

  In addition to the processing similar to the space covariance calculation unit 12 of the first embodiment or the space covariance calculation unit 22 of the second embodiment, the space covariance calculation unit 32 performs a noisy target signal space covariance matrix and noise. Update the spatial covariance matrix.

  First, the spatial covariance calculation unit 32 receives an acoustic signal and a keyword section. Then, the space covariance calculation unit 32 calculates a noisy target signal space covariance matrix which is a space covariance matrix based on the sound signal of the keyword section, and generates space covariance based on the sound signal of a predetermined time section other than the keyword section. Compute the noise space covariance matrix, which is the variance matrix. At this time, the space covariance calculation unit 32 calculates the equations (2-1) and (2-2), or the equations (8-1) and (8-2), or the equations (10-1) and (10). Each space covariance matrix is calculated according to equation (2).

  Next, the space covariance calculation unit 32 further calculates, for each time frequency point, a time frequency mask indicating which of the target voice and noise is dominant for the sound signal of the time period after the keyword period, The mask is used to update the noisy target signal space covariance matrix and the noise space covariance matrix.

The space covariance calculation unit 32 can calculate the time frequency mask in the same manner as the modification of the first embodiment. In addition, the space covariance calculation unit 32 updates the noisy target signal space covariance matrix _{s s} and the noise space covariance matrix 雑 音_n based on the time frequency mask and the acoustic signal.

Now, a set of time numbers included in a time interval from the end of the keyword section to the end of the target voice is denoted as _Tafter . Then, the space covariance calculation unit 32 calculates Φ _s and Φ _n determined by the equations (2-1) and (2-2), for example, the following equations (11-1) and (11-2), respectively. It can be updated as a formula.

In addition, the space covariance calculation unit 32 calculates Φ _s and ( _n determined by the equations (8-1) and (8-2), for example, the following equations (12-1) and (12-2), respectively. It can be updated as a formula.

Here, {T _in , T _after } represents a union of T _in and T _after .

In addition, the space covariance calculation unit 32 calculates Φ _s and Φ _n obtained by the equations (10-1) and (10-2), for example, the following equations (13-1) and (13-2), respectively. It can be updated as a formula.

  Furthermore, the spatial covariance calculation unit 32 receives the acoustic signal sequentially input according to the method described in Non-Patent Document 1, and the above-mentioned (11-1) formula and (11-2), (12 The target signal space covariance matrix and the noise space covariance matrix under noise according to the -1) and (12-2) or (13-1) and (13-2) can be updated sequentially.

  The noise suppression unit 33 performs the same processing as the noise suppression unit 13 of the first embodiment. That is, the noise suppression unit 33 receives the input acoustic signal, the noise space covariance matrix, and the noisy target signal space covariance matrix. Then, the noise suppression unit 33 calculates a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix, applies the noise suppression filter to the input acoustic signal, and extracts the target voice. .

  Here, when the space covariance calculation unit 32 sequentially updates the noisy target signal space covariance matrix and the noise space covariance matrix, the noise suppression unit 33 changes the equations (3-1) and (3-2). The noise suppression filter can be sequentially updated by the equation (4), and extraction of the target speech can be performed sequentially.

(Modification)
A modification of the third embodiment will be described. In the modification of the third embodiment, the voice section detection unit 35 detects a voice section which is a time section after the keyword section and in which the target voice is spoken. At this time, the space covariance calculation unit 32 further updates the noisy target signal space covariance matrix and the noise space covariance matrix using information on the speech segment. Conventionally, various methods have been known as methods for detecting a voice section, and the voice section detection unit 35 may use any of these methods.

Now, the speech section detecting unit 35, a set of decision time number contains speech is denoted by T _on, denoted the set of time numbers determined to not contain speech and T _off To be. Further, it is assumed that denoted the number of time number included in the T _off and N _off.

In addition, the space covariance calculation unit 32 has already applied the time frequency mask M _{t, f} at each time frequency point (t, f) to the acoustic signal after the keyword section in the same manner as in the third embodiment. Assume that you are calculating. Then, the space covariance calculation unit 32 calculates Φ _s and Φ _n determined by the equations (2-1) and (2-2), for example, the following equations (14-1) and (14-2), respectively. It can be updated as a formula.

In addition, the space covariance calculation unit 32 calculates Φ _s and Φ _n obtained by the equations (8-1) and (8-2), for example, the following equations (15-1) and (15-2), respectively. It can be updated as a formula.

In addition, the space covariance calculation unit 32 calculates Φ _s and ( _n determined by the equations (10-1) and (10-2), for example, the following equations (16-1) and (16-2), respectively. It can be updated as a formula.

  Further, in the modification of the third embodiment, it is possible to adopt a configuration of processing in which the time frequency mask is not obtained. This can be achieved, for example, by assuming that speech is present at all time frequency points instead of determining the time frequency mask within the speech segment.

Specifically, in the above update equation, the space covariance calculation unit 32 always sets M _{t, f} = 1 so that the target signal space covariance under noise does not need to be estimated even when the time frequency mask is not estimated. Matrix and noise space covariance matrix can be updated.

  Also, as in the third embodiment, when the space covariance calculation unit 32 sequentially updates the noisy target signal space covariance matrix and the noise space covariance matrix, the noise suppression unit 33 The noise suppression filter can be sequentially updated by the equation 1) and the equation (3-2), and the target speech can be extracted sequentially.

[Process of Third Embodiment]
Here, the process flow of the target speech extraction apparatus according to the third embodiment will be described with reference to FIG. FIG. 7 is a flowchart showing the process flow of the target speech extraction device according to the third embodiment.

  First, as shown in FIG. 7, the keyword detection unit 31 monitors an acoustic signal until a keyword is detected (No in step S301). Here, when the keyword detection unit 31 detects a keyword (Yes in step S301), the keyword detection unit 31 calculates a keyword section (step S302).

  The space covariance calculation unit 32 calculates a noisy target signal space covariance matrix based on the acoustic signal in the keyword section (step S303). Next, the space covariance calculation unit 32 calculates a noise space covariance matrix based on the acoustic signal of the section other than the keyword section (step S304).

  Then, the noise suppression unit 33 calculates a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix (step S305). Here, the noise suppression unit 33 extracts the target voice from the acoustic signal using the noise suppression filter (step S306).

  Here, when the extraction of the target voice is not continued (No in step S307), the target voice extraction device 30 ends the processing. On the other hand, when extraction of the target voice is continued (Yes at step S307), the space covariance calculation unit 32 calculates a time frequency mask based on the acoustic signal of the section after the keyword section (step S308).

  Here, the target voice extraction device 30 can determine whether or not to continue the extraction of the target voice based on the conditions set in advance. For example, whether or not the target voice extraction device 30 starts extraction of the target voice and a predetermined length of time has elapsed, and whether or not the time in which the target voice can not be extracted continues for a predetermined length or longer, etc. It can be determined whether to continue extracting the target voice.

  Then, the space covariance calculation unit 32 updates the noisy target signal space covariance matrix and the noise space covariance matrix using the time frequency mask (step S309). Furthermore, the target speech extraction device 30 proceeds to step S305. The processing is returned, and the noise suppression filter is calculated and the target speech is extracted.

  Here, the flow of processing of the target speech extraction device according to the modification of the third embodiment will be described using FIG. FIG. 8 is a flowchart showing the process flow of the target speech extraction apparatus according to the modification of the third embodiment.

  First, as shown in FIG. 8, the keyword detection unit 31 monitors an acoustic signal until a keyword is detected (No in step S 351). Here, when the keyword detection unit 31 detects a keyword (Yes in step S351), the keyword detection unit 31 calculates a keyword section (step S352).

  The space covariance calculation unit 32 calculates a noisy target signal space covariance matrix based on the acoustic signal in the keyword section (step S353). Next, the space covariance calculation unit 32 calculates a noise space covariance matrix based on the acoustic signals of the sections other than the keyword section (step S354).

  Then, the noise suppression unit 33 calculates a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix (step S355). Here, the noise suppression unit 33 extracts the target voice from the acoustic signal using the noise suppression filter (step S356).

  Here, if the extraction of the target voice is not continued (No at Step S357), the target voice extraction device 30 ends the processing. On the other hand, when extraction of the target voice is continued (Yes at step S357), the speech segment detection unit 35 detects a speech segment after the keyword segment (step S358). Then, the space covariance calculation unit 32 calculates a time frequency mask based on the acoustic signal of the voice section and the acoustic signal of the section other than the voice section (step S359).

  Then, the space covariance calculation unit 32 updates the noisy target signal space covariance matrix and the noise space covariance matrix using the time frequency mask (step S360). Furthermore, the target speech extraction device 30 proceeds to step S355. The processing is returned, and the noise suppression filter is calculated and the target speech is extracted.

[Effect of the third embodiment]
In the third embodiment, the space covariance calculation unit 32 further calculates a time frequency mask indicating which of the target voice and noise dominates at each time frequency point for the sound signal of the time period after the keyword period. And update the noisy target signal space covariance matrix and the noise space covariance matrix using the time frequency mask. Thus, in the third embodiment, changes in two space covariance matrices can be tracked even if the speaker's position moves or the nature of background noise changes after the keyword period. As a result, the target speech can be extracted more accurately.

  In the modification of the third embodiment, the voice section detection unit 35 detects a voice section which is a time section after the keyword section and in which the target voice is spoken. Further, the space covariance calculation unit 32 further updates the noisy target signal space covariance matrix and the noise space covariance matrix using the information on the speech segment. Thereby, in the third embodiment, it is possible to reduce an error detected when the target speech is erroneously included in the time interval in which the target speech is not included, and the changes of the two space covariance matrices are made more accurate. You will be able to track well.

[System configuration etc.]
Further, each component of each device illustrated in the drawings is functionally conceptual, and does not necessarily have to be physically configured as illustrated. That is, the specific form of the dispersion and integration of each device is not limited to that shown in the drawings, and all or a part thereof is functionally or physically dispersed in any unit depending on various loads, usage conditions, etc. It can be integrated and configured. Furthermore, all or any part of each processing function performed by each device is realized by a central processing unit (CPU) and a program analyzed and executed by the CPU, or hardware by wired logic Can be realized as

  Further, among the processes described in the embodiment, all or part of the processes described as being automatically performed can be manually performed, or all the processes described as being manually performed. Alternatively, some of them can be performed automatically by known methods. In addition to the above, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified.

[program]
In one embodiment, the target voice extraction device 10 can be implemented by installing a target voice extraction program for executing extraction of the target voice as package software or online software on a desired computer. For example, the information processing apparatus can be functioned as the target voice extraction apparatus 10 by causing the information processing apparatus to execute the above-described target voice extraction program. The information processing apparatus referred to here includes a desktop or laptop personal computer. In addition, the information processing apparatus also includes mobile communication terminals such as smartphones, cellular phones and PHS (Personal Handyphone System), and slate terminals such as PDA (Personal Digital Assistant).

  The target voice extraction device 10 can also be implemented as a target voice extraction server device that uses a terminal device used by a user as a client and provides the client with a service related to the above-described target voice extraction. For example, the target voice extraction server device is implemented as a server device that provides a target voice extraction service in which an audio signal is input and a target voice is output. In this case, the target voice extraction server device may be implemented as a Web server, or may be implemented as a cloud that provides a service related to the extraction of the target voice by outsourcing.

  FIG. 9 is a diagram illustrating an example of a computer that executes a target voice extraction program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a read only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. Disk drive interface 1040 is connected to disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, the display 1130.

  The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program defining each process of the target speech extraction device is implemented as a program module 1093 in which a computer-executable code is described. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, the hard disk drive 1090 stores a program module 1093 for executing the same processing as the functional configuration of the target voice extraction device. The hard disk drive 1090 may be replaced by a solid state drive (SSD).

  The setting data used in the process of the above-described embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 as needed, and executes them.

  The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, and may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.

10, 20, 30 Target voice extraction device 11, 21, 31 Keyword detection unit 12, 22, 32 Space covariance calculation unit 13, 23, 33 Noise suppression unit 24 Storage unit 35 Voice section detection unit

Claims (8)

A keyword detection step of detecting that a keyword has been spoken based on the input acoustic signal and calculating a keyword section which is a time section in which the keyword is spoken;
Noise target signal space covariance matrix which is a space covariance matrix based on acoustic signals of the keyword section is calculated, and noise space covariance which is a space covariance matrix based on acoustic signals of a predetermined time section other than the keyword section Space covariance calculation step of calculating a matrix;
A noise suppression step of calculating a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix, applying the noise suppression filter to the input acoustic signal, and extracting a target voice;
A target speech extraction method characterized in that   The keyword detecting step detects an end time at which the keyword has been spoken, and calculates the keyword section based on the end time and a predetermined time for talking the keyword. The target speech extraction method according to claim 1.   The space covariance calculation step further calculates the short time space covariance matrix, which is a space covariance matrix based on each of the short time sound signals obtained by dividing the sound signal into predetermined time intervals, When a space-time covariance matrix is stored in a storage unit, and the keyword section is calculated by the keyword detection process, a time interval including the keyword section among the short time space covariance matrix stored in the storage unit Calculating the noisy target signal space covariance matrix based on the short time space covariance matrix of the short time space of the time intervals not including the keyword interval among the short time spatial covariance matrices stored in the storage unit; 3. The target speech extraction method according to claim 1, wherein the noise space covariance matrix is calculated based on a time space covariance matrix.   The space covariance calculation step further calculates, for each time frequency point, a time frequency mask indicating which one of the target voice and noise is dominant for the sound signal in the keyword section, using the time frequency mask. The target speech extraction method according to any one of claims 1 to 3, wherein a noisy target signal space covariance matrix and the noise space covariance matrix are calculated.   The space covariance calculation step further calculates, for each time frequency point, a time frequency mask indicating which one of the target voice and the noise is dominant for the sound signal of the time period after the keyword period, and the time frequency mask The target speech extraction method according to any one of claims 1 to 4, wherein the noisy target signal space covariance matrix and the noise space covariance matrix are updated using. The method further includes a voice period detection step of detecting a voice period which is a time period in which a target voice is being spoken after the keyword period,
The space covariance calculation step further updates the noisy target signal space covariance matrix and the noise space covariance matrix using information on the speech segment. The target voice extraction method according to item 1 or 2. A keyword detection unit that detects that a keyword has been spoken based on the input acoustic signal and calculates a keyword section that is a time section in which the keyword is spoken;
Noise target signal space covariance matrix which is a space covariance matrix based on acoustic signals of the keyword section is calculated, and noise space covariance which is a space covariance matrix based on acoustic signals of a predetermined time section other than the keyword section A spatial covariance calculator that calculates a matrix;
A noise suppression unit that calculates a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix, applies the noise suppression filter to the input acoustic signal, and extracts the target speech;
An object voice extraction device characterized by having. On the computer
A keyword detection step of detecting that a keyword has been spoken based on the input acoustic signal, and calculating a keyword section which is a time section in which the keyword is spoken;
Noise target signal space covariance matrix which is a space covariance matrix based on acoustic signals of the keyword section is calculated, and noise space covariance which is a space covariance matrix based on acoustic signals of a predetermined time section other than the keyword section A spatial covariance calculation step to calculate a matrix,
A noise suppression step of calculating a noise suppression filter based on the noisy target signal space covariance matrix and the noise space covariance matrix, applying the noise suppression filter to the input acoustic signal, and extracting a target voice;
A target speech extraction program characterized by performing.

Images