JP6930408B2 - Estimator, estimation method and estimation program - Google Patents

Description

The present invention relates to an estimation device, an estimation method and an estimation program.

Conventionally, a sound source separation technique for separating an acoustic signal of a target sound source from background noise has been disclosed in order to perform voice recognition or convert the observed sound into a sound that is easy for humans to hear. In such a conventional technique, the estimated value of the acoustic signal of the target sound source is obtained by learning using the training data in which the mixed sound and the acoustic signal of the target sound source as the correct answer value are paired, and the sound source is separated. (See Non-Patent Documents 1 and 2). Non-Patent Document 3 discloses a beamforming technique that suppresses noise and emphasizes sound.

Felix Weninger, Hakan Erdogan, Shinji Watanabe, Emmanuel Vincent, Jonathan Le Roux, John R. Hershey, Bjorn Schuller, “Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR”, International Conference on Latent Variable Analysis and Signal Separation, Springer, 2015, pp.91-99 Santiago Pascual, Antonio Bonafonte, Joan Serra, “SEGAN: Speech Enhancement Generative Adversarial Network”, arXiv preprint arXiv: 1703.09452v3, 2017 T.Higuchi, N.Ito, S.Araki, T.Yoshioka, M.Delcroix, T.Nakatani, “Online MVDR Beamformer Based on Complex Gaussian Mixture Model With Spatial Prior for Noise Robust ASR”, IEEE / ACM Transactions on Audio, Speech, and Language Processing, val.25, No.4, 2017, pp.780-793

However, in many cases, the data of the target sound source as a pair of correct answer values cannot be obtained from the actually recorded data, and the data artificially created by simulation must be used as training data for learning. Did not get.

The present invention has been made in view of the above, and an object of the present invention is to learn and separate sound sources without using training data in which a mixed sound and a target sound source are paired.

In order to solve the above-mentioned problems and achieve the object, the estimation device according to the present invention accepts the input of the observation signal of the mixed sound including the acoustic signal of the target sound source recorded by the microphone, and time from the observation signal. An observation signal for each frequency point is extracted, and a time-frequency analysis unit that generates an observation vector composed of the observation signal for each extracted time frequency point and a predetermined acoustic feature amount are extracted from the generated observation vector. When the feature amount extraction unit and the separator that outputs the estimated value of the acoustic signal of the target sound source are created by learning using the acoustic feature amount, the estimated value of the acoustic signal of the target sound source output by the separator is created. Is a means for learning, and receives an input of an acoustic signal of a true target sound source or an estimated acoustic signal of the target sound source, and whether the acoustic signal of the target sound source is the acoustic signal of the true target sound source or the estimated purpose. It is characterized by including an estimation learning unit that learns the estimated value so that the classifier distinguishes it from the sound signal of the true target sound source when it is input to the classifier that identifies the sound signal of the sound source. do.

According to the present invention, it is possible to perform learning and sound source separation without using training data in which a mixed sound and a target sound source are paired.

FIG. 1 is a schematic diagram showing a schematic configuration of a conventional estimation device. FIG. 2 is a schematic diagram showing a schematic configuration of an estimation system according to the present embodiment. FIG. 3 is a schematic diagram showing a schematic configuration of an estimation device according to the present embodiment. FIG. 4 is a flowchart showing the estimation processing procedure of the present embodiment. FIG. 5 is a schematic diagram showing a schematic configuration of an estimation device according to another embodiment. FIG. 6 is a diagram showing an example of a computer that executes an estimation program.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. The present invention is not limited to this embodiment. Further, in the description of the drawings, the same parts are indicated by the same reference numerals.

[Conventional estimation processing]
First, with reference to FIG. 1, a conventional estimation process for sound source separation will be described. FIG. 1 is a schematic diagram showing a schematic configuration of a conventional estimation device. Conventionally, as shown in FIG. 1, the estimation device 200 that separates sound sources includes a time-frequency analysis unit 201, a feature amount extraction unit 202, and a target sound source estimation unit 203. First, the time-frequency analysis unit 201 applies short-time signal analysis to the observation signals recorded by one microphone in a situation where acoustic signals corresponding to N target sound sources are mixed, and observes sound for each time-frequency point. Extract the signal.

Next, the feature amount extraction unit 202 extracts the acoustic feature amount from the observed acoustic signal. Further, the target sound source estimation unit 203 estimates N target sound sources for each time frequency point by repeating the non-linear conversion using the separator 204 that has been learned and held in advance with the acoustic feature amount as an input. This separator 204 is realized by a neural network and is learned in advance using training data in which a mixed sound and a target sound source included in the mixed sound are paired. As will be described later, this point is the present embodiment. Different from the estimation device.

Here, the observed acoustic signals obtained by using a short-time signal analysis such as a short-time Fourier transform are represented _{by yf and t.} However, t is a time number represented by an integer of 1 to T, and f is a frequency number represented by an integer of 0 to F. When n sound source signals are mixed and observed, the observed acoustic signals yf _{and t at} each time frequency point can be modeled as in the following equation (1).

As shown in the above equation (1), the acoustic signals of the respective target sound sources are estimated from the observation signals obtained by mixing the acoustic signals of the n target sound sources using a neural network. First, by the feature amount extraction process, a set of acoustic feature amounts represented by the following equation (2) can be obtained from the observed acoustic signal.

Next, using the transformation f (.) By the neural network, a set of estimated values of the acoustic signal of the target sound source shown in the following equation (3) is obtained as shown in the following equation (4).

The parameters of the neural network are obtained in advance by minimizing the objective function shown in the following equation (5) by using the training data in which the mixed sound and each objective sound source are paired.

In this way, the target sound source estimation unit 203 minimizes the distance between the estimated value of the acoustic signal of the target sound source estimated from the mixed sound and the correct answer value of the target sound source constituting the mixed sound, thereby causing the neural network. Learn network parameters to create separator 204.

The estimated value of the acoustic signal of the target sound source can be directly obtained as the output of the neural network. Alternatively, when a neural network obtains a mask having a value between 0 and 1 representing the ratio of the target sound source to the mixed sound, the mask is multiplied by the observed acoustic signal to estimate the acoustic signal of the target sound source. (See Non-Patent Document 1).

[Estimation system configuration]
Next, the estimation system 1 according to the present embodiment will be described with reference to FIG. FIG. 2 is a schematic diagram showing a schematic configuration of the estimation system 1 according to the present embodiment. As shown in FIG. 2, the estimation system 1 includes an identification device 10 and an estimation device 20.

In this estimation system 1, the discriminator 10 includes a discriminator 11 and learns the discriminator 11. The classifier 11 discriminates whether the input acoustic signal of the target sound source is the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source. Further, the estimation device 20 uses the separator 24 to separate the acoustic signal of the target sound source from the input observation signal, and outputs the estimated value. Further, the estimation device 20 learns the separator 24 so that the classifier 11 mistakenly identifies the acoustic signal of the target sound source estimated by the separator 24 as the acoustic signal of the true target sound source. I do.

[Identification device configuration]
The identification device 10 according to the present embodiment is realized by a general-purpose computer such as a workstation or a personal computer, and a control unit realized by using a CPU (Central Processing Unit) or the like executes a processing program stored in a memory. As a result, as shown in FIG. 2, it functions as the identification learning unit 12 and the identification unit 13. Further, the classifier 11 is held in a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage unit realized by a storage device such as a hard disk or an optical disk.

The classifier 11 receives the input of the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source, and the acoustic signal of the target sound source is the acoustic signal of the true target sound source or the sound of the estimated target sound source. Identify whether it is a signal.

The discrimination learning unit 12 creates the classifier 11 by learning. Specifically, as will be described later, when the identification learning unit 12 inputs the estimated value of the acoustic signal of the target sound source output by the estimation device 20 to the classifier 11, the discriminator 11 determines the estimated target sound source. Learn to distinguish from acoustic signals.

In the present embodiment, the classifier 11 is realized by BLSTM (Bi-directional Long Short-Term Memory), which is a kind of neural network. Further, the classifier 11 is prepared separately for the target sound source and for the background noise. Each classifier 11 is learned to output 1 when the input acoustic signal is only the target sound source or background noise, that is, when the input acoustic signal is a true target sound source or background noise acoustic signal. Further, each classifier 11 is learned to output 0 in the case of an acoustic signal of a target sound source or background noise estimated by the separator 24 of the estimation device 20 described later.

That is, the discrimination learning unit 12 discriminates the discriminator 11 so that the discriminant value output by the discriminator 11 becomes close to 0 when the estimated value of the sound signal of the target sound source or the background noise output from the estimation device 20 is input. Optimize 11 parameters. Further, the discrimination learning unit 12 determines the parameters of the discriminator 11 so that the discrimination value output by the discriminator 11 becomes close to 1 when the acoustic data of the true target sound source included in the data set of the target sound source is input. Optimize. At the time of learning, the discrimination learning unit 12 obtains parameters using an optimization algorithm called RMSprop and stores them in the storage unit.

Further, the identification unit 13 uses the classifier 11 to discriminate whether the input acoustic signal is the acoustic signal of the true target sound source or the estimated acoustic signal of the target sound source. That is, when the output identification value is close to 1, the identification unit 13 assigns a label 1 indicating that it is an acoustic signal of a true target sound source. Further, the identification unit 13 assigns a label 0 indicating that it is an estimated false target sound source acoustic signal when the output identification value is close to 0.

[Configuration of estimation device]
Next, the estimation device 20 will be described with reference to FIG. FIG. 3 is a schematic diagram showing a schematic configuration of the estimation device 20 according to the present embodiment. As shown in FIG. 3, the estimation device 20 is realized by a general-purpose computer such as a workstation or a personal computer, and a control unit realized by using a CPU or the like executes a processing program stored in a memory. As shown in 3, it functions as a time-frequency analysis unit 21, a feature amount extraction unit 22, and a target sound source estimation unit 23. Further, the separator 24 is held in a storage unit realized by a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The above-mentioned classifier 11 learned in advance may be held in the storage unit.

The time-frequency analysis unit 21 receives the input of the observation signal of the mixed sound including the acoustic signal of the target sound source recorded by the microphone, extracts the observation signal for each time-frequency point from the observation signal, and extracts the time-frequency point. Generates an observation vector composed of each observation signal.

In the following description, it is assumed that the acoustic signals of one target sound source are recorded by M microphones (M ≧ 1) under background noise. _{Here, let ym} and τ be the observation signals recorded by the microphone m. In this case, as shown in the following equation (6), the observed signals ym _{, τ are acoustic signals x m, τ} ⁽ⁿ⁾ (n = 1) derived from each target sound source of N (N> 0). , ..., N).

Specifically, first, the time-frequency analysis unit 21 receives the input of the observation signal represented by the above equation (6) recorded by all the microphones, and receives a short-time signal for _{each observation signal ym, τ.} Apply the analysis to calculate _{the signal features Y m, f, t for each time frequency.} At that time, the short-time signal analysis is not particularly limited, and for example, a short-time discrete Fourier transform, a short-time discrete cosine transform, and the like are applied.

Next, the time-frequency analysis unit 21 ^{summarizes the signals Y (m)} (f, t) obtained at each time-frequency point for all microphones, and observes the M-dimensional vertical vector shown in the following equation (7). It constitutes _{y f and t.}

The feature amount extraction unit 22 extracts a predetermined acoustic feature amount from the generated observation vector. Specifically, the feature amount extraction unit 22 takes the logarithm of the absolute value of one element, which is a component recorded by one predetermined microphone, in the observation vector, and uses it as an acoustic feature amount.

The target sound source estimation unit 23 creates, as an estimation learning unit, a separator 24 that outputs an estimated value of the acoustic signal of the target sound source by learning using the acoustic features. At that time, the target sound source estimation unit 23 is a means for learning the estimated value of the acoustic signal of the target sound source output by the separator 24 as an estimation learning unit, and when input to the above-mentioned classifier 11, the target sound source estimation unit 23 is used. The classifier 11 learns the estimated value so that it distinguishes it from the acoustic signal of the true target sound source.

Specifically, the target sound source estimation unit 23 as the estimation learning unit inputs the acoustic feature amount to the separator 24, so that the target sound source and the background noise are generated for each time frequency point, and the mixed sound is generated. Obtain a mask represented by a value between 0 and 1 representing the proportion. The separator 24 is realized by BLSTM like the classifier 11.

The target sound source estimation unit 23 creates a separator 24 by learning as an estimation learning unit, as described below. The target sound source estimation unit 23 obtains an estimated value of the acoustic signal between the target sound source and the background noise by multiplying the signal feature amount for each time frequency by the mask estimated by the separator 24.

Further, when the target sound source estimation unit 23 as the estimation learning unit inputs the estimated value of the obtained target sound source or background noise acoustic signal to the classifier 11, the classifier 11 determines the true target sound source or background noise. The separator 24 is trained so that it is erroneously identified as an acoustic signal and 1 is output. That is, the target sound source estimation unit 23 optimizes the parameters of the separator 24 so that the output of the classifier 11 is close to 1 when the obtained estimated value is input to the classifier 11. At the time of learning, the target sound source estimation unit 23 obtains the parameters of the separator 24 using RMSprop and holds them in the storage unit.

In this way, in the estimation system 1, the discriminator 11 and the separator 24 are alternately trained so that the separator 24 is indistinguishable from the acoustic signal of the target sound source or the background noise. It is possible to estimate a close acoustic signal.

Further, in the estimation system 1, it is not necessary that the data set of the mixed sound and the data set of the target sound source are paired, and it is possible to use the actual recorded data as the data set of the mixed sound.

Further, the target sound source estimation unit 23 uses the separator 24 to output an estimated value of the acoustic signal of the target sound source included in the input observation signal. The estimated value of the acoustic signal of the target sound source can be obtained, for example, by multiplying the observation vector by the mask obtained by the separator 24. Further, the estimated value of the acoustic signal of the target sound source can be directly obtained as the output of the separator 24.

Alternatively, the estimated value of the acoustic signal of the target sound source can be obtained by beamforming using the mask obtained from the separator 24 and the spatial correlation matrix estimated from the observation vector (see Non-Patent Document 3). That is, the target sound source estimation unit 23 multiplies the mask obtained from the separator 24 by the outer product of the observation vectors and then takes a weighted sum to obtain the spatial correlation matrix of the nth target sound source and the nth target sound source. The beam former is designed by calculating the spatial correlation matrix of the interference sound of. The estimated value of the acoustic signal of each target sound source is obtained by multiplying the observation vector by the beamformer at each frequency.

[Estimation processing]
Next, the estimation process of the estimation system 1 will be described with reference to FIG. FIG. 4 is a flowchart showing the estimation processing procedure of the present embodiment. The flowchart of FIG. 4 is started, for example, at the timing when there is an operation input instructing the start of processing.

First, the time-frequency analysis unit 21 receives the input of the observation signal of the mixed sound including the acoustic signal of the target sound source recorded by the microphone (step S1). Further, the time-frequency analysis unit 21 performs short-time signal analysis, extracts the observation signal for each time-frequency point from the observation signal, and generates an observation vector composed of the extracted observation signal for each time-frequency point (step). S2). Further, the feature amount extraction unit 22 extracts a predetermined acoustic feature amount from the generated observation vector (step S3).

Next, the target sound source estimation unit 23 creates a separator 24 that outputs an estimated value of the acoustic signal of the target sound source by learning using the acoustic features. At that time, when the target sound source estimation unit 23 inputs the estimated value of the acoustic signal of the target sound source output by the separator 24 to the discriminator 11 as the estimation learning unit, the discriminator 11 inputs the sound of the true target sound source. The estimated value is learned so as to distinguish it from the signal (step S4).

Further, the target sound source estimation unit 23 uses the separator 24 to output an estimated value of the acoustic signal of the target sound source included in the input observation signal. As a result, a series of estimation processes are completed.

As described above, in the estimation device 20 of the present embodiment, the time frequency analysis unit 21 receives the input of the observation signal of the mixed sound including the acoustic signal of the target sound source recorded by the microphone, and the time from the observation signal. The observation signal for each frequency point is extracted, and an observation vector composed of the extracted observation signal for each time frequency point is generated. In addition, the feature amount extraction unit 22 extracts a predetermined acoustic feature amount from the generated observation vector. Further, the target sound source estimation unit 23 creates a separator 24 by learning that outputs an estimated value of the acoustic signal of the target sound source using the acoustic features. At that time, the target sound source estimation unit 23 is a means for learning the estimated value of the sound signal of the target sound source output by the separator 24, and is a means for learning the sound signal of the true target sound source or the sound signal of the estimated target sound source. When the input is received and input to the classifier 11 that identifies whether the acoustic signal of the target sound source is the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source, the discriminator 11 is the true purpose. The estimated value is learned so as to distinguish it from the acoustic signal of the sound source.

As a result, the estimation device 20 can perform sound source separation by learning the separator 24 using the actual recorded data instead of using the data set in which the mixed sound and the target sound source are paired. As described above, the estimation device 20 of the present embodiment can learn and separate the sound sources without using the training data in which the mixed sound and the target sound source are paired.

Further, the estimation system 1 includes an identification device 10 that creates an identification device 11 by learning. In the identification device 10, when the identification learning unit 12 inputs the estimated value of the acoustic signal of the target sound source output by the separator 24, that is, the target sound source estimation unit 23, to the classifier 11, the target sound source estimated by the discriminator 11 Learn to distinguish from the acoustic signal of. In this way, by alternately competing the learning of the classifier 11 and the separator 24, the separator 24 can estimate the acoustic signal with higher accuracy.

Note that FIG. 5 is a schematic diagram showing a schematic configuration of the estimation device 20 according to another embodiment. As shown in FIG. 5, the identification device 10 of the above embodiment may be mounted on the same hardware as the estimation device 20. In this case, as shown in FIG. 5, the estimation device 20 includes an identification function unit 100 corresponding to the identification device 10 of the above embodiment. Since each functional unit included in the identification function unit 100 and other functional units of the estimation device 20 are the same as the above-described embodiment described with reference to FIG. 2, the description thereof will be omitted.

[Example]
Using the estimation system 1 according to the above embodiment, in an environment where background noise is present, such as in a bus or in a cafe, a voice of one speaker reading a sentence toward the tablet is transmitted to the tablet with M = 6. An experiment was conducted to confirm the voice recognition performance when recording with individual microphones. Here, the learning rate of RMSProp was set to 1 × 10 ^-3 . The batch size at the time of learning was set to 40.

In this case, the word recognition error rate when voice recognition was performed without using the estimation system 1 was 15.6%. On the other hand, the word recognition error rate was 7.58% when speech enhancement was performed by beamforming using the estimation system 1. In this way, the effect of the estimation process by the estimation system 1 of the present embodiment can be confirmed.

[program]
It is also possible to create a program in which the processing executed by the identification device 10 and the estimation device 20 of the estimation system 1 according to the above embodiment is described in a language that can be executed by a computer. In one embodiment, the identification device 10 and the estimation device 20 can be implemented by installing an estimation program that executes the above estimation process as package software or online software on a desired computer. For example, by causing the information processing device to execute the above estimation program, the information processing device can function as the identification device 10 or the estimation device 20. The information processing device referred to here includes a desktop type or notebook type personal computer. In addition, the information processing device includes smartphones, mobile communication terminals such as mobile phones and PHS (Personal Handyphone System), and slate terminals such as PDAs (Personal Digital Assistants). Further, the terminal device used by the user may be used as a client, and the terminal device may be implemented as a server device that provides the service related to the above estimation process to the client. For example, the estimation device 20 is implemented as a server device that provides an estimation processing service that receives an observation signal as an input and outputs an estimated value. In this case, the estimation device 20 may be implemented as a Web server, or may be implemented as a cloud that provides services related to the above estimation processing by outsourcing. An example of a computer that executes an estimation program that realizes the same function as the identification device 10 or the estimation device 20 will be described below.

FIG. 6 is a diagram showing an example of a computer that executes an estimation program. The computer 1000 has, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. Each of these parts is connected by a bus 1080.

The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1031. The disk drive interface 1040 is connected to the disk drive 1041. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. For example, a mouse 1051 and a keyboard 1052 are connected to the serial port interface 1050. For example, a display 1061 is connected to the video adapter 1060.

Here, the hard disk drive 1031 stores, for example, the OS 1091, the application program 1092, the program module 1093, and the program data 1094. Each table used for processing is stored in, for example, a hard disk drive 1031 or a memory 1010.

Further, the estimation program is stored in the hard disk drive 1031 as, for example, a program module 1093 in which a command executed by the computer 1000 is described. Specifically, the program module 1093 in which each process executed by the identification device 10 or the estimation device 20 described in the above embodiment is described is stored in the hard disk drive 1031.

Further, the data used for information processing by the estimation program is stored as program data 1094 in, for example, the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as needed, and executes each of the above-described procedures.

The program module 1093 and program data 1094 related to the estimation program are not limited to the case where they are stored in the hard disk drive 1031. For example, they are stored in a removable storage medium and read by the CPU 1020 via the disk drive 1041 or the like. May be done. Alternatively, the program module 1093 and the program data 1094 related to the estimation program are stored in another computer connected via a network such as a LAN (Local Area Network) or WAN (Wide Area Network), and are stored via the network interface 1070. It may be read by the CPU 1020.

Although the embodiment to which the invention made by the present inventor is applied has been described above, the present invention is not limited by the description and the drawings which form a part of the disclosure of the present invention according to the present embodiment. That is, all other embodiments, examples, operational techniques, and the like made by those skilled in the art based on the present embodiment are included in the scope of the present invention.

1 Estimating system 10 Discriminator 11 Discriminator 12 Discriminating learning unit 13 Discriminating unit 20,200 Estimating device 21,201 Time frequency analysis unit 22,202 Feature extraction unit 23,203 Target sound source estimation unit 24,204 Separator 100 Identification function Department

Claims (8)

It accepts the input of the observation signal of the mixed sound including the acoustic signal of the target sound source recorded by the microphone, extracts the observation signal for each time frequency point from the observation signal, and composes the observation signal for each time frequency point extracted. The time-frequency analysis unit that generates the observation vector to be performed,
A feature amount extraction unit that extracts a predetermined acoustic feature amount from the generated observation vector, and a feature amount extraction unit.
It is a means for learning the estimated value of the acoustic signal of the target sound source output by the separator when creating a separator that outputs the estimated value of the acoustic signal of the target sound source by using the acoustic feature amount. Then, it accepts the input of the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source, and determines whether the acoustic signal of the target sound source is the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source. An estimation learning unit that learns the estimated value so that the classifier distinguishes it from the acoustic signal of the true target sound source when input to the discriminating device.
An estimation device comprising. It accepts the input of the observation signal of the mixed sound including the acoustic signal of the target sound source recorded by the microphone, extracts the observation signal for each time frequency point from the observation signal, and composes the observation signal for each time frequency point extracted. The time-frequency analysis unit that generates the observation vector to be performed,
A feature amount extraction unit that extracts a predetermined acoustic feature amount from the generated observation vector, and a feature amount extraction unit.
It is a means for learning the estimated value of the acoustic signal of the target sound source output by the separator when creating a separator that outputs the estimated value of the acoustic signal of the target sound source by using the acoustic feature amount. Then, it accepts the input of the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source, and determines whether the acoustic signal of the target sound source is the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source. An estimation learning unit that learns the estimated value so that the classifier distinguishes it from the acoustic signal of the true target sound source when input to the discriminating device.
When the classifier is created by learning, when the estimated value of the acoustic signal of the target sound source output by the separator is input to the classifier, the classifier distinguishes it from the estimated acoustic signal of the target sound source. With the identification learning department that learns to do
An estimation device comprising. It accepts the input of the observation signal of the mixed sound including the acoustic signal of the target sound source recorded by the microphone, extracts the observation signal for each time frequency point from the observation signal, and composes the observation signal for each time frequency point extracted. The time-frequency analysis unit that generates the observation vector to be performed,
A feature amount extraction unit that extracts a predetermined acoustic feature amount from the generated observation vector, and a feature amount extraction unit.
When creating a separator that outputs the ratio of the acoustic signal of the target sound source to the mixed sound using the acoustic feature amount by learning, the ratio of the acoustic signal of the target sound source output by the separator to the mixed sound. Is a means for learning, and accepts the input of the sound signal of the true target sound source or the sound signal of the estimated target sound source, and whether the sound signal of the target sound source is the sound signal of the true target sound source or the estimated purpose. Estimated learning to learn the ratio so that when the estimated value of the acoustic signal of the target sound source is input to the classifier that identifies whether it is the acoustic signal of the sound source, the classifier distinguishes it from the sound signal of the true target sound source. Department and
Using the separator, a target sound source estimation unit that outputs an estimated value of an acoustic signal of the target sound source included in the input observation signal, and a target sound source estimation unit.
An estimation device comprising. It accepts the input of the observation signal of the mixed sound including the acoustic signal of the target sound source recorded by the microphone, extracts the observation signal for each time frequency point from the observation signal, and composes the observation signal for each time frequency point extracted. The time-frequency analysis unit that generates the observation vector to be performed,
A feature amount extraction unit that extracts a predetermined acoustic feature amount from the generated observation vector, and a feature amount extraction unit.
When creating a separator that outputs the ratio of the acoustic signal of the target sound source to the mixed sound using the acoustic feature amount by learning, the ratio of the acoustic signal of the target sound source output by the separator to the mixed sound. Is a means for learning, and accepts the input of the sound signal of the true target sound source or the sound signal of the estimated target sound source, and whether the sound signal of the target sound source is the sound signal of the true target sound source or the estimated purpose. When an estimated value of the sound signal of the target sound source is input to the classifier that outputs the identification value indicating whether it is the sound signal of the sound source, the classifier outputs the discrimination value indicating the sound signal of the true target sound source. , The estimation learning unit that learns the above ratio,
Using the separator, a target sound source estimation unit that outputs an estimated value of an acoustic signal of the target sound source included in the input observation signal, and a target sound source estimation unit.
When the discriminator is created by learning, when the estimated value of the acoustic signal of the target sound source output by the target sound source estimation unit is input to the discriminator, the acoustic signal of the target sound source estimated by the discriminator is input. The identification learning unit that learns to output the identification value indicating
Using the classifier, an identification unit that discriminates whether the input acoustic signal is the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source, and
An estimation device comprising. An estimation method performed by an estimation device
It accepts the input of the observation signal of the mixed sound including the acoustic signal of the target sound source recorded by the microphone, extracts the observation signal for each time frequency point from the observation signal, and composes the observation signal for each time frequency point extracted. The time-frequency analysis process to generate the observation vector to be performed,
A feature amount extraction step for extracting a predetermined acoustic feature amount from the generated observation vector, and
This is a step of learning the estimated value of the acoustic signal of the target sound source output by the separator when creating a separator that outputs the estimated value of the acoustic signal of the target sound source using the acoustic feature amount. Then, it accepts the input of the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source, and determines whether the acoustic signal of the target sound source is the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source. An estimation learning step of learning the estimated value so that the classifier distinguishes it from the acoustic signal of the true target sound source when input to the discriminating device.
An estimation method characterized by including. An estimation method performed by an estimation device
It accepts the input of the observation signal of the mixed sound including the acoustic signal of the target sound source recorded by the microphone, extracts the observation signal for each time frequency point from the observation signal, and composes the observation signal for each time frequency point extracted. The time-frequency analysis process to generate the observation vector to be performed,
A feature amount extraction step for extracting a predetermined acoustic feature amount from the generated observation vector, and
This is a step of learning the estimated value of the acoustic signal of the target sound source output by the separator when creating a separator that outputs the estimated value of the acoustic signal of the target sound source using the acoustic feature amount. Then, it accepts the input of the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source, and determines whether the acoustic signal of the target sound source is the acoustic signal of the true target sound source or the acoustic signal of the estimated target sound source. An estimation learning step of learning the estimated value so that the classifier distinguishes it from the acoustic signal of the true target sound source when input to the discriminating device.
When the classifier is created by learning, when the estimated value of the acoustic signal of the target sound source output by the separator is input to the classifier, the classifier distinguishes it from the estimated acoustic signal of the target sound source. The identification learning process that learns to do
An estimation method characterized by including. An estimation method performed by an estimation device
It accepts the input of the observation signal of the mixed sound including the acoustic signal of the target sound source recorded by the microphone, extracts the observation signal for each time frequency point from the observation signal, and composes the observation signal for each time frequency point extracted. The time-frequency analysis process to generate the observation vector to be performed,
A feature amount extraction step for extracting a predetermined acoustic feature amount from the generated observation vector, and
When creating a separator that outputs the ratio of the acoustic signal of the target sound source to the mixed sound using the acoustic feature amount by learning, the ratio of the acoustic signal of the target sound source output by the separator to the mixed sound. In the process of learning, the sound signal of the true target sound source or the sound signal of the estimated target sound source is received, and the sound signal of the target sound source is the sound signal of the true target sound source or the estimated purpose. Estimated learning to learn the ratio so that when the estimated value of the acoustic signal of the target sound source is input to the classifier that identifies whether it is the acoustic signal of the sound source, the classifier distinguishes it from the sound signal of the true target sound source. Process and
A target sound source estimation step of outputting an estimated value of an acoustic signal of the target sound source included in the input observation signal using the separator, and a target sound source estimation step.
An estimation method characterized by including. An estimation program for operating a computer as an estimation device according to any one of claims 1 to 4.

Images