JP2011107603A - Speech recognition device, speech recognition method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and a method for performing sound source separation and speech recognition to a mixed signal from a plurality of sound sources, and efficiently obtaining a necessary recognition result. <P>SOLUTION: A separation signal is created by processing applying independent component analysis (ICA), on an observation signal composed of the mixed signal in which outputs from a plurality of sound sources are mixed, and speech recognition processing is performed on each separation signal. Then, additional information is created as evaluation information of a speech recognition result. As the additional information, a recognition reliability degree of the speech recognition result, and an utterance degree in task, which indicates whether the speech recognition result is a recognition result relating to the task which is assumed in the speech recognition device. A score of the speech recognition result corresponding to each channel is calculated by applying the additional information, and the recognition result of a high score is selected and output. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a voice recognition device, a voice recognition method, and a program. More specifically, the present invention relates to a speech recognition apparatus, a speech recognition method, and a program for performing speech recognition by separating a mixed signal of a plurality of speech signals using independent component analysis (ICA).

  Independent component analysis (ICA: Independent Component Analysis) is known as a process for separating a mixed signal of a plurality of audio signals. By applying voice recognition to the separation result obtained by independent component analysis (ICA), the target sound source is separated from the target sound and the other sound, and the target sound source is obtained. Can be recognized with high accuracy.

  There are already several systems that combine sound source separation processing based on independent component analysis (ICA) and speech recognition processing, but conventional systems use a plurality of sound sources obtained as a result of ICA. The target channel (sound source) is selected from a plurality of output channels corresponding to, and is used as an input for speech recognition.

First, an outline of independent component analysis (ICA) will be described as background art of the present invention.
ICA is a type of multivariate analysis, and is a technique for separating multidimensional signals using the statistical properties of signals. For details of the ICA itself, see, for example, Non-Patent Document 1 ["Introduction / Independent Component Analysis" (Noboru Murata, Tokyo Denki University Press)].

Hereinafter, the ICA of the sound signal, particularly the ICA in the time frequency domain will be described.
As shown in FIG. 1, a situation is considered in which different sounds are produced from N sound sources and these are observed by n microphones. There is a time delay or reflection until the sound (original signal) emitted by the sound source reaches the microphone. Therefore, the signal (observation signal) observed by the microphone k is expressed by a summation of the convolution of the original signal and the transfer function for all sound sources as shown in Equation [1.1]. be able to. This mixture is referred to below as “convolutive mixture”.
Note that an observation signal of the microphone n is x _n (t). The observation signals of the microphone 1 and the microphone 2 are x ₁ (t) and x ₂ (t), respectively.
When the observation signals for all the microphones are represented by one equation, it can be represented by the following equation [1.2].

However, x (t), s (t) respectively x k (t), a column vector with s k (t) of the elements, A [l] is n × N whose elements a [l] kj It is a matrix. Hereinafter, it is assumed that n = N.

  It is known that the convolutional mixing in the time domain is represented by instantaneous mixing in the time-frequency domain, and the ICA in the time-frequency domain uses this feature.

  Regarding the time-frequency domain ICA itself, Non-Patent Document 2 [“19.2.4. Fourier transform method” in “Detailed Independent Component Analysis”] and Patent Document 1 (Japanese Patent Laid-Open No. 2006-238409 “Audio Signal Separation Device”). Refer to “Noise reduction device and method”).

  When both sides of the above equation [1.2] are subjected to a short-time Fourier transform, the following equation [2.1] is obtained.

In the above equation [2.1],
ω is the frequency bin number,
t is the frame number,
It is.

  If ω is fixed, this equation can be regarded as instantaneous mixing (mixing without time delay). Therefore, in order to separate the observation signals, the calculation formula [2.5] of the separation result [Y] is prepared, and the separation matrix so that each component of the separation result: Y (ω, t) becomes the most independent. Determine W (ω). By such processing, a separated signal is obtained from the mixed audio signal.

  By inputting the separated signal obtained by the independent component analysis (ICA) to the speech recognition system, the recognition result corresponding to each sound source can be obtained with high accuracy. FIG. 2 shows a system example in which a general sound source separation process based on independent component analysis (ICA) and a speech recognition unit are combined.

  Sound is collected by the plurality of microphones 101-1 to 101 -N, and an input waveform corresponding to the sound signal acquired by each of the microphones 101-1 to 101 -N is sent to the sound source separation unit 102. The sound source separation unit 102 performs a process of separating the mixed sound of a plurality of sound sources into individual sound sources derived from each sound source by the independent component analysis (ICA) described above. When the channel selection unit 103 performs channel selection based on the sound source direction, the sound source separation unit simultaneously performs sound source direction estimation.

From the sound source separation unit 102, a separated waveform indicating individual sound signals corresponding to the sound source and sound source direction information are output and input to the channel selection unit 103.
The channel selection unit 103 selects a channel including the target sound from the separated waveforms corresponding to each sound source input from the sound source separation unit 102. For example, it is selected by user designation. One selected separated waveform is output to the voice recognition unit 104.

  The speech recognition unit 104 performs speech recognition using the separated waveform indicating the sound signal corresponding to a certain sound source input from the channel selection unit 103 as input, and outputs a speech recognition result of a specific sound source (target sound).

  A system combining sound source separation processing based on ICA and speech recognition processing has a configuration for obtaining a recognition result of a target sound source through such processing. However, such a system has problems related to uncertainty of ICA output and channel selection for selecting a target voice. Hereinafter, these problems will be described.

  First, the ICA output uncertainty and the channel selection method for selecting the target voice will be described.

(About ICA output uncertainty)
In ICA, since it is uncertain which channel the sound of the separation result corresponding to the original sound source is output to, the channel containing the target sound needs to be selected in some form. For example, Patent Document 2 (Japanese Patent Laid-Open No. 2009-53088) describes the ICA output uncertainty.

(About channel selection method for selecting target audio)
In the case where the output of ICA is output to the post-processing unit and some processing is performed, it is necessary to determine to which channel the sound of the separation result corresponding to the original sound source is output. For example, when speech recognition processing is performed as a post-processing unit, it is necessary to determine which channel the recognition target speech is being output to.
In ICA, for example, when there are N microphones, N channel input is performed, and N channel separation results are output. However, the number of sound sources varies. When the number of sound sources is smaller than the number of input channels, the output channels (sound source channels) corresponding to the sound sources and the output channels (reverberation channels) where reverberation-like sounds that do not correspond to any sound source are observed are obtained as observation information. It will be.

When processing combining ICA and speech recognition is considered, ICA output channels can be classified as follows.
(1) Sound source channels that support actual sound sources (2) Reverberation channels that do not support sound sources

Further, (1) sound source channels can be subdivided as follows.
(1-1) A channel that is speech (1-1-1) An utterance channel that includes content assumed by the speech recognition system as an input (utterance within a task)
(1-1-2) An utterance channel that includes content that the speech recognition system does not assume as input (utterance outside task)
(1-2) Channels other than voice (including chats between people who are not input to the system, for example)

As a system for performing speech recognition based on the sound source separation result by ICA,
(1-1-1) An utterance channel including utterances assumed by the voice recognition system as input (utterance in task)
It is important to recognize the voice of the above channel (utterance in task).

As a method of selecting a channel corresponding to such a target sound source, for example, there are the following methods.
(A) Selection is made based on the magnitude of power (volume).
This is a method of discriminating whether a channel is a target sound source channel or a reverberation channel based on the power value of each channel output, and selecting a channel with the maximum power.
(B) Estimate the sound source direction and select the one closest to the front.
This is a method of performing ICA and simultaneously estimating a sound source arrival direction and selecting a channel from which a sound source closest to the front direction is output as a target sound.
(C) Selection is made by comparing voice / non-voice discrimination with past data.
For example, it is determined whether or not the sound of each channel is a human voice signal, and the channel determined to be the channel of the human voice signal is compared with the past frequency feature value held. In this method, the voice of a specific person is discriminated. This technique is described in, for example, Japanese Patent Application Laid-Open No. 2007-279517.

Summary of problems in the conventional system For example, in the system combining the sound source separation processing based on ICA and the speech recognition processing shown in FIG. The problem is whether to decide whether to select the target voice from the plurality of generated channels.

The problems in the conventional system are summarized and listed as follows.
(A) Problems of applying speech recognition after channel selection (A1) When only one channel is selected,
When a plurality of sounds are sounding, a sound other than the target sound may be selected.
(A2) When selecting multiple channels:
A plurality of speech recognition results are obtained, but it is necessary to select again among them.

(B) Problems of the conventional method of channel selection The problems of the three conventional methods described above are listed.
(A) Problems with the channel selection method based on the magnitude of power There is a possibility that a sound source other than sound may be selected by mistake only with the magnitude of power. For example, a sound source channel and a reverberation channel can be distinguished, but voice and non-voice cannot be distinguished.
(B) Problems of the method of estimating the sound source direction and selecting the one closest to the front The target speech does not necessarily come from the front.
(C) Problems of the method of selecting by a combination of voice / non-voice discrimination and comparison with past data In voice / non-voice discrimination, it is not possible to discriminate whether or not the utterance content of a task assumed by the voice recognition system.
While speech signals and other utterances can be distinguished, intra-task utterances and non-task utterances cannot be distinguished.
As described above, the conventional channel selection method has various problems.

JP 2006-238409 A JP 2009-53088 A JP 2007-279517 A

“Introduction and Independent Component Analysis” (Noboru Murata, Tokyo Denki University Press) “19.2.4. Fourier transform method” in “Detailed analysis of independent components”

  The present invention has been made in view of such circumstances, and performs separation processing for each sound source signal unit using independent component analysis (ICA), and further performs speech recognition processing of the target sound. An object of the present invention is to provide a voice recognition device, a voice recognition method, and a program.

The first aspect of the present invention is:
A sound source separation unit that separates a mixed signal output from a plurality of sound sources into a signal corresponding to each sound source to generate a separated signal of a plurality of channels;
A plurality of channels separated signals generated by the sound source separation unit are input to perform speech recognition processing, generate speech recognition results corresponding to each channel, and additional information serving as evaluation information for the speech recognition results corresponding to each channel. A voice recognition unit to be generated;
A speech recognition apparatus having a channel selection unit that inputs the speech recognition result and the additional information, calculates a score of the speech recognition result corresponding to each channel by applying the additional information, and selectively outputs a speech recognition result having a high score It is in.

  Furthermore, in an embodiment of the speech recognition apparatus of the present invention, the speech recognition unit calculates a recognition reliability of a speech recognition result as the additional information, and the channel selection unit applies the recognition reliability to each The score of the speech recognition result corresponding to the channel is calculated.

  Furthermore, in an embodiment of the speech recognition apparatus of the present invention, the speech recognition unit includes, as the additional information, in a task indicating whether the speech recognition result is a recognition result related to a task assumed in the speech recognition apparatus. The utterance level is calculated, and the channel selection unit calculates the score of the speech recognition result corresponding to each channel by applying the intra-task utterance level.

  Furthermore, in an embodiment of the speech recognition apparatus of the present invention, the channel selection unit determines whether or not the speech recognition result is a recognition reliability, or whether the speech recognition result is a recognition result related to a task assumed in the speech recognition apparatus. Is applied as score calculation data, and a score is calculated by combining at least one of voice power and sound source direction information.

  Furthermore, in one embodiment of the speech recognition apparatus of the present invention, the speech recognition unit is constituted by a plurality of speech recognition units equal to the number of channels of the separated signals of the plurality of channels generated by the sound source separation unit. The plurality of speech recognition units are configured to input a separation signal corresponding to each channel of the separation signals of the plurality of channels generated by the sound source separation unit, and execute speech recognition processing in parallel.

Furthermore, the second aspect of the present invention provides
A speech recognition method executed in a speech recognition device,
A sound source separation unit that separates a mixed signal output from a plurality of sound sources into a signal corresponding to each sound source to generate a separated signal of a plurality of channels;
The speech recognition unit inputs the separation signals of the plurality of channels generated by the sound source separation unit, executes speech recognition processing, generates a speech recognition result corresponding to each channel, and evaluation information of the speech recognition result corresponding to each channel A speech recognition step for generating additional information to be
A channel selection step in which a channel selection unit inputs the speech recognition result and the additional information, calculates a score of the speech recognition result corresponding to each channel by applying the additional information, and selectively outputs a speech recognition result having a high score When,
There is a speech recognition method.

Furthermore, the third aspect of the present invention provides
A program for executing voice recognition processing in a voice recognition device,
A sound source separation step for causing the sound source separation unit to generate a multi-channel separation signal by separating the mixed signal of the output of the plurality of sound sources into a signal corresponding to each sound source;
The speech recognition unit inputs the separation signals of the plurality of channels generated by the sound source separation unit and executes speech recognition processing to generate a speech recognition result corresponding to each channel, and evaluation information of the speech recognition result corresponding to each channel A speech recognition step for generating additional information to be
A channel selection step of inputting the voice recognition result and the additional information to a channel selection unit, calculating a score of the voice recognition result corresponding to each channel by applying the additional information, and selectively outputting a voice recognition result having a high score When,
It is in the program that executes

  The program of the present invention is a program that can be provided by, for example, a storage medium or a communication medium provided in a computer-readable format to an image processing apparatus or a computer system that can execute various program codes. By providing such a program in a computer-readable format, processing corresponding to the program is realized on the information processing apparatus or the computer system.

  Other objects, features, and advantages of the present invention will become apparent from a more detailed description based on embodiments of the present invention described later and the accompanying drawings. In this specification, the system is a logical set configuration of a plurality of devices, and is not limited to one in which the devices of each configuration are in the same casing.

  According to the configuration of an embodiment of the present invention, a separation signal is generated by processing that applies independent component analysis (ICA) to an observation signal that is a mixed signal obtained by mixing outputs from a plurality of sound sources, and each separation is performed. Perform voice recognition processing on the signal. Further, additional information is generated as evaluation information of the speech recognition result. As additional information, the recognition reliability of the speech recognition result and the intra-task utterance level indicating whether or not the speech recognition result is a recognition result related to a task assumed in the speech recognition apparatus are calculated. By applying these additional information, the score of the speech recognition result corresponding to each channel is calculated, and the recognition result having a high score is selected and output. By these processes, sound source separation and speech recognition for mixed signals from a plurality of sound sources are realized, and a necessary recognition result can be obtained more reliably.

It is a figure explaining the situation where different sounds are sounding from N sound sources and observing them with n microphones. It is a figure explaining the example of a system which combined the sound source separation process by general independent component analysis (ICA), and the speech recognition part. It is a figure explaining the whole structure and the outline | summary of a process of the speech recognition apparatus of this invention. It is a figure explaining the detailed structure of the sound source separation part 202, and the specific example of a process. It is a figure which shows the structure of one speech recognition part of the speech recognition parts 203-1 to N provided corresponding to each channel. It is a figure explaining the detailed structure of the channel selection part 204, and the specific example of a process. It is a figure which shows the flowchart which shows the flow of the whole process which the speech recognition apparatus of this invention performs. It is a figure which shows the flowchart which shows the detail of the speech recognition process of step S103 in the flow shown in FIG. It is a figure which shows the flowchart which shows the detail of the channel selection process of step S104 in the flow shown in FIG.

The details of the speech recognition apparatus, speech recognition method, and program of the present invention will be described below with reference to the drawings. The description will be made according to the following items.
1. 1. Outline of overall configuration example and processing of speech recognition apparatus of present invention 2. Detailed configuration of sound source separation unit and specific example of processing 3. Detailed configuration of voice recognition unit and specific example of processing 4. Detailed configuration of channel selection unit and specific example of processing About the sequence of processing executed by the speech recognition device

[1. Example of overall configuration and processing outline of speech recognition apparatus of present invention]
First, with reference to FIG. 3, the overall configuration and processing outline of the speech recognition apparatus of the present invention will be described. The speech recognition device of the present invention is a device that performs sound source separation by inputting a mixed signal of sounds output from a plurality of sound sources, and further performs speech recognition processing using a sound source separation result.
FIG. 3 shows a configuration example of a speech recognition apparatus 200 according to an embodiment of the present invention.

Sound is collected by the plurality of microphones 201-1 to 201-N, and an input waveform corresponding to the sound signal acquired by each of the microphones 201-1 to 20-N is sent to the sound source separation unit 202. The sound source separation unit 202 performs a process of separating a mixed sound of a plurality of sound sources into individual sound sources derived from each sound source by, for example, independent component analysis (ICA) (Independent Component Analysis). By this separation processing, for example, a separation waveform of sound originating from each sound source is generated and output.
In addition to the sound source separation process, the sound source separation unit 202 also executes a process for estimating the sound source direction from which the sound corresponding to each separated waveform arrives.

  Note that N separated waveforms corresponding to the number of inputs (N) are generated by the separation processing by independent component analysis (ICA) executed by the sound source separation unit 202. Here, the number (N) of separated waveforms is the number of channels. The sound source separation unit 202 generates an N-channel separation waveform of channel 1 to channel N. However, the number of sound sources is not necessarily equal to N. In some cases, only a part of the N channel outputs a sound separation waveform corresponding to a specific sound source, and the other is a channel composed of only noise.

Each of the plurality of separated waveforms derived from each sound source generated by the sound source separation unit 202 is individually output to the channel selection unit 204, and further input to the speech recognition units 203-1 to 203-N set individually for each separated waveform. Is done.
A plurality of sound source direction information derived from each sound source generated by the sound source separation unit 202 is individually output to the channel selection unit 204.

  Each of the speech recognition units 203-1 to 203-N performs a speech recognition process on each of the separated waveforms output from the sound source separation unit 202. Each of the speech recognition units 203-1 to 203-N adds the reliability of the recognition result and the degree of utterance within a task (intra-task utterance level) as additional information to the channel selection unit 204 along with the speech recognition result. Output.

  The intra-task utterance degree is a degree of whether or not the utterance of the task assumed by the speech recognition apparatus 200 is assumed. Specifically, for example, when the device including the speech recognition device 200 is a television, when an operation request for the television, for example, a volume (volume) change request or a channel change request is included in the speech recognition result. It is highly possible that the utterance is in the task, and the information with the utterance level in the task set to high is output. In this determination process, a statistical language model held in a memory in the speech recognition apparatus 200 is used. The statistical language model is data in which an index value indicating whether or not the word is related to a task is set in advance for various words.

The channel selection unit 204 inputs a separation waveform corresponding to each sound source from the sound source separation unit 202, and from each of the speech recognition units 203-1 to 203-N,
Speech recognition results corresponding to each separated waveform,
Additional information (reliability of recognition result and utterance level in task),
Enter this information.

  The channel selection unit 204 applies these input information to select and output a speech recognition result of a channel including the target sound.

3 is executed under the control of a control unit not shown in FIG. The control unit is constituted by a CPU or the like, and executes a program stored in a storage unit (not shown) to control processing of each component unit shown in FIG.
A detailed configuration of each component shown in FIG. 3 and a specific example of processing to be executed will be described with reference to FIG.

[2. Detailed configuration of sound source separation unit and specific processing example]
First, a detailed configuration of the sound source separation unit 202 and a specific example of processing will be described with reference to FIG.
As shown in FIG. 4, the sound source separation unit 202 includes an A / D conversion unit 301, a short-time Fourier transform (FT) unit 302, a signal separation unit 303, an inverse Fourier transform (FT) unit 304, and a D / A conversion unit 305. And a sound source direction estimating unit 306.

  Individual input waveforms from the microphones 201-1 to 201-N are converted into digital observation signals by the A / D conversion unit 301 and input to the short-time Fourier transform (FT) unit 302.

  A short-time Fourier transform (FT) unit 302 performs short-time Fourier transform (FT) processing on the input signal converted into a digital signal, converts the input signal into a spectrogram, and inputs the spectrogram to the signal separation unit 303. Note that the spectrogram of each observation signal obtained by this short-time Fourier transform (FT) process is the signal of equation [2.1] described above, that is, X (ω, t).

  The signal separation unit 303 receives the spectrogram of each observation signal generated by the short-time Fourier transform (FT) unit 302 and executes the above-described independent component analysis (ICA) to generate the separation result Y. This separation result becomes N separation results corresponding to the number of N channels. This separation result Y is input to an inverse Fourier transform (FT) unit 304.

  The inverse Fourier transform (FT) unit 304 performs an inverse Fourier transform process on the spectrogram corresponding to each sound source signal, converts the spectrogram to a signal in the time domain, and the sound source separation estimated to correspond to each sound source. Generate a signal. This separated signal is generated as the number of channels, that is, N signals.

  These N separated signals are input to the D / A converter 305 and converted into N separated waveforms as analog signals by D / A conversion. Each of the N separated waveforms is output to the voice recognition units 203-1 to 203 -N corresponding to the channels 1 to N and the channel selection unit 204.

  The sound source direction estimation unit 306 estimates the arrival direction of each independent signal using a part of the estimation result from the signal separation unit 303. This estimation information is also N sound source direction information corresponding to channels corresponding to the number of N channels. The N pieces of sound source direction information generated by the sound source direction estimation unit 306 are output to the channel selection unit 204.

[3. Detailed configuration of speech recognition unit and specific processing example]
Next, with reference to FIG. 5, a detailed configuration and specific example of processing of the speech recognition units 203-1 to 203 -N will be described. FIG. 5 is a diagram showing a configuration of one voice recognition unit of the voice recognition units 203-1 to 203-N provided corresponding to each channel. Each of the N speech recognition units 203-1 to 203-N has the configuration shown in FIG.

As illustrated in FIG. 5, the speech recognition unit 203 includes an A / D conversion unit 401, a feature amount extraction unit 402, a speech recognition processing unit 403, and an additional information calculation unit 407. The additional information calculation unit includes a recognition reliability calculation unit 408 and an intra-task utterance level calculation unit 409.
In the speech recognition unit 203, an acoustic model 404, an in-task statistical language model 405, and an out-of-task statistical language model 406 are stored, and processing using these data is executed.

  The input of the speech recognition unit 203 shown in FIG. 5 is one separated waveform corresponding to one channel k (k = 1 to N) among the N channels separated by the sound source separation unit 202. Each of the speech recognition units 203-1 to 203-1 to N inputs the separated waveform of the channel k (k = 1 to N), and executes speech recognition processing based on the separated waveform of each channel in parallel.

  As described above, the speech recognition units 203-1 to 203-N perform processing on N separated waveforms of N channels in parallel. With reference to FIG. 5, the process for the separated waveform corresponding to one channel will be described.

A separated waveform corresponding to one channel is first input to the A / D converter 401.
The A / D conversion unit 401 converts the separated waveform, which is an analog signal, into a digital observation signal. The digital observation signal is input to the feature amount extraction unit 402.

  The feature amount extraction unit 402 receives the digital observation signal from the A / D conversion unit 401 and extracts the feature amount used for speech recognition from the digital observation signal. The feature amount extraction process can be executed according to an existing speech recognition algorithm. The extracted feature amount is input to the speech recognition processing unit 403.

The speech recognition processing unit 403 executes speech recognition processing using the feature amount input from the feature amount extraction unit 402.
In addition to the acoustic model 404, the speech recognition processing unit 403 performs a plurality of recognitions using different language models of speech recognition processing using the in-task statistical language model 405 and speech recognition processing using the non-task statistical language model 406. Execute the process.

For example, a word registered in the in-task statistical language model 405 is compared with a word obtained as a speech recognition processing result, a matching word is selected, a recognition result is acquired, and a score corresponding to the degree of matching is further obtained. Is calculated.
Further, the word registered in the non-task statistical language model 406 is compared with the word obtained as a result of the speech recognition processing, the matching word is selected, the recognition result is obtained, and the score corresponding to the degree of matching is further obtained. Is calculated.
A result with the highest recognition score is selected from a plurality of recognition results using the different models and output as a speech recognition result.
Note that a plurality of different models can be used as the in-task statistical language model 405 and the out-task statistical language model 406.

  The voice recognition result generated by the voice recognition processing unit 403 is output to the channel selection unit 204 and also output to the additional information calculation unit 407 in the voice recognition unit 203. The information output to the additional information calculation unit 407 includes the above score information.

The additional information calculation unit 407 includes a recognition reliability calculation unit 408 and an intra-task utterance level calculation unit 409.
The recognition reliability calculation unit 408 calculates the recognition reliability of the speech recognition result generated by the speech recognition processing unit 403. The recognition reliability of the speech recognition result is determined, for example, by evaluating the validity of the recognized word sequence using evaluation reference data stored in advance in a memory. Specifically, for example, the recognition reliability can be calculated by applying the configuration described in JP-A-2005-275348.

  The in-task utterance level calculation unit 409 calculates the in-task utterance level of the speech recognition result generated by the speech recognition processing unit 403. The intra-task utterance degree is a degree of whether or not the utterance of the task assumed by the speech recognition apparatus 200 is, as described above. Specifically, for example, when the device including the speech recognition device 200 is a television, a word included in the speech recognition result generated by the speech recognition processing unit 403 is an operation request to the television, for example, a change in volume (volume). In the case of a word such as a request or a channel change request, there is a high possibility that it is an utterance within a task, and the degree of utterance within a task increases. When the speech recognition result includes many words unrelated to such a task, the intra-task utterance level is set low.

As a specific process, the intra-task utterance level can be calculated by a process using the score obtained by the process of the voice recognition processing unit 403 described above.
That is, the first score corresponding to the degree of coincidence between the word obtained as a result of the speech recognition processing and the registered word of the in-task statistical language model 405,
A second score corresponding to the degree of coincidence between the word obtained as a result of the speech recognition processing and the registered word of the statistical language outside task 406;
Perform a comparison of
When the first score is higher than the second score, the intra-task utterance level is set high, and when the second score is higher than the first score, the intra-task utterance level is set low.

  The additional information calculation unit 407 uses the recognition reliability calculated by the recognition reliability calculation unit 408 and the intra-task utterance degree calculated by the intra-task utterance degree calculation unit 409 as additional information corresponding to the speech recognition result. Output to.

[4. Detailed configuration of channel selector and specific example of processing]
Next, a detailed configuration of the channel selection unit 204 and a specific example of processing will be described with reference to FIG.
As illustrated in FIG. 6, the channel selection unit 204 includes channel score calculation units 501 to 1 to N and a selection channel determination unit 502.

Channel score calculation units 501-1 to 501 -N are provided corresponding to the respective channels 1 to N. Each of the channel score calculation units 501-1 to 501 -N has as channel correspondence information,
Speech recognition result from voice recognition unit 203 and additional information (recognition reliability and utterance level in task)
From the sound source separation unit 202, a separated waveform, sound source direction information,
Enter this information.

The channel score calculation units 501-1 to 501-N calculate the score of the speech recognition result of each channel using the channel correspondence information.
For example,
Recognition reliability = p
In-task utterance level = q
Power of separated waveform = r
And
The recognition reliability = p is such that the higher the reliability, the larger the value of p,
In-task utterance level = q, the higher the possibility of in-task utterance, the larger the value of q,
The separation waveform power = r is set to a larger value r as the power (volume) is larger.

In this case, the score Sk of the channel k is
Sk = ap + bq + cr
Calculate as
However, a, b, and c are preset coefficients (weighting coefficients).

Furthermore, considering the sound source direction, as an evaluation value that the sound source direction becomes higher as the front of the device,
Sound source direction evaluation value = h
Use
Sk = ap + bq + cr + dh
It is good also as a structure calculated as.
However, a, b, c, and d are preset coefficients (weight coefficients).

  These channel-corresponding scores Sk (k = 1 to N) are calculated by the respective channel score calculation units 501-1 to 501-N and input to the selected channel determination unit 502.

  The selected channel determination unit 502 inputs scores S1 to SN corresponding to each of the N channels input from the channel score calculation units 501-1 to 501-N, executes a comparison process of these scores, and selects a high score channel. A speech recognition result is selected and output as a recognition result.

  Note that the selected channel determination unit 502 outputs M recognition results set in advance from the recognition results of the channels with high scores. The output number M has a configuration that can be set by the user from the outside, for example.

The selected channel determination unit 502 outputs the recognition results for the top M channels of the score as the selected recognition results. The value of the selected channel number M is set according to the usage mode. For example, if there is only one user, only one utterance can be input at the same time.
M = 1
And A value larger than 1 is set when there is a possibility that a plurality of people input utterances at the same time.

[5. About the sequence of processing executed by the speech recognition device]
Next, a processing sequence of processing executed by the speech recognition apparatus of the present invention will be described with reference to the flowchart in FIG.

The flowchart shown in FIG. 7 is a flowchart showing the overall flow of processing executed by the voice recognition apparatus of the present invention. FIG. 8 is a flowchart showing details of the voice recognition processing in step S103 in the flow shown in FIG.
FIG. 9 is a flowchart showing details of the channel selection processing in step S104 in the flow shown in FIG.

  Note that the processing according to the flowcharts shown in FIGS. 7 to 9 is executed under the control of a control unit configured by a CPU or the like as described above. The control unit executes a program stored in the storage unit, thereby appropriately outputting a command or the like to each component unit described with reference to FIGS. The process according to the flowchart is executed.

  First, the overall flow of processing executed by the speech recognition apparatus of the invention will be described with reference to the flowchart shown in FIG. The processing of each processing step will be described in correspondence with the configuration diagram of FIG.

  In step S101, voice input processing from the microphones 201-1 to 201-N is performed. Sound is collected and input using N microphones arranged at various positions. If there are N microphones, an N-channel input waveform is obtained.

  In step S102, a sound source separation process is executed. This is the process of the sound source separation unit 202 shown in FIG. 3, and corresponds to the process described with reference to FIG. As described above with reference to FIG. 3, the sound source separation unit 202 performs sound source separation processing by ICA on the input waveforms for N channels to generate separated waveforms for N channels. In this process, sound source direction information corresponding to the separation waveform of each channel may be acquired.

The process of the next step S103 is a voice recognition process. This voice recognition process is a process executed in the voice recognition units 203-1 to 203-N shown in FIG. 3, and corresponds to the process described with reference to FIG. In the speech recognition process in step S103, a speech recognition result corresponding to each channel, a recognition reliability as additional information, and an intra-task utterance level are generated.
Details of the voice recognition processing in step S103 will be described later with reference to the flowchart of FIG.

The next step S104 is a channel selection process. This channel selection processing is processing performed in the channel selection unit 204 shown in FIG. 3, and corresponds to the processing described with reference to FIG. In the channel selection process in step S104, a score corresponding to the channel is calculated from the result of the speech recognition process and the additional information, and the highest score is selected.
The details of the channel selection process in step S104 will be described later with reference to the flowchart of FIG.

  The next step S105 is recognition result output processing. This recognition result output processing is also processing performed in the channel selection unit 204 shown in FIG. 3, and corresponds to the processing described with reference to FIG. In the recognition result output process in step S105, M speech recognition results are output in descending order of the channel-corresponding scores calculated in step S104 according to the preset output number (M).

  Next, a detailed sequence of the speech recognition process in step S103 in the flowchart of FIG. 7 will be described with reference to the flowchart shown in FIG. This voice recognition process is a process executed in the voice recognition units 203-1 to 203-N shown in FIG. 3, and corresponds to the process described with reference to FIG.

  Here, the process in the k channel among the channels 1 to N (the process of the speech recognition unit 203-k) will be described. Since there is no dependency between channels in the speech recognition process, each speech recognition can be processed sequentially or in parallel.

In step S201, the data of the output channel k, which is the separation processing result of the sound source separation unit 202, is received.
In step S202, feature amount extraction processing is executed. This feature quantity extraction process is a process of the feature quantity extraction unit 402 shown in FIG. The feature amount extraction unit 402 extracts a feature amount used for speech recognition from the observation signal.

  Next, in step S203, voice recognition processing is executed. This voice recognition process is a process of the voice recognition processing unit 403 shown in FIG. As described above, the speech recognition processing unit 403 applies different language models of the speech recognition process using the in-task statistical language model 405 and the speech recognition process using the non-task statistical language model 406 in addition to the acoustic model 404. A plurality of recognition processes are executed.

  Next, in step S204, a reliability calculation process is executed. This reliability calculation process is a process executed by the recognition reliability calculation unit 408 of the additional information calculation unit 407 shown in FIG.

  The recognition reliability calculation unit 408 calculates the recognition reliability of the speech recognition result generated by the speech recognition processing unit 403. For example, the recognition reliability is calculated using evaluation reference data in which the validity of the recognized word sequence is stored in advance in the memory.

  Next, in step S205, an intra-task utterance degree calculation process is executed. This intra-task utterance degree calculation process is a process executed by the intra-task utterance degree calculation unit 409 of the additional information calculation unit 407 shown in FIG.

  The in-task utterance level calculation unit 409 calculates the in-task utterance level of the speech recognition result generated by the speech recognition processing unit 403. If the word included in the speech recognition result generated by the speech recognition processing unit 403 includes many words related to the task, there is a high possibility that it is an intra-task utterance and the intra-task utterance level is high. When the speech recognition result includes many words unrelated to such a task, the intra-task utterance level is set low.

The voice recognition unit 203 follows the flowchart shown in FIG. 8 as data corresponding to each channel.
Speech recognition results,
Additional information (recognition reliability, utterance level in task)
These data are generated and supplied to the channel selection unit 204.

  Next, a detailed sequence of the channel selection process in step S104 in the flowchart of FIG. 7 will be described with reference to the flowchart shown in FIG. This channel selection process is a process executed by the channel selection unit 204 shown in FIG. 3, and corresponds to the process described with reference to FIG.

In step S301, output list initialization processing is performed. The output list is a list in which recognition results of channels 1 to N are arranged in descending order of score. The selected channel determination unit 502 shown in FIG. 6 selects and outputs a predetermined number (M) of recognition results from the high score recognition results according to the output list.
In step S301, the output list is initialized, that is, the list is reset.

The processing of the next steps S302 to S304 is a loop processing that is repeatedly executed corresponding to the data of channels k = 1 to N.
In step S303, the score corresponding to channel k is calculated.
The calculation of the score is, for example, as described above,
Recognition reliability = p
In-task utterance level = q
Power of separated waveform = r
The score Sk of channel k is
Sk = ap + bq + cr
Calculate as
However, a, b, and c are preset coefficients (weighting coefficients).
Or consider the direction of the sound source,
Sound source direction evaluation value = h
Use
Sk = ap + bq + cr + dh
Calculate as
With such processing, the score of channel k is calculated.

  In steps S302 to S304, N scores S1 to SN corresponding to speech recognition results corresponding to N channels 1 to N are calculated.

  Finally, in step S305, a number of recognition results corresponding to a predetermined number of outputs (M) are selected and output from the top of the channel score. This process is the process of the selected channel determination unit 502 shown in FIG.

  The selected channel determination unit 502 inputs scores S1 to SN corresponding to each of the N channels input from the channel score calculation units 501-1 to 501-N, executes a comparison process of these scores, and selects a high score channel. A speech recognition result is selected and output as a recognition result.

  Thus, in the speech recognition apparatus of the present invention, speech recognition is applied to each output channel of sound source separation by ICA, and a channel corresponding to the target sound is selected based on the result. ICA output channel selection error problem by assigning information on reliability of speech recognition result and whether or not the speech is within the task assumed by the speech recognition apparatus and performing channel selection based on the additional information Can be eliminated.

As the effects of the processing executed by the speech recognition apparatus of the present invention, for example, there are the following effects.
(A) By utilizing the reliability of voice recognition, the problem of erroneously selecting a channel other than the target voice is solved.
(B) With settings that do not use information on the sound source direction, channel selection independent of the direction of arrival of the target speech is possible.
(C) By using the information on whether the utterance content is in-task, it is possible to reject disturbing speech that the speech recognition system does not assume as input.
Such an effect can be produced.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. In other words, the present invention has been disclosed in the form of exemplification, and should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims should be taken into consideration.

  The series of processing described in the specification can be executed by hardware, software, or a combined configuration of both. When executing processing by software, the program recording the processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various processing. It can be installed and run. For example, the program can be recorded in advance on a recording medium. In addition to being installed on a computer from a recording medium, the program can be received via a network such as a LAN (Local Area Network) or the Internet and can be installed on a recording medium such as a built-in hard disk.

  Note that the various processes described in the specification are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. Further, in this specification, the system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same casing.

  As described above, according to the configuration of one embodiment of the present invention, the separated signal is obtained by processing that applies independent component analysis (ICA) to an observation signal that is a mixed signal obtained by mixing outputs from a plurality of sound sources. And a speech recognition process for each separated signal. Further, additional information is generated as evaluation information of the speech recognition result. As additional information, the recognition reliability of the speech recognition result and the intra-task utterance level indicating whether or not the speech recognition result is a recognition result related to a task assumed in the speech recognition apparatus are calculated. By applying these additional information, the score of the speech recognition result corresponding to each channel is calculated, and the recognition result having a high score is selected and output. By these processes, sound source separation and speech recognition for mixed signals from a plurality of sound sources are realized, and a necessary recognition result can be obtained more reliably.

DESCRIPTION OF SYMBOLS 101 Microphone 102 Sound source separation part 103 Channel selection part 104 Speech recognition part 200 Speech recognition apparatus 201 Microphone 202 Sound source separation part 203 Speech recognition part 204 Channel selection part 301 A / D conversion part 302 Short-time Fourier transform (FT) part 303 Signal separation Unit 304 inverse Fourier transform (FT) unit 305 D / A conversion unit 306 sound source direction estimation unit 401 A / D conversion unit 402 feature quantity extraction unit 403 speech recognition processing unit 404 acoustic model 405 statistical language model in task 406 statistical language outside task Model 407 Additional information calculation unit 408 Recognition reliability calculation unit 409 In-task utterance level calculation unit 501 Channel score calculation unit 502 Selected channel determination unit

Claims (7)

A sound source separation unit that separates a mixed signal output from a plurality of sound sources into a signal corresponding to each sound source to generate a separated signal of a plurality of channels;
A plurality of channels separated signals generated by the sound source separation unit are input to perform speech recognition processing, generate speech recognition results corresponding to each channel, and additional information serving as evaluation information for the speech recognition results corresponding to each channel. A voice recognition unit to be generated;
A speech recognition apparatus having a channel selection unit that inputs the speech recognition result and the additional information, calculates a score of the speech recognition result corresponding to each channel by applying the additional information, and selectively outputs a speech recognition result having a high score . The voice recognition unit
Calculate the recognition reliability of the speech recognition result as the additional information,
The channel selector
The speech recognition apparatus according to claim 1, wherein a score of a speech recognition result corresponding to each channel is calculated by applying the recognition reliability. The voice recognition unit
As the additional information, an in-task utterance degree indicating whether or not the speech recognition result is a recognition result related to a task assumed in the speech recognition apparatus is calculated,
The channel selector
The speech recognition apparatus according to claim 1, wherein a score of a speech recognition result corresponding to each channel is calculated by applying the intra-task utterance degree. The channel selector
At least one of the recognition reliability of the speech recognition result or the speech utterance degree indicating whether or not the speech recognition result is a recognition result related to the task assumed in the speech recognition apparatus is applied as score calculation data, and the speech The speech recognition apparatus according to claim 1, wherein the score is calculated by combining at least one of power and sound source direction information. The voice recognition unit
A plurality of speech recognition units equal to the number of channels of the plurality of separated signals generated by the sound source separation unit.
The plurality of voice recognition units are
The speech recognition apparatus according to any one of claims 1 to 4, wherein a speech recognition process is performed in parallel by inputting a separation signal corresponding to each channel of the separation signals of a plurality of channels generated by the sound source separation unit. A speech recognition method executed in a speech recognition device,
A sound source separation unit that separates a mixed signal output from a plurality of sound sources into a signal corresponding to each sound source to generate a separated signal of a plurality of channels;
The speech recognition unit inputs the separation signals of the plurality of channels generated by the sound source separation unit, executes speech recognition processing, generates a speech recognition result corresponding to each channel, and evaluation information of the speech recognition result corresponding to each channel A speech recognition step for generating additional information to be
A channel selection step in which a channel selection unit inputs the speech recognition result and the additional information, calculates a score of the speech recognition result corresponding to each channel by applying the additional information, and selectively outputs a speech recognition result having a high score When,
A speech recognition method comprising: A program for executing voice recognition processing in a voice recognition device,
A sound source separation step for causing the sound source separation unit to generate a multi-channel separation signal by separating the mixed signal of the output of the plurality of sound sources into a signal corresponding to each sound source;
The speech recognition unit inputs the separation signals of the plurality of channels generated by the sound source separation unit and executes speech recognition processing to generate a speech recognition result corresponding to each channel, and evaluation information of the speech recognition result corresponding to each channel A speech recognition step for generating additional information to be
A channel selection step of inputting the voice recognition result and the additional information to a channel selection unit, calculating a score of the voice recognition result corresponding to each channel by applying the additional information, and selectively outputting a voice recognition result having a high score When,
A program that executes

Images