KR101253610B1 - Apparatus for localization using user speech and method thereof - Google Patents

Abstract

The present invention relates to a location tracking device using the user's voice and a method thereof, and to remove scattering noise from each sound source signal separated by a sound source separation unit for separating the input two-channel sound source signal for each sound source, Stereo winner filter for filtering the residual signal components for sound source location tracking, a speech recognition unit for recognizing the user's voice and measuring the reliability of the speech recognition result, the speech recognition result and the speech recognition from the speech recognition unit And a channel selector for selecting the target channel based on the reliability of the result, and a sound source position tracker for analyzing the signal of the target channel and the signal of the interference channel to track the sound source position. According to the present invention, by integrating an implicit sound source separation technology, a stereo winner filter technology, a voice recognition and speech verification technology, and a sound source location tracking technology, there is an advantage that the user voice location tracking that is more accurate and robust to the surrounding environment is possible.

Description

Apparatus for localization using user speech and method

The present invention relates to a location tracking device using a user voice and a method thereof, and to overcome the problems of performance degradation of voice recognition and speaker location tracking due to additional noise and echo in the home, and thus more accurate sound source location using the user voice. The present invention relates to a location tracking device using a user voice and a method for enabling tracking.

The present invention is derived from the research conducted as part of the IT growth engine technology development industry of the Ministry of Knowledge Economy [Task management number: 2006-S-036-04, Task name: Development of large-capacity interactive distributed processing voice interface technology for new growth engine industry] ].

Since there are various kinds of additional noise sources in the domestic environment in which we live daily life, the speaker's voice signal spoken in the home environment is originally characterized by the additional noise sources. There are many cases of this damage.

Additional sources of noise commonly found in home environments include multimedia electronic devices such as televisions, audio and acoustic devices such as radios, various game devices, vacuum cleaners, water facilities, refrigerators, air conditioners, and other domestic appliances. have.

It is very difficult for current digital signal processing technology to track the position of a voice speaker from a voice signal damaged by such an additional noise source.

In addition, when the voice is uttered in the indoor space of the home, which is our residential space, a reverberation signal component generated by reflecting the voice signal on the wall or the furniture is generated in addition to the original voice signal.

The characteristics of reverberation sound are not kept constant even in the same indoor environment due to the influence of the size of the room, the material of the wall, the arrangement of the furniture and its materials, the location of the speaker, the temperature and humidity.

These echoes create a virtual speech signal component in addition to the speaker's original speech signal component, and in addition to additional noise, the automatic speech recognition system and the speaker in the home environment It is known to be a major cause of degrading the performance of the speaker localization system.

The technology of removing echo components from voice signals contaminated by additional noise is difficult to solve even at the current technology level, and thus, the generalized cross-correlation (GCC) method, phase-transform (PHAT) method, etc. Even if you apply this, the performance is greatly reduced.

In addition, a sound source separation algorithm has been recently applied to track the location of a user sound source that is robust against additional noise and echo noise. In this method, a linear approximation of the surrounding environment is performed by a signal mixing filter to separate sound sources.

Then, each sound source is separated using the inverse transform filter of the signal mixing filter thus obtained, the voice of the main speaker is detected, and the channel output including the voice signal of the main speaker is selected.

Then, the selected target channel signal is restored to the multi-channel input signal using the signal mixing filter used for sound source separation. Finally, the target speaker is found by applying the generalized cross-correlation (GCC) method and the phase-transform (PHAT) method.

When the algorithm is applied, a large amount of computation is required for the process of finding the output channel of the target speaker and restoring the multi-channel signal by applying the voice signal and the signal mixture filter of the target speaker.

An object of the present invention for solving the above problems is, automatic speech recognition (ASR), speech using a two-channel microphone, blind source separation (BSS), Hidden Markov Model (HMM) Location tracking using user's voice to organically integrate verification (UV) technology and sound source localization (SSL) technology to overcome the problem of speech recognition degradation caused by in-house, side noise and echo An apparatus and a method thereof are provided.

In addition, another object of the present invention is to solve the problem of speech recognition performance degradation due to additional noise and echo, and location tracking apparatus and method using the user's voice to enable more accurate sound source location tracking using the user's voice In providing.

The position tracking device using the user's voice according to the present invention for achieving the above object, a sound source separation unit for separating the input two-channel signal for each sound source, each sound source signal separated by the sound source separation unit A stereo winner filter unit for removing scattering noise from the filter and filtering the signal to emphasize residual signal components, a voice recognition unit for recognizing a user's voice from each of the sound source signals, and measuring a reliability of a voice recognition result, the voice recognition unit A channel selector which selects a target channel based on a speech recognition result from the speech recognition result and a reliability of the speech recognition result, and analyzes a signal of a target channel selected by the channel selector and a signal of an interference channel to track a sound source position And a sound source position tracking unit.

The sound source separating unit is configured to separate the signals of the two channels for each sound source by using an implicit sound source separating technique.

The stereo winner filter unit may filter each of the sound source signals using a stereo winner filter technology.

The stereo Wiener filter unit may include: a voice activity detector detecting frame energy based voice activity on an input sound source signal, a Wiener filter coefficient estimator estimating a Wiener filter coefficient based on a signal input from the voice activity detector; And a Wiener filter unit for performing stereo Wiener filtering on the signal of the input channel by using the estimated Wiener filter coefficients, and a signal reconstruction unit for reconstructing the signal of each filtered channel.

The apparatus may further include a voice endpoint detector configured to detect an endpoint of the voice signal from the signal of each channel.

The voice recognition unit may include a feature extractor that extracts a voice feature for voice recognition from a sound source signal of each channel, and a voice recognition that recognizes a user's voice with respect to a signal of each channel based on the voice feature extracted from the feature extractor. And a speech verification unit for measuring a speech recognition reliability of the speech recognition result from the speech recognition decoder and verifying the speech recognition result from the speech recognition decoder.

The channel selector may select a corresponding channel as a target channel when the user's voice is detected only in one of the sound source signals of each channel, and the reliability of the voice recognition result is higher than a threshold.

The channel selector, when all of the user's voices are detected in the sound source signals of the respective channels, the target channel is a channel having a higher reliability of the speech recognition result than the threshold value, and a reliability of the speech recognition result among the signals of the two channels. Characterized in that the selection.

And a voiced sound frame detector for detecting voiced sound frames from signals of the target channel selected by the channel selector.

The voiced sound frame detector includes at least one of a change time frequency characteristic, a high frequency to low frequency band energy ratio, a zero crossing ratio, a level crossing ratio, a normalized autocorrelation maximum value, a voiced sound probability, a peak to valley ratio of an autocorrelation function, and an AMDF minimum value. The voiced sound frame is detected using one voiced sound feature.

The voiced sound frame detection unit estimates energy by extracting a frame from a signal of an input target channel, and calculates energy in a specific frequency region based on a result of power estimation for the extracted frame. The change time frequency characteristic is obtained from the set energy.

The voiced sound frame detector is configured to calculate a voice feature ratio by comparing the voiced voice feature with a threshold value, and if the calculated voice feature ratio is larger than a predefined threshold value, determining the corresponding voice frame as a voiced voice frame. It features.

The apparatus may further include a band pass filter that filters the voice frequency section with respect to the signal of the target channel selected by the channel selector and the signal of the interference channel not selected as the target channel.

On the other hand, the location tracking method using the user's voice for achieving the above object, the step of separating the input two-channel signal for each sound source, filtering each of the sound source signal separated in the sound source separation step Detecting an end point of a voice from the respective sound source signals, and recognizing a voice using the detected signal, based on a voice recognition result of the step of recognizing the voice and a reliability of the voice recognition result. Selecting a target channel, and analyzing the voiced sound frame detected from the signal of the target channel, and the voice frequency section of the target channel and the interference channel to track the sound source position.

The filtering may include detecting frame energy-based voice activity for each input sound source signal, estimating a Wiener filter coefficient based on the voice activity detection result and the PSD spectrum estimation result, and the estimated Wiener. And performing stereo winner filter on each of the input sound source signals using a filter coefficient, and restoring the respective stereo winner signal filtered by the sound source signal.

The method may further include detecting end points of a voice signal from the sound source signals of the respective channels.

The recognizing of the voice may include extracting a voice feature for voice recognition from a sound source signal of each channel, recognizing a voice of a user with respect to the sound source signal of each channel based on the extracted voice feature, and And verifying the speech recognition result by measuring the speech recognition reliability of the speech recognition result.

The selecting of the target channel may include selecting a corresponding channel as a target channel when the user's voice is detected only in one of the sound source signals of each channel, and the reliability of the voice recognition result is higher than a threshold value. When the user's voice is detected in the sound source signal of each channel, the reliability of the voice recognition result is higher than the threshold value, and among the signals of the two channels, the channel having high reliability of the voice recognition result is selected as a target channel. do.

The method may further include detecting a voiced sound frame from a signal of the target channel selected in the selecting of the target channel, wherein detecting the voiced sound frame comprises comparing a voiced sound feature with a threshold value. ), And if the calculated voice feature ratio is greater than a predefined threshold, the corresponding frame is determined as a voiced sound frame.

The method may further include filtering to emphasize a voice frequency section with respect to the signal of the selected target channel and the signal of the interference channel not selected as the target channel in the selecting of the target channel.

According to the present invention, interference noise, scattering noise, and reflection sound are organically integrated by integrating an implicit sound source separation technology, a stereo winner filter technology, a voice recognition and speech verification technology, and a sound source location technology that emphasizes human voice signal components. It solves the problem of speech recognition deterioration and the performance degradation of the speaker location tracking algorithm, thereby enabling the user voice location tracking that is more accurate and robust to the surrounding environment.

Hereinafter, specific embodiments of the present invention will be described with reference to the accompanying drawings.

Location tracking apparatus using the user's voice and the method according to the present invention is a two-channel microphone, BSS (Blind Source Separation, BSS) technology, Automatic Speech Recognition (ASR) using HMM (Hidden Markov Model) The technology, Utterance Verification (UV) technology, and Sound Source Localization (SSL) technology are organically integrated to overcome the problem of performance degradation of speaker location tracking by additional noise and echo in the home.

First, BSS (Blind Source Separation) technology is a multi-channel input, multi-channel output (MIMO) type of sound source separation and sound quality enhancement technology. It is used to improve the sound quality for the signal.

At this time, the tacit sound separation technology is mainly developed to solve various cocktail party problems.

Here, the Cocktail party problem refers to the ability to concentrate on the voice of a specific speaker when several speakers talk at the same time as in a cocktail party.

The principle of the implicit sound source separation technique is to estimate the environmental parameters from the multi-channel input signals and filter by using an inverse filter, thereby filtering the sound source. Each of these is to restore the original signal.

In addition, each output channel has an interference signal component called an interference signal called a residual signal.

Therefore, in the present invention, the location of the user is tracked using the user voice signal obtained by applying Real-time Blind Source Separation (RT BSS) technology and the residual signal components of the user voice signal.

In addition, the implicit sound source separation technology applied to the location tracking device using the user's voice according to the present invention applies a 2channel time-domain independent component analysis (TD-ICA) algorithm, and further improves the sound quality of the separated user voice signal. Stereo Wiener Filter technology is applied to emphasize the user's voice signal remaining in the interference channel.

The location tracking apparatus using the user's voice according to the present invention can solve the problem of performance degradation of the speaker location tracking algorithm by the additional noise and the echo by using the implicit sound source separation technique as the front-end processor. .

For a detailed description of the location tracking apparatus using the user's voice according to the present invention, refer to the following embodiments in conjunction with the drawings.

First, Figure 1 is a block diagram referred to explain the configuration of a location tracking device using the user's voice according to the present invention.

As shown in Figure 1, the position tracking apparatus using the user's voice according to the present invention is a two-channel microphone, sound source separation unit 10, stereo winner filter unit 20, 30, end point extraction unit 40, 50 , Voice recognition unit 60, 70, channel selector 80, channel buffering unit 90, voiced sound frame detection unit 100, band pass filter unit 110, 120, and sound source position tracking unit 130. Include.

First, the sound source separating unit 10 separates a signal in which several sound sources of two channels input from a two-channel microphone are mixed for each sound source. At this time, the sound source separation unit 10 is separated by applying an implicit sound source separation technology based on time-domain independent component analysis (TD-ICA) algorithm. At this time, each sound source separated by the sound source separation unit 10 is transmitted to the stereo winner filter unit 20, 30.

In addition, the stereo winner filter unit 20, 30 filters each sound source signal separated by the sound source separation unit 10. In this case, the stereo winner filter units 20 and 30 include a first filter unit 20 and a second filter unit 30.

Here, the stereo winner filter units 20 and 30 filter the input sound source signal by using the stereo wiener filter. In this case, the stereo winner filter units 20 and 30 remove the scattered noise remaining after the sound source is separated and emphasize the residual signal components remaining in other channels.

For example, the first filter unit 20 estimates the Wiener filter coefficient using the channel 1 as a reference signal, and applies the same to the output signals of the channels 1 and 2 separated by the sound source separation unit 10 to the channel. Emphasize the residual signal component of channel 1 remaining in channel 2.

In contrast to the first filter unit 20, the second filter unit 30 estimates Wiener filter coefficients using channel 2 as a reference signal and separates the channel 1 separated by the sound source separation unit 10. In this case, the residual signal component of the channel 2 remaining in the channel 1 is emphasized by applying the same to the output signal of the 2.

The stereo winner filter units 20 and 30 remove scattered noise that degrades the performance of automatic speech recognition from each of the separated sound source signals, and emphasizes residual signal components remaining in each sound source signal, In recognition of speech, more accurate speech recognition results can be obtained. Detailed operations of the stereo winner filter units 20 and 30 will be described with reference to FIG. 2.

Meanwhile, the endpoint detectors 40 and 50 include a first endpoint detector 40 that detects and buffers the endpoint from the signal of channel 1, and a second endpoint detector 50 that detects and buffers the endpoint from the signal of channel 2. do.

The first and second end point detectors 40 and 50 detect the end points of the voices using the signals of the respective channels input from the stereo winner filter units 20 and 30, and the signals of each channel from which the endpoints are detected. And buffers the signals of the corresponding other channels.

For example, the first endpoint detector 40 detects an end point of the voice by using the signal of channel 1 and buffers the signal of channel 1 and the corresponding channel 2 signal at which the endpoint is detected. .

On the other hand, the second end point detector 50 detects the end point of the voice by using the signal of the channel 2, and buffers the signal of the channel 2 and the channel 1 signal corresponding to the end point detected. In the future, the signals buffered by the first endpoint detector 40 and the second endpoint detector 50 are used to track the location of the user.

In addition, the voice recognition units 60 and 70 recognize voices from the signals of the respective channels where the endpoints are detected by the endpoint detection units 40 and 50 using an automatic voice recognition technique. In this case, the voice recognition units 60 and 70 include a first voice recognition unit 60 that recognizes the voice from the signal of the channel 1 and a second voice recognition unit 70 that recognizes the voice from the signal of the channel 2. .

Since the signal input to the voice recognition units 60 and 70 is a signal having an improved sound quality since the implicit sound source separation technique and the Wiener filter technique are already applied, the signal is more easily recognized.

Meanwhile, the channel selector 80 selects a channel including the user's voice from the signals of the two channels input from the first voice recognition unit 60 and the second voice recognition unit 70.

At this time, the channel selector 80 includes a target channel including the user's voice based on the voice recognition result and the voice recognition reliability measurement values of the first and second voice recognition units 60 and 70. Select.

If a voice is detected in only one channel and the result of the automatic voice recognition is higher than the threshold value, the channel selector 80 selects the channel as the target channel.

On the other hand, when voice is detected simultaneously in two channels, the channel selector 80 selects a channel having a higher reliability than a threshold value as a result of automatic voice recognition and having a relatively high reliability among the two channels as a target channel.

Detailed operations of the channel selector 80 will be described with reference to FIGS. 3 and 4.

When the target channel is selected by the channel selector 80, the channel buffering unit 90 buffers an output signal from the stereo Wiener filter of the selected target channel.

If channel 1 is selected as the target channel, buffer the signal filtered by the first filter unit 20, and if channel 2 is selected as the target channel, buffer the signal filtered by the second filter unit 30. do.

The voiced sound frame detection unit 100 detects the voiced sound frame from the signal buffered by the channel buffering unit 90. In this case, the voiced sound frame detector 100 detects the voiced sound frame from the target channel signal.

The detailed operation of the voiced sound frame detector 100 will be described with reference to FIG. 5.

The band pass filter units 110 and 120 filter each channel signal by using the signal buffered by the channel buffering unit 90. In this case, the band pass filter units 110 and 120 emphasize the frequency section of the voice included in each channel signal.

In addition, the band pass filter unit 110 and 120 may include a first band pass filter unit 110 for filtering a signal of a target channel, and an interference channel (a channel except for a target channel including a user's voice, among the signals of the two channels). And a second band pass filter unit 120 for filtering a signal of the interference channel.

Here, the first band pass filter unit 110 emphasizes the voice frequency range (2 to 4 kHz) for the target channel, and the second band pass filter unit 120 adjusts the voice frequency range (2 to 4 kHz) of the interference channel. Emphasize.

In this case, the equation applied to the voice frequency section for each channel in the band pass filter (110, 120) is as shown in [Equation 1].

The band pass filter unit 110 or 120 transmits a signal in which a voice frequency section of each channel is emphasized to the sound source position tracking unit 130.

Therefore, the sound source position tracking unit 130 receives a signal in which the voice frequency section for each channel is emphasized from the band pass filter unit 110 and 120, so that an accurate position can be tracked in tracking the sound source position. There is this.

In this case, the sound source position tracking unit 130 applies a phase-transform (PHAT) algorithm to track the sound source position of the user.

Here, phase-transform (PHAT) algorithm, "A. Brutti, M. Omologo, and P Svizer, Comparison between different sound source localization techniques based on a real data collection, ◎ in Proc. Joint Workshop on Hands - Free Speech Communication and Microphone Arrays , pp. 69-72, May 2008 ".

As described above, the stereo Wiener filter uses a signal of a predetermined channel as a reference signal, and obtains a Wiener filter-based transfer function based on the reference signal. By applying the Wiener Filter Transfer Function equally to the two-channel signal, the control channel not only improves the sound quality of the control signal but also emphasizes the residual signal components of the control signal remaining in the interference channel. It will play a role. Thus, the two-channel signal thus obtained will be used for speaker location tracking in the future after the target channel is selected.

FIG. 2 is a block diagram referred to to explain the configuration of the stereo winner filter unit according to the present invention, and is a block diagram showing the configuration of the first filter unit in detail.

As shown in FIG. 2, the first filter unit 20 includes a signal framing unit 21, a voice activity detector 22, a Fourier transform unit 23, a PSD spectrum estimation unit 24, and a Wiener filter coefficient weight. And a government unit 25, a winner filter unit 26, and a signal recovery unit 27. Here, the first filter unit 20 receives not only the signal of the channel 1 separated by the sound source separating unit 10 but also the signal of the channel 2.

The signal framing unit 21 includes a first signal framing unit 21a that performs a time-domain signal framing function with a signal of channel 1, and a second signal that performs a time-domain signal framing function with a signal of channel 2. The framing section 21b is included.

In this case, the first signal framing unit 21a transmits the signal of the channel 1 to the voice activity detecting unit 22 and the Fourier transform unit 23. On the other hand, the second signal framing unit 21b transmits the signal of the channel 2 to the winner filter unit (the second winner filter unit 26b).

In addition, the voice activity detection unit 22 performs a voice activity detection (VAD) function based on frame energy for the signal of channel 1.

The Fourier transform unit 23 performs a fast Fourier transform (FFT) function on the signal of the channel 1.

The PSD spectrum estimator 24 performs a function of estimating a (Power Spectral Density, PSD) spectrum from a signal of Fourier transformed channel 1.

The Wiener filter coefficient estimator 25 estimates a Wiener filter coefficient for channel 1 based on the signals input from the voice activity detector 22 and the PSD spectrum estimator 24. . At this time, the winner filter coefficient estimator 25 transmits the estimated winner filter coefficients to the winner filter 26.

In addition, the winner filter unit 26 performs a winner filter function on the input signal. At this time, the Wiener filter unit 26 performs a Wiener filter function on the signal of the channel 1 by using the Wiener filter coefficient estimated from the Wiener filter coefficient estimator 25 and the channel 2 of the first Wiener filter unit 26a. And a second winner filter 26b which performs a winner filter function using the winner filter coefficients estimated by the winner filter coefficient estimator 25.

The signal recovery unit 27 performs a time-domain de-noised signal reconstruction function on the signal filtered by the Wiener filter unit 26. At this time, the signal recovery unit 27 restores the first signal recovery unit 27a for restoring the signal filtered by the first Wiener filter unit 26a and the signal filtered by the second Wiener filter unit 26b. And a second signal recovery unit 27b.

Of course, FIG. 2 illustrates the configuration of the first filter unit 20 in detail, but it is natural that the second filter unit 30 also operates in the same manner. However, in the second filter unit 30, channel 1 and channel 2 are applied oppositely.

3 is a block diagram referred to to explain the configuration of the speech recognition unit according to the present invention. In particular, FIG. 3 illustrates a relationship between the speech recognizer, the endpoint detector, and the channel selector.

As described above, the target channel selection algorithm is used to detect the target channel in which the user's voice signal exists from the output signals of the two channels.

As shown in FIG. 3, the speech recognition unit according to the present invention includes a feature extractor, a speech recognition decoder, and a speech verification unit.

First, the feature extractor includes a first feature extractor 61 that receives and processes a signal of channel 1, and a second feature extractor 71 that receives and processes a signal of channel 2.

In this case, the first feature extractor 61 receives a signal from the first voice endpoint detector 40 which extracts an endpoint from the signal of channel 1. In addition, the second feature extractor 71 receives a signal from the second voice endpoint detector 50 that extracts an endpoint from the signal of channel 1.

Meanwhile, the speech recognition decoder includes a first speech recognition decoder 65 that receives and processes a signal of channel 1, and a second speech recognition decoder 75 that receives and processes a signal of channel 2.

In addition, the speech verification unit (Utterance verification) includes a first speech verification unit 69 for receiving and processing the signal of the channel 1, and a second speech verification unit 79 for receiving and processing the signal of the channel 2.

First, the first voice endpoint detector 40 detects and buffers the voice endpoint from the signal of channel 1 and transfers the buffered signal to the first feature extractor 61.

In this case, the first feature extractor 61 extracts a voice feature for speech recognition from the signal of channel 1. Thereafter, the first feature extractor 61 transmits the extracted voice feature information to the first voice recognition decoder 65.

Meanwhile, the first voice recognition decoder 65 recognizes a user's voice with respect to the signal of the channel 1 based on the voice feature extracted from the first feature extractor 61.

Thereafter, the first speech verification unit 69 verifies the reliability of the speech recognition result from the first speech recognition decoder 65 and verifies the speech signal. Here, the first speech verification unit 69 transmits the verification result of the audio signal of channel 1 to the channel selector 80.

Meanwhile, the second voice endpoint detector 50 detects and buffers the voice endpoint from the signal of channel 1 and transmits the buffered signal to the second feature extractor 71.

In this case, the second feature extractor 71 extracts a voice feature for speech recognition from the signal of channel 2. Thereafter, the second feature extractor 71 transmits the extracted voice feature information to the first voice recognition decoder 75.

Meanwhile, the second voice recognition decoder 75 recognizes the user's voice with respect to the signal of the channel 2 based on the voice feature extracted from the second feature extractor 71.

The second speech verification unit 79 determines the reliability of the speech recognition result from the second speech recognition decoder 75 and verifies the speech signal. The second speech verification unit 79 transmits the verification result of the audio signal of the channel 2 to the channel selector 80.

In this case, the channel selector 80 selects a target channel based on the speech recognition reliability from the first speech verification unit 69 and the second speech verification unit 79 for channels 1 and 2.

An operation of selecting a target channel among channel 1 and channel 2 by the channel selector 80 is described with reference to FIG. 4.

4 illustrates a programming source for selecting a target channel in the channel selector 80.

Referring to FIG. 4, the channel selector 80 determines whether an end point is detected from the channel 1 and the channel 2. If it is determined that both the end points are detected in the channel 1 and the channel 2, the channel selector 80 compares the reliability (CV_ch1, CV_ch2) of the channel 1 and the channel 2 with the threshold Th.

The channel selector 80 compares the CV_ch1 and the CV_ch2 when both the CV_ch1 and the CV_ch2 are larger than the threshold Th, and selects the channel 1 as the target channel when the CV_ch1 is large, and selects the channel 2 as the target channel when the CV_ch2 is large. Choose.

In addition, the channel selector 80 selects channel 1 as the target channel when CV_ch1 is larger than the threshold Th and CV_ch2 is smaller than the threshold Th, and selects channel 2 as the target channel when the CV_ch1 is smaller than the threshold Th. do.

On the other hand, when the endpoint is detected only in channel 1, the channel selector 80 selects channel 1 as the target channel when CV_ch1 is larger than the threshold Th.

If the endpoint is detected only in channel 2, the channel selector 80 selects channel 2 as the target channel if CV_ch2 is larger than the threshold Th.

5 is a block diagram referred to to explain the configuration of the voiced sound frame detection unit 100 according to the present invention.

The voiced sound frame detector 100 detects the voiced sound frame section from the input signal from the channel buffering unit 90. In this case, the voiced sound frame detection unit 100 detects the voiced sound frame using the following eight voiced sound features.

1. Modified time-frequency (TF) feature based on noise robust band energy.

2. High-to-low frequency band energy ratio (HLFBER) at high frequencies.

3. Zero crossing rate (ZCR).

4. Level crossing rate (LCR).

5. Normalized autocorrelation maximum value (NAMV).

6. Voicing probability based on YIN algorithm.

7. Peak-to-valley ratio (PVR) of the autocorrelation function.

8. Minimum magnitude difference function (AMDF) minimum value.

Among these, FIG. 5 shows a configuration that is applied to obtain a band energy-based change time frequency characteristic resistant to noise, which is the first voiced sound characteristic.

As shown in FIG. 5, the voiced sound frame detection unit 100 according to the present invention includes a frame blocking unit 101, an energy estimation unit 102, a Fourier transform unit 103, an FFT power estimation unit 104, and an energy calculation. A government unit 105, an adjustment unit 106, and a quantization unit 107.

First, the frame blocking unit 101 blocks a frame from an input signal from the channel buffering unit 90 and transfers the frame to the energy estimating unit 102 and the Fourier transformer 103.

The energy estimator 102 estimates energy of a time-domain frame among the frames blocked by the frame blocking unit 101. Then, the energy estimated by the energy estimator 102 is used to obtain a band energy based change time frequency characteristic that is resistant to noise together with the energy calculated by the energy calculation unit 105.

Meanwhile, the Fourier transform unit 103 performs a fast Fourier transform (FFT) on the frame blocked by the frame blocking unit 101, and transmits the result to the FFT power estimation unit 104. In this case, the FFT power estimator 104 estimates the power of the fast Fourier transformed frame from the Fourier transform unit 103.

Thereafter, the energy calculation unit 105 calculates energy in a specific frequency region based on the power estimation result of the power estimation unit 104. In other words, the energy calculation unit 105 calculates energy in the 250 to 3600 Hz frequency domain.

The adjusting unit 106 adjusts the energy estimated by the energy estimating unit 102, and adjusts the energy calculated by the energy calculating unit 105.

The quantization unit 107 performs quantization using the energy estimated by the energy estimator 102 and the energy calculated by the energy new government, thereby obtaining a band energy-based change time frequency characteristic resistant to noise.

Meanwhile, the voiced sound frame detector 100 calculates a band energy ratio HLFBER from the high frequency to the low frequency, which is the second voiced sound feature, using Equation 2 below.

As shown in Equation 2, the band energy ratio HLFBER from high frequency to low frequency can be obtained by dividing highbandE, which is a high frequency region energy, by lowbandE, which is a low frequency region energy. At this time, the high frequency region is a 4kHz to 8kHz region, highbandE refers to the energy of the 4kHz to 8kHz region. In addition, the low frequency region is 0kHz to 4kHz region, lowbandE refers to the energy of 0kHz to 4kHz region.

In addition, the third to eighth voiced features can be obtained using a commonly known method, which will not be described in detail.

The voiced sound frame detector 100 calculates a voice feature ratio by comparing the eight voiced voice features detected above with a threshold. In this case, the voiced sound frame detection unit 100 calculates a voice feature ratio, using Equation 3 below.

Here, the Voicing Counter is a value obtained by counting the cases where the feature value can be distinguished as a voiced sound by comparing with a predefined threshold value.

On the other hand, the voiced sound frame detection unit 100 determines that the frame is a voiced sound frame when the voice feature ratio calculated by Equation 3 is higher than a predefined threshold value.

At this time, the predefined threshold and threshold can be obtained by a priori method.

The voiced sound frame detector 100 transmits the voiced sound frame detected by the above-described method to the sound source position tracking unit 130.

Referring to the operation method of the present invention configured as described above are as follows.

6 is a flowchart illustrating an operation flow of a location tracking method using a voice of a user according to the present invention.

As illustrated in FIG. 6, when a signal in which several sound sources of two channels are mixed through a two-channel microphone is input (S600), the sound source separating unit 10 performs an implicit sound separation on the input two-channel signal. (S610). At this time, the sound source separating unit 10 separates the signals of the two channels for each sound source.

When the sound source separation is completed in the 'S610' process, the stereo winner filter unit filters each sound source signal separated in the 'S610' process (S620). In this case, the 'S620' process filters each sound source signal using a stereo witter filter.

In addition, the first end point detector 40 and the second end point detector 50 detect an end point of the voice from each channel signal and then perform buffering (S630). In the process 'S630', the channel where the endpoint is not detected is recognized as an interference channel.

Subsequently, the voice recognition operation is performed using the signal of the channel at which the endpoint is detected in step S630 (S640). The channel selector 80 compares the voice recognition result of step 'S640' for each channel (S650). In operation S660, a target channel is selected. At this time, the channel selector 80 selects a channel including the user's voice as a target channel among the signals of each channel as a result of the speech recognition.

Meanwhile, the channel recognizer excludes the channel for which the endpoint is not detected in the 'S630' process from the target channel selection target.

The output signal of the stereo Wiener filter for the target channel signal selected in step S660 is buffered (S670). In this case, the voiced sound frame detector 100 detects the voiced sound frame from the signal buffered in step S670 (S680).

In addition, the band pass filter unit filters a voice frequency section for each of the target channel signal and the interference channel signal selected in step S660 (S690).

Finally, the sound source position tracking unit 130 tracks the sound source position from the results of the process 'S680' and 'S690' (S700), and outputs the sound source position tracking result (S710).

The sound source position tracking apparatus and method using the user's voice according to the present invention separate the signal for each channel by using the implicit sound source separation technology, and further improve the sound quality of the target channel signal by using a stereo Witter filter, interference The voice signal component of the user present in the channel is also emphasized, making it easy to track the source location.

As described above, the apparatus and method for tracking a location using a user voice according to the present invention are not limited to the configuration and method of the embodiments described above, but the embodiments may be modified in various ways. All or some of the embodiments may be optionally combined.

1 is a block diagram illustrating a configuration of a location tracking apparatus using a voice of a user according to the present invention.

2 is a block diagram showing a detailed configuration of a stereo winner filter unit according to the present invention.

3 is a block diagram showing the configuration of a speech recognition unit according to the present invention.

4 is an exemplary view referred to in describing an operation of a channel selector according to the present invention.

5 is a block diagram showing the detailed configuration of the voiced sound frame detection unit according to the present invention.

6 is a flowchart illustrating an operation flow of a location tracking method using a voice of a user according to the present invention.

Claims (20)

A sound source separation unit for separating the input two-channel signal for each sound source; A stereo winner filter unit for removing scattering noise from each of the sound source signals separated by the sound source separating unit and filtering to emphasize residual signal components; A voice recognition unit recognizing a user's voice from each of the sound source signals and measuring reliability of a voice recognition result; A channel selector which selects a target channel based on a speech recognition result from the speech recognition unit and a reliability of the speech recognition result; And And a sound source position tracker for analyzing a signal of a target channel selected by the channel selector and a signal of an interference channel to track the sound source position. The method according to claim 1, The sound source separation unit, Positioning apparatus using a user's voice, characterized in that the signal of the two channels are separated by each sound source using an implicit sound source separation technology. The method according to claim 1, The stereo winner filter unit, The apparatus of claim 1, wherein the respective sound source signals are filtered using stereo winner filter technology. The method according to claim 1, The stereo winner filter unit, A voice activity detector for detecting a frame energy based voice activity on the input sound source signal; A Wiener filter coefficient estimator for estimating a Witter filter coefficient based on a signal input from the voice activity detector; A winner filter unit for performing stereo winner filter on the input channel signal using the estimated winner filter coefficients; And And a signal recovery unit for restoring the signal of each of the filtered channels. The method according to claim 1, And a voice end point detector for detecting end points of voice signals from the signals of the respective channels. The method according to claim 1, The speech recognition unit, A feature extractor for extracting a voice feature for speech recognition from a sound source signal of each channel; A voice recognition decoder recognizing a user's voice with respect to a signal of each channel based on the voice feature extracted from the feature extractor; And And a speech verification unit for measuring a speech recognition reliability of the speech recognition result from the speech recognition decoder and verifying the speech recognition result from the speech recognition decoder. The method according to claim 1, The channel selector, When the user's voice is detected only in one of the sound source signals of the respective channels, and the reliability of the voice recognition result is higher than the threshold value, the corresponding source is selected as the target channel. Tracking device. The method according to claim 1, The channel selector, When the user's voice is detected in the sound source signal of each channel, the reliability of the voice recognition result is higher than the threshold value, and among the signals of the two channels, the channel having high reliability of the voice recognition result is selected as a target channel. Location tracking device using the user's voice. The method according to claim 1, And a voiced sound frame detector for detecting a voiced sound frame from a signal of the target channel selected by the channel selector. The method of claim 9, The voiced sound frame detector, Utilizes at least one voiced feature of a change-time frequency feature, a high frequency to low frequency band energy ratio, a zero crossing ratio, a level crossing ratio, a normalized autocorrelation maximum value, a voiced sound probability, a peak-to-valley ratio of an autocorrelation function, and an AMDF minimum value Position tracking device using the user's voice, characterized in that for detecting the voiced sound frame. The method of claim 10, The voiced sound frame detector, The energy is extracted by extracting a frame from a signal of an input target channel, and an energy in a specific frequency region is calculated based on a result of power estimation for the extracted frame. Position tracking device using the user's voice, characterized in that the frequency characteristics are obtained. The method of claim 10, The voiced sound frame detector, A voice feature ratio is calculated by comparing the voiced sound feature to a threshold value, and when the calculated voice feature ratio is larger than a predetermined threshold value, the corresponding voice is determined as a voiced sound frame. Position tracking device using. The method according to claim 1, And a band pass filter for filtering the audio frequency section with respect to the signal of the target channel selected by the channel selector and the signal of the interference channel not selected as the target channel. Location tracking device. Separating the input two channel signals for each sound source; Filtering each sound source signal separated in the sound source separation; Detecting an end point of the voice from the respective sound source signals and recognizing the voice using the signal from which the end point is detected; Selecting a target channel based on a voice recognition result of the step of recognizing the voice and a reliability of the voice recognition result; And And analyzing the voiced sound frame detected from the signal of the target channel and the voice frequency section of the target channel and the interference channel to track the location of the sound source. The method according to claim 14, The filtering step, Detecting frame energy-based speech activity for each input sound source signal; Estimating a Wiener filter coefficient based on the voice activity detection result and the PSD spectrum estimation result; Performing stereo Wiener filtering on each of the input sound source signals using the estimated Wiener filter coefficients; And Restoring each of the stereo Wiener filtered sound source signal; location tracking method using the user's voice, characterized in that it further comprises. The method according to claim 14, And detecting end points of the voice signals from the sound source signals of the respective channels. The method according to claim 14, Recognizing the voice, Extracting a speech feature for speech recognition from a sound source signal of each channel; Recognizing a user's voice with respect to a sound source signal of each channel based on the extracted voice feature; And And verifying the speech recognition result by measuring a speech recognition reliability of the speech recognition result. The method according to claim 14, Selecting the target channel, If the user's voice is detected only in one of the sound source signals of the respective channels, and the reliability of the voice recognition result is higher than the threshold, the corresponding channel is selected as the target channel, When the user's voice is detected in the sound source signal of each channel, the reliability of the voice recognition result is higher than the threshold value, and among the signals of the two channels, the channel having high reliability of the voice recognition result is selected as a target channel. Location tracking method using the user's voice. The method according to claim 14, Detecting a voiced sound frame from a signal of the selected target channel in the selecting of the target channel; The detecting of the voiced sound frame may include: comparing the voiced sound feature with a threshold to calculate a voice feature ratio, and if the calculated voice feature ratio is greater than a predefined threshold, converting the frame into a voiced sound frame. Position tracking method using the user's voice, characterized in that the discriminating. The method according to claim 14, And filtering to emphasize the voice frequency section with respect to the signal of the selected target channel and the signal of the interference channel not selected as the target channel in the step of selecting the target channel. Location tracking method.

Images