KR20060020363A - Apparatus and method for recognizing a voice for an audio-visual (av) system - Google Patents

Abstract

The present invention is to accurately recognize the user's voice. An echo sound removing unit for receiving sounds including echo sound, user voice, and noise of the AV system and removing the echo sound from the received sounds based on user voice related information; A voice detector for detecting a voice of the user from the received sounds and generating the user voice related information based on the detected user voice; And a voice recognition unit for comparing the detected user voice with voice patterns included in at least one model to recognize the detected user voice.

Echo, adaptive filter

Description

Apparatus and method for recognizing a voice for an audio-visual (AV) system}

1 is a view showing an AV device including a speech recognition apparatus of the present invention

2 is a block diagram showing the configuration of a speech recognition apparatus of the present invention;

3 is a view illustrating in detail the echo cancellation unit of FIG.

4 is a flowchart illustrating a speech recognition method of the present invention.

Fig. 5 shows an example of a detected voice waveform of a user.

Explanation of symbols on the main parts of the drawings

21: echo cancellation unit 22: voice detection unit

23: speech recognition unit 24: memory

211: first filter bank 212: second filter bank

213: third filter bank 215: adaptive filter unit

The present invention relates to voice recognition, and more particularly, to a voice recognition device and a method for recognizing a user's command for controlling a device such as an AV (audio-visual) device.

In general, in order to control household appliances such as TV and audio, a remote controller or a button provided for the household appliance must be pressed. When using a remote control, the remote control should always be near the user and hassle-free operation by hand. In addition, when the remote control is lost, the user must directly operate the home appliance. In particular, the use of the remote control is difficult even for the disabled or the elderly. In order to solve this problem, voice recognition technology is applied to home appliances.

Speech recognition is a technology that communicates the human will to a machine or computer and acts according to the voice as the most natural means of communication. While such speech recognition can provide a lot of convenience to people in various fields at present, it is still difficult to control AV devices using speech recognition. When using an AV device, not only the ambient noise but also the sound output from the AV device and the distance between the AV device and the user are not close, so the accuracy of speech recognition is not high. As such, there are problems to overcome in order to accurately control the AV device according to the voice of the user.

The present invention has been made to solve the above-mentioned problems of the prior art, and an object thereof is to provide a speech recognition apparatus and method capable of accurately recognizing a user's voice even in an environment in which echo and noise exist.

The speech recognition apparatus of the present invention for solving the above technical problem, receives the sound including the echo sound, user voice, noise of the AV system and the echo sound from the received sounds based on the user voice related information Echo cancellation unit for removing; A voice detector for detecting a voice of the user from the received sounds and generating the user voice related information based on the detected user voice; And a voice recognition unit for comparing the detected user voice with voice patterns included in at least one model to recognize the detected user voice.

The echo canceling unit includes: a first filter bank for dividing an output sound of the AV system by frequency bands; A second filter bank for dividing the received sounds into frequency bands; Adaptive filters for removing echoes from the sounds divided by frequency bands using the signals output from the first filter bank; And a third filter bank integrating the signals output from the adaptive filters.

The echo canceling unit controls at least one of cutoff frequency, pass frequency, and echo cancellation rate of the filters included in the echo canceling unit based on the generated user voice related information. The user voice related information includes a start point timing and an end point timing of the user voice, and further includes at least one of a frequency band, an amplitude, and a waveform of the user voice.

The speech recognition unit calculates a first probability that the detected user's voice matches the voice pattern included in the first model and a second probability that the detected user's voice matches the voice pattern included in the second model. . Here, the first model includes voice patterns corresponding to preset words, and the second model accumulates voice patterns corresponding to non-set words.

The speech recognition unit calculates a probability that the speech patterns included in the first model coincide with the detected user's voice, and determines the largest probability among the calculated probabilities as the first probability. The probability that the speech patterns included in the two models coincide with the detected user's voice are respectively calculated, and the largest probability among the calculated probabilities is determined as the second probability.

The speech recognition unit determines whether to recognize the detected user's speech based on a ratio of the first probability and the second probability. The speech recognition unit compares the ratio of the first probability to the second probability with reference values, and recognizes the detected user's voice according to the comparison result.

The speech recognition method of the present invention comprises the steps of: (a) receiving sounds including echoes, user voices and noises of an AV system; (b) detecting a user voice from the received sounds and generating user voice related information based on the detected user voice; (c) removing echoes by passing the received sounds through a plurality of filters controlled according to the user voice related information; (d) comparing the detected user's voice with voice patterns included in at least one model.

In the removing of the echo, the parameter values of the filters are fixed during a period between the start point timing and the end point timing of the user voice. Herein, the parameters of the filters include at least one of a cutoff frequency, a pass frequency, and an echo cancellation rate of the filters. In the removing of the echo sound, at least one of a cutoff frequency, a pass frequency, and an echo cancellation rate of the filters is controlled based on the user voice related information.

Comparing the detected voice of the user with the voice patterns included in at least one model, the first probability that the detected user's voice matches the voice pattern included in the first model and the detected user's voice Calculating a second probability that the speech will match the speech pattern included in the second model; Calculating a ratio of the first probability and the second probability; Comparing the ratio of the first probability to the second probability with reference values; And recognizing the voice of the detected user according to the comparison result.

According to another aspect of the present invention, there is provided a speech recognition apparatus, comprising: a voice detector for detecting a voice of a user from received sounds; Calculate a first probability that the detected user's voice matches the voice pattern included in the first model, and a second probability that the detected user's voice matches the voice pattern included in the second model; And a voice recognition unit recognizing the detected user's voice according to a ratio of a probability and the second probability.

Another form of speech recognition method in accordance with the present invention comprises the steps of: (a) detecting a voice of a user from received sounds; (b) calculating a first probability that the detected user's voice matches the voice pattern included in the first model and a second probability that the detected user's voice matches the voice pattern included in the second model Wow; (c) recognizing the voice of the detected user according to the ratio of the first probability and the second probability.

The calculating of the first and second probabilities may include calculating a probability that the voice patterns included in the first model match the detected voice of the user; And determining the largest probability among the calculated probabilities as the first probability.

The calculating of the first and second probabilities may include: calculating a probability that voice patterns included in the second model match the detected voice of the user; And determining the largest probability among the calculated probabilities as the second probability.

Recognizing the voice of the detected user according to the ratio of the first probability and the second probability comprises: comparing the ratio of the first probability and the second probability with first and second reference values; And determining whether to recognize the detected user's voice based on the comparison result. For example, when the ratio of the first probability and the second probability is greater than or equal to the first reference value, the voice of the detected user is recognized, and the ratio of the first probability and the second probability is the first reference value. If smaller and greater than or equal to the second reference value, a word corresponding to the detected user's voice is displayed on the screen. If the ratio between the first probability and the second probability is smaller than the second reference value, the detected user's voice is not recognized.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 shows an example of an AV device including a voice recognition device of the present invention, and FIG. 2 shows a configuration of the voice recognition device of FIG. As shown in FIG. 1, the speech recognition apparatus 20 receives a user's voice along with echo and noise. The reflection sound is a sound of the TV 10 reflected by a wall or an object. In order to most effectively recognize a user's voice, the voice recognition device 20 may be located in front of the TV 10.

As shown in FIG. 2, the microphone of the TV 10 or the voice recognition device 20 receives various sounds such as TV echo, noise, and the like, and converts them into frequency signals. The echo canceling unit 21 receives the frequency signals through the microphone of the TV 10 and outputs only a user's voice signal from the frequency signals. Low volume noise can be easily separated from the user's voice, but high volume TV echoes are difficult to separate. Accordingly, as shown in FIG. 3, the echo canceling unit 21 of the present invention uses IIR filter banks 211, 212, and 213 to effectively separate / remove TV echoes. Include. The first filter bank 211 has M channel filters H0, H1, H2, and HM-1 in order to divide audio signals among broadcast signals by frequency band (sub band). The second filter bank 212 has M channel filters H0, H1, H2, and HM-1 for dividing the signals received from the microphone by frequency band. , H2 ,,, HM-1) pass signals of different frequency bands, respectively. Each of the adaptive filters W0, W1, W2, and WM-1 of the adaptive filter unit 215 receives the signals separated by the first and second filter banks 211 and 212. Each of the adaptive filters W0, W1, W2,, and WM-1 receives a signal corresponding to a corresponding frequency band from the first and second filter banks 211 and 212. For example, the adaptive filter W0 receives a signal output from the channel filter H0 of the first filter bank 211 and the channel filter H0 of the second filter bank 212. The adaptive filters W0, W1, W2,, WM-1 compare the signals output from the first filter bank 211 and the second filter bank 212 to detect TV echo. Look for signals with the same or similar frequency and amplitude. Among the signals output from the second filter bank 212, the coincidence or similar signals are regarded as a reverberation signal, and the adaptive filters W0, W1, W2,, and WM-1 are converted into the second filter bank. Remove the echo signal from the signals received from 212. As a result, the adaptive filter unit 215 outputs signals for each frequency band that do not include the echo sound signal. The third filter bank 213 receives signals from which the echo signal is removed for each frequency band from the adaptive filter unit 215 and integrates the signals for each frequency band. The integrated signal includes a user's voice received through a microphone, and may sometimes include noise. Thus, the present invention may include a noise canceller for removing noise and residual reflections. The noise canceller filters the signal output from the third filter bank 213 according to a set threshold.

The voice detector 22 receives a signal output from the echo canceling unit 21, detects a voice of the user from the received signal, and extracts voice related information. For example, to detect a voice section, the voice detector 22 determines a start point and an end point of the voice, and detects voice features such as frequency band, amplitude, and waveform of the voice. The voice detector 22 provides the voice related information to the echo canceling unit 21 and the voice recognition unit 23.

The memory 24 stores statistical information of patterns corresponding to speech in the form of a probability model. When the user's voice is detected, the voice recognition unit 23 compares the detected user's voice with the probabilistic models stored in the memory 24 and calculates the probability (similarity) of each model being the detected voice. do. Then, it is determined whether the voice of the user is a command and what command the voice of the user is based on the calculated probability. The controller 11 controls the TV 10 according to a command corresponding to the voice of the user.

Hereinafter, the speech recognition method according to the present invention will be described in detail. 4 is a flowchart illustrating a speech recognition method of the present invention. When the user speaks a command (eg, "volume-down") while watching TV 10, the command "volume-down" is provided to the echo canceling unit 21 through the microphone. At this time, echo and noise are input to the echo canceling unit 21 together with the user's voice.

The user's voice, echo, and noise input to the echo canceling unit 21 are separated according to frequency bands, and the separated echoes are applied to the adaptive filters W0, W1, W2, and WM-1. The noise is removed by the noise canceller (S30). The echo canceling unit 21 removes the echoes using a subband adaptive filtering method using the IIR filter banks 211, 212, and 213.

The voice detector 22 receives the signal from which the echo and noise are removed from the echo canceler 21, and as shown in FIG. 5, detects the user's voice from the received signal. An audio waveform is detected for each frame (S31). In addition, the voice detector 22 detects a start point and an end point of the voice, determines their timing, and detects a frequency band, amplitude, and the like of the voice. The voice detector 22 compares the energy of the voice with predetermined thresholds to detect the start and end points of the voice. For example, if the energy of the signal input to the voice detector 22 is greater than the first threshold value, it is determined as the start point of the voice, and the energy of the signal input to the voice detector 22 is the second threshold value for a predetermined period. If smaller, it is determined as the end point of the voice.

When the start point of the voice is detected, the adaptive filter unit 215 of the echo canceling unit 21 stops adaptation. When the end point of the voice is detected, the adaptive filter unit 215 starts the adaptation again. Here, the adaptation is a real-time estimation of the echo path of the sound output from the TV 10, and the filtering parameter value, for example, the adaptive filters according to the changing echo path (for example, due to the movement of the viewer). The cutoff frequency, the pass frequency, and the echo cancellation rate of (W0, W1, W2 ,, WM-1) are changed.

However, since the correct filtering parameter value is not known when the user's voice and the echo sound are received together with the echo canceling unit 21, the filtering parameter value of the adaptive filter unit 215 is fixed and only the echo sound is received. Only when entering the echo canceling unit 21, the filtering parameter value of the adaptive filter unit 215 is changed according to the echo path.

The voice recognition unit 23 calculates a probability that the detected user's voice is a key word (preset word) and a non-core word (non-preset word) (S32). In order to obtain a keyword probability, the speech recognizer 23 sets the detected speech in predetermined speech patterns (eg, "channel change", "volume-up", "volume-down") included in the first model. , &Quot; external input ", and calculate the probability (i.e. similarity) that each speech pattern is the detected speech. Here, the first model is a hidden markov model (HMM) trained using voices corresponding to each word. The speech recognition unit 23 determines the largest probability among the calculated probabilities as the keyword probability. In addition, the speech recognition unit 23 uses a second model to obtain a non-core word probability. Here, the second model is a filler model in which non-preset voice patterns are accumulated and formed. The speech recognizer 23 calculates probabilities (similarities) of the speech patterns of the second model as the detected speech. The speech recognition unit 23 determines the largest probability among the calculated probabilities as the non-core word probability.

The speech recognizer 23 calculates a ratio of the calculated key word probability and the non-key word probability (key word probability / non-key word probability), and compares the ratio of the key word probability and the non-key word probability with a first reference value and a second reference value. (S33, S35). The first reference value is a ratio of the key word probability and the non-core word probability when the probability of malfunction (misrecognition of speech recognition) is 0.5%, and the second reference value is the key word probability and ratio when the probability of the malfunction is 5%. The ratio of key word probabilities. The first and second reference values are obtained through experiments.

When the ratio of the key word probability and the non-key word probability is greater than or equal to the first reference value, the voice recognition unit 23 recognizes the detected user's voice. For example, the speech recognizer 23 identifies a speech pattern having the greatest probability (similarity) among the speech patterns of the first model, and sends a command corresponding to the confirmed speech pattern to the controller 11. To pass. Then, the controller 11 controls the TV 10 according to the received command (S34).

If the ratio of the key word probability and the non-key word probability is less than the first reference value and greater than or equal to the second reference value, the speech recognition unit 23 does not immediately recognize the detected user's voice and provides the user with the user's voice. The word corresponding to the voice is displayed on the screen. For example, the voice recognition unit 23 checks the voice pattern having the greatest probability (similarity) among the voice patterns of the first model, and displays the checked voice pattern on the screen. Request (S36). Then, the controller 11 displays the confirmed voice pattern, for example, "volume-down" on the screen, and the user waits for confirmation. If the user's approval such as "yes", "OK", "select", etc. is received (S37), the controller 11 decreases the volume of the TV 10 (S38). When a user's voice such as "No", "cancel", or a new user's command is received, the control unit 11 deletes the voice pattern "volume-down" displayed on the screen.

When the ratio of the key word probability and the non-key word probability is smaller than the second reference value, the speech recognizer 23 does not find a speech pattern that matches the detected user's voice and does not provide any signal to the controller 11. (S39) Thus, the controller 11 does not respond to the detected voice of the user.

As described above, the present invention recognizes the user's voice based on the ratio of the key word probability and the non-key word probability, so that only the user's voice is correctly recognized and the corresponding command is performed even in the presence of echo and noise.

The present invention can be applied not only to AV equipment, but also to various fields such as automatic interpretation devices, various home appliances, mobile phones, toys, and the like.

Those skilled in the art will appreciate that various changes and modifications can be made without departing from the spirit of the present invention. Therefore, the technical scope of the present invention should not be limited to the contents described in the embodiments, but should be defined by the claims.

Claims (35)

An echo sound removing unit for receiving sounds including echo sound, user voice, and noise of the AV system and removing the echo sound from the received sounds based on user voice related information; A voice detector for detecting a voice of the user from the received sounds and generating the user voice related information based on the detected user voice; And a speech recognizer configured to compare the detected user's voice with voice patterns included in at least one model to recognize the detected user's voice. The method of claim 1, The echo sound removing unit, A first filter bank for dividing the output sound of the AV system by frequency bands; A second filter bank for dividing the received sounds into frequency bands; Adaptive filters for removing echoes from the sounds divided by frequency bands using the signals output from the first filter bank; And a third filter bank for integrating the signals output from the adaptive filters. The method of claim 1, And a noise canceller for removing noise and residual echo from the echo-removed sounds. The method of claim 1, The echo sound removing unit, And at least one of a cutoff frequency, a pass frequency, and an echo cancellation rate of the filters included in the echo canceling unit based on the generated user speech related information. The method of claim 1, And the user voice related information comprises a start point timing and an end point timing of the user voice. The method of claim 5, wherein And the user voice related information further comprises at least one of a frequency band, an amplitude, and a waveform of the user voice. The method of claim 1, The speech recognition unit, Calculating a first probability that the detected user's voice matches the voice pattern included in the first model and a second probability that the detected user's voice matches the voice pattern included in the second model. Speech recognition device. The method of claim 7, wherein The speech recognition unit, And determining whether to recognize the detected user's voice based on a ratio of the first probability and the second probability. The method of claim 7, wherein The first model includes voice patterns corresponding to preset words. The method of claim 7, wherein The second model accumulates speech patterns corresponding to words that are not preset. The method of claim 7, wherein The speech recognition unit, And comparing the ratio of the first probability to the second probability with reference values, and recognizing the detected user's voice according to the comparison result. The method of claim 7, wherein The speech recognition unit, A voice, characterized in that for calculating the probability that the voice patterns included in the first model to match the detected user's voice, and determine the largest probability among the calculated probabilities as the first probability Recognition device. The method of claim 7, wherein The speech recognition unit, A voice, characterized in that for calculating the probability that the voice patterns included in the second model to match the detected user's voice, and determine the largest probability among the calculated probabilities as the second probability Recognition device. (a) receiving sounds including echoes, user voices, noises in the AV system; (b) detecting a user voice from the received sounds and generating user voice related information based on the detected user voice; (c) removing echoes by passing the received sounds through a plurality of filters controlled according to the user voice related information; (d) comparing the detected user's voice with voice patterns included in at least one model. The method of claim 14, In the step of removing the echo, And a parameter value of the filters is fixed during a period between a start point timing and an end point timing of the user voice. The method of claim 15, The parameter of the filters, at least one of the cut-off frequency and pass frequency of the filters, the echo cancellation rate. The method of claim 14, Removing noise and residual echo from the echo-removed sounds. The method of claim 14, In the step of removing the echo, And controlling at least one of a cutoff frequency, a pass frequency, and an echo cancellation rate of the filters based on the user voice related information. The method of claim 14, Comparing the detected voice of the user with voice patterns included in at least one model, Calculating a first probability that the detected user's voice matches the voice pattern included in the first model and a second probability that the detected user's voice matches the voice pattern included in the second model. Speech recognition method, characterized in that. The method of claim 19, Calculating a ratio of the first probability and the second probability; Comparing the ratio of the first probability to the second probability with reference values; And recognizing the detected voice of the user according to the comparison result. The method of claim 19, The first model includes voice patterns corresponding to preset words. The method of claim 19, The second model accumulates speech patterns corresponding to words that are not preset. A voice detector for detecting a voice of the user from the received sounds; Calculate a first probability that the detected user's voice matches the voice pattern included in the first model, and a second probability that the detected user's voice matches the voice pattern included in the second model; And a speech recognizer configured to recognize the detected voice of the user based on a ratio between a probability and the second probability. The method of claim 23, The first model includes voice patterns corresponding to preset words. The method of claim 23, The second model accumulates speech patterns corresponding to words that are not preset. The method of claim 23, The speech recognition unit, And comparing the ratio of the first probability to the second probability with reference values, and recognizing the detected user's voice according to the comparison result. (a) detecting a user's voice in received sounds; (b) calculating a first probability that the detected user's voice matches the voice pattern included in the first model and a second probability that the detected user's voice matches the voice pattern included in the second model Wow; and (c) recognizing the voice of the detected user according to a ratio of the first probability and the second probability. The method of claim 27, The first model includes voice patterns corresponding to preset words. The method of claim 27, The second model accumulates speech patterns corresponding to words that are not preset. The method of claim 27, Computing the first and second probabilities, Calculating a probability that voice patterns included in the first model match the detected voice of the user; And determining the largest probability among the calculated probabilities as the first probability. The method of claim 27, Computing the first and second probabilities, Calculating a probability that the voice patterns included in the second model match the detected voice of the user; And determining the largest probability among the calculated probabilities as the second probability. The method of claim 27, Recognizing the voice of the detected user according to the ratio of the first probability and the second probability, Comparing the ratio of the first probability to the second probability with first and second reference values; And determining whether to recognize the detected user's voice based on the comparison result. The method of claim 32, And if the ratio between the first probability and the second probability is greater than or equal to the first reference value, recognizing the voice of the detected user. The method of claim 32, And when the ratio of the first probability and the second probability is less than the first reference value and greater than or equal to the second reference value, displaying a word corresponding to the detected user's voice on the screen. The method of claim 32, If the ratio of the first probability and the second probability is less than the second reference value, the detected voice of the user is not recognized.

Images