KR20180062127A - The apparatus and method for communicating between multiple users using voice recognition - Google Patents

Abstract

According to an embodiment of the present invention, a wireless multi-party communication device using voice recognition includes: a voice collection unit which receives signals of the voice spoken by a user; a voice detection unit for detecting voice signals and non-voice signals from the signals of the voice spoken by the user in a separate manner; a voice recognition unit for determining whether the detected voice signals include a predetermined command for executing a voice-operated exchange (VOX) function; and a control unit which enables a communication unit to transmit the signals of the voice spoken by the user to a wireless communication device of another user if the command is included in the detected voice signals and enables the communication unit to receive the signals of the voice spoken by another user if the command is not included in the detected voice signals.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a multi-

More particularly, the present invention relates to a device capable of performing multi-party wireless communication by acquiring a voice right through voice recognition instead of a click of a button in a multi-party conversation situation, and more particularly, It is about the method.

Until recently, various methods of increasing the user interface using voice have been attempted to pursue the convenience of human life. With the development of communication and smart fields, not only the application fields such as mobile phones and home appliances have been widely developed, but also smart car development has progressed, and all the products we use are becoming smart. Especially, when we use refrigerator and TV in the home environment we live in, we do not move directly but control everything by voice, and the range of application from object to temperature and natural environment control is getting wider. As described above, accurate voice detection is important from the time the user speaks the word to the voice recognition and control.

At this time, a technique required for accurate voice detection is Voice Activity Detection (VAD). The voice section detection is a method of detecting a voice signal section and a non-voice signal section from a voice signal. Most human speech signals are time-domain continuous signals that change over time, which can be reinterpreted in the frequency domain. So far, various studies on voice detection by the characteristics of time domain and frequency domain have been continuously carried out. A variety of speech detection methods based on FFT (Fast Fourier Transform), which simplifies complex energies of frame energy and DFT (Discrete Fourier Transform), which can be obtained through a simple procedure in the time domain, have been studied Patent No. 1056511 (registered on August 05, 2011)). In most of the studies, voice detection based on the characteristics of the time domain or the frequency domain showed some performance in discriminating the voice signal and the non-voice signal, but the voice detection performance in various environments such as the noise environment was not considered. In addition, speech detection without post processing is highly likely to have two types of errors, false alarm and false rejection, which adversely affect accurate voice detection. That is, only voice detection based on the characteristics of each area may cause deterioration of voice detection performance.

1. Korean Patent No. 1056511 (Registration date: August 05, 2011)

The present invention improves a voice activity detection (VAD) technique for detecting a voice interval, and recognizes and discriminates whether a detected voice interval is a predetermined command, so that voice control and confusion in a large number of inter- To minimize the occurrence of problems.

As one embodiment of the present invention, a multi-party wireless communication apparatus through voice recognition can be provided.

A multi-party wireless communication apparatus using voice recognition according to an exemplary embodiment of the present invention includes a voice collection unit for receiving a voice signal from a user, a voice recognition unit for separating a voice signal from a non- A voice recognition unit for determining whether or not a predetermined command for performing a voice control function (VOX) is included in the detected voice signal, and a voice recognition unit for, if the command is included in the detected voice signal, The control unit may cause the communication unit to receive a signal of a voice uttered by another user if the command is not included in the detected voice signal .

In addition, the multi-party wireless communication apparatus using voice recognition according to an embodiment of the present invention may further include an output unit for outputting a sound signal of a voice uttered by another user.

In the voice detector according to the embodiment of the present invention, the voice signal section and the non-voice signal section are detected from the voice signal, respectively, and the detection can be performed based on the energy value in the frequency domain.

The end point of the speech in the detection of the speech signal interval according to the embodiment of the present invention may be determined based on the number of frames whose energy value is smaller than the threshold value in the power spectrum of the frame of the speech signal in the frequency domain.

A multi-party wireless communication method using voice recognition according to an embodiment of the present invention includes receiving a voice signal from a user, discriminating a voice signal from a voice signal and detecting the voice signal, Determining whether or not a predetermined command for performing a voice control function (VOX) is included in the signal, and if the command is included in the detected voice signal, And receiving the voice signal from the other user if the command is not included in the detected voice signal.

Meanwhile, as an embodiment of the present invention, a computer-readable recording medium on which a program for causing the computer to execute the above-described method may be provided.

According to an embodiment of the present invention, voice control can be efficiently controlled in a multi-user voice communication environment by using VOX technology based on voice recognition using a multi-party wireless communication device using voice recognition.

In addition, it can contribute to the precise and accurate control of the system through speech recognition even when the hands are not free, such as when driving a car.

A typical known method for VOX function to activate voice transmission in a multi-party communication environment is a push-to-talk (PTT) transmission / reception switching switch. However, the inconvenience that the user has to continuously turn on / off the button during the transmission / reception process has been increased, and a technique of using the voice signal input to the microphone for the purpose of solving the problem has been studied come. Typically, the method measures the sound pressure level (dB) of the input voice to determine whether to activate the transmission. This method is advantageous in that it is easy to implement and can be operated in low-capacity hardware. However, in a real environment where noise is introduced, There is a problem that the reliability is low. Instead of the above-described PTT button or the algorithm using the sound pressure level, a multi-way wireless communication apparatus and method using speech recognition according to an embodiment of the present invention can realize a more stable and accurate VOX function .

FIG. 1 is a block diagram illustrating a multi-party wireless communication apparatus using speech recognition according to an exemplary embodiment of the present invention. Referring to FIG.
2 is a flowchart illustrating a multi-party wireless communication method using speech recognition according to an exemplary embodiment of the present invention.
3 is a flowchart illustrating an example of a wireless communication method according to an embodiment of the present invention.
4 is a flowchart showing a method for voice detection in (a) time domain and (b) frequency domain.
5 is a flowchart illustrating a post-processing method for voice detection according to an embodiment of the present invention.
6 shows an example of distribution of energy values of speech signals in (a) time domain and (b) frequency domain.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

The terms used in this specification will be briefly described, and the present invention will be described in detail.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements as well, without departing from the spirit or scope of the present invention. Also, the terms "part," " module, "and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software . In addition, when a part is referred to as being "connected" to another part throughout the specification, it includes not only "directly connected" but also "connected with other part in between".

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating a multi-party wireless communication apparatus using speech recognition according to an embodiment of the present invention, and FIG. 3 is a flowchart illustrating an example of a wireless communication method according to an embodiment of the present invention. 4 is a flowchart showing a method for voice detection in (a) time domain and (b) frequency domain, and Fig. 5 is a flowchart showing a post-processing method for voice detection according to an embodiment of the present invention 6 shows an example of distribution of energy values of speech signals in (a) time domain and (b) frequency domain.

The time domain and frequency domain energy can be used to process the speech signal. Although the energy of the time domain and the energy of the frequency domain measured in the same voice interval are not different from each other, the method using the energy of the frequency band in which the voice signal is mainly distributed among the frequency bands is different from the method using the time- The ratio of the false alarm to the false reject is relatively low and the high voice detection performance is experimentally confirmed as described below.

By using the post-processing method of voice detection according to an embodiment of the present invention, the error of voice detection can be drastically reduced compared with the conventional method. Such a post-processing method terminates the speech when the energy value of the speech signal inputted in real time is smaller than the threshold value and continues for a certain frame in the non-speech signal period. As a result of comparing with the conventional voice detection method, the voice detection method using post-processing according to an embodiment of the present invention reduces both false alarms and false rejects as two types of errors as compared with the case without post processing. It is confirmed that the performance of voice detection is improved.

In the voice detection using the power spectral energy of the voice distribution band in the frequency domain, the energy value is concentrated at the average point of the voice signal section and the non-voice signal section as compared with the time domain in which the sample energy value is dispersed (for example, It is easy to judge it by dividing it by the threshold value, thereby achieving high voice detection performance.

Among the existing researches, there is a method of discriminating a voice signal and a non-voice signal using Euclidean distance and FFT, and a method of discriminating a voice signal and a non-voice signal using zero crossing ratio and frame energy.

The FFT (Fast Fourier Transform) based speech detection method is based on the FFT of a speech signal, the power spectrum of the power spectrum, and the power spectrum of the entire sentence, and discriminates the speech signal when the speech signal is above a predetermined threshold value. The speech detection formula using FFT is as follows.

,

는 각 주파수 영역에서 구한 잡음 및 음성신호 스펙트럼이고

는 음성신호인지 비음성신호인지 구분하기 위한 임계치,

는 전체 문장의 프레임,

는 문장의 잡음구간을 처음으로 나타내는 프레임 개수를 의미한다.In Equation (1)

,

Is the noise and speech signal spectrum obtained in each frequency domain

A threshold for distinguishing between a voice signal and a non-voice signal,

Is a frame of the whole sentence,

Means the number of frames that represent the noise section of the sentence for the first time.

We used cepstrum distance applying Euclidean distance as another voice detection method. A voice signal can be separated into a voice signal and a non-voice signal by using the Euclidean distance. In order to extract the speech signal in the cepstrum domain, the FFT-processed speech signal is logarithmically processed, then the IFFT (Inverse Fast Fourier Transform) is performed, and the speech signal in the cepstrum domain is extracted by multiplying the cepstrum domain . The extracted voice signal can be identified as a voice signal when the cepstrum distance is measured based on the cepstrum center, and a non-voice signal when the cepstrum distance is below a predetermined threshold value. Equation (2) is a basic equation of Euclidean distance, and the cepstrum distance formula applied is as follows.

은 입력된 음성의 켑스트럼 계수이고,

은 프레임의 켑스트럼을 나타낸다.

은 켑스트럼 중심을 의미한다.In Equation 2,

Is the cepstral coefficient of the input speech,

Indicates the frame strum of the frame.

Is the center of the string.

Zero-crossing and frame energy bases have also been used for a long time. The zero crossing rate is the rate at which the sign of the signal changes as the voice signal crosses zero. However, when there are many noises similar to those of voiced sounds, it is difficult to distinguish between voice signals and non-voice signals only by the zero crossing rate. Therefore, it is possible to discriminate between the speech signal and the non-speech signal based on the energy characteristic of the time domain of the speech signal having a large energy value in the speech interval and a small value in the non-speech interval while using the zero crossing rate. However, this method also has a high energy value in a noisy environment, and it is difficult to find a threshold value for determining a speech signal interval and a non - speech signal interval.

The problems of the prior art described above are as follows:

- It is mainly dependent on time domain, and is vulnerable to speech detection when background noise is included.

- In case of existing frequency system, it is difficult to handle in small handset such as walkie-talkie because of high computational burden.

The main goal of the VAD is to accurately detect the end point (e.g., end point) of a speech. In case of distinguishing speech / non-speech based on a threshold value for each frame as shown in FIG. 4A, (For example, when the word "school" is spoken, and "school" and "bridge" are determined after each VAD, respectively)

According to an embodiment of the present invention, a VAD is performed based on energy in a frequency domain in which a speech signal is distributed (for example, spectral energy after FFT) Unlike energy, spectral energy has a characteristic in which energy of voice and non-voice is concentrated and distributed, so that detection of voice and non-voice can be easier than conventional. (See Fig. 6)

Further, when a frame representing an energy smaller than a threshold value continuously continues for a specific number or more, the end point (e.g., end point) of the voice can be accurately detected by determining the end point as the end point.

The voice can be obtained through a VOX command (for example, a short vocabulary such as " voice start "), which is a predetermined vocabulary between users, by using the multi-party wireless communication apparatus using speech recognition according to an embodiment of the present invention. The following description can be carried out on its own within a small appliance such as a walkie-talkie. That is, a communication method according to an embodiment of the present invention, which is not a method of processing a message generated by a client, which is a conventional technique related to speech recognition, at a server terminal and providing the message to a corresponding client and other clients, The amount of computation can be realized. For example, speech recognition may be performed by applying HMM (Hidden Markov Model) or the like.

The multi-party wireless communication apparatus 1000 using voice recognition according to an embodiment of the present invention includes a voice collecting unit 1100 for receiving a voice signal uttered by a user, a voice and non- A voice recognition unit 1300 for determining whether a predetermined command for performing a voice control function (VOX) is included in the detected voice signal, and a voice recognition unit If the detected voice signal is included in the detected voice signal, the communication unit 1400 causes the communication unit 1400 to transmit a voice signal of the user's voice to the wireless communication apparatus of another user. If the voice communication signal is not included in the detected voice signal, And a control unit 1500 for receiving a voice signal from another user. For example, the sound collection unit 1100 may include an electronic device for acquiring sound such as a microphone or the like.

In addition, the multi-user wireless communication apparatus 1000 using voice recognition according to an embodiment of the present invention may further include an output unit 1600 through which a sound signal of a voice uttered by another user is output as a sound. Output 1600 may include an electronic device for outputting sound, such as a speaker.

In the voice detector 1200 according to an embodiment of the present invention, a voice signal section and a non-voice signal section are detected from a voice signal, respectively, and the detection can be performed based on the energy value in the frequency domain. The voice detection can be performed in the manner described below with reference to Fig.

The end point of the speech in the detection of the speech signal interval according to the embodiment of the present invention may be determined based on the number of frames whose energy value is smaller than the threshold value in the power spectrum of the frame of the speech signal in the frequency domain.

The speech detection method using the sample energy in the time domain (see FIG. 4A) measures the sample energy in units of frames from the input speech signal, compares the sample energy with a threshold value for each frame, and calculates a speech signal and a non- The signal can be discriminated. Therefore, if the sample energy value of the input speech signal is larger than the threshold value, it is determined as a speech signal, and if it is small, the speech signal can be determined as a non-speech signal.

는 시간 영역에서의 한 프레임 단위의 샘플 에너지를 나타낸다.

은 입력된 음성신호에서

번째 샘플의 음성신호를 의미하며,

은 한 프레임의 샘플의 개수를 의미한다.

는 임계치를 나타내며, 실시간으로 입력된 음성신호의 샘플 에너지 값 중에서 최적의 에너지 값이 임계치로 사용될 수 있다. In Equation 3,

Represents the sample energy of one frame unit in the time domain.

Lt; RTI ID = 0.0 >

Th sample audio signal,

Means the number of samples of one frame.

And the optimum energy value among the sample energy values of the speech signal input in real time can be used as a threshold value.

In the method of performing speech detection using the power spectrum of the frequency domain characteristic (see FIG. 4 (b)), the input speech signal is divided into fixed lengths, and FFT is performed to obtain power spectrum energy per frame. The power spectral energy of a frequency band (mainly, a low frequency band) in which speech signals are distributed among the energy of each frequency band is compared with a threshold value on a frame-by-frame basis to discriminate a speech signal and a non-speech signal according to the following mathematical expression. If the power spectrum value of the input voice signal is larger than the threshold value as in the time domain voice detection algorithm, the voice signal is discriminated as a voice signal. If the power spectrum value is small, the voice signal can be discriminated as a non-voice signal.

는 주파수 영역에서의 한 프레임 단위의 전력 스펙트럼을 나타낸다.

,

은

번째 시간 영역의 음성신호를 FFT 취하여 주파수 영역에서 실수와 허수로 나타낸 것을 의미한다.

는 한 프레임의 FFT 크기보다 작은 값으로 음성 신호가 주로 분포하는 주파수 대역의 수를 의미한다.

는 임계치를 나타내며, 실시간으로 입력된 음성신호의 전력 스펙트럼 에너지 값 중에서 최적의 값이 임계치로 사용될 수 있다.

Represents a power spectrum of one frame unit in the frequency domain.

,

silver

Th time domain speech signal is represented by a real number and an imaginary number in the frequency domain.

Means the number of frequency bands in which the speech signal is mainly distributed with a value smaller than the FFT size of one frame.

And the optimum value among the power spectral energy values of the voice signal inputted in real time can be used as a threshold value.

Referring to FIG. 5, a post-processing method for detecting end points of speech is based on energy in the frequency domain. A post-processing algorithm (method) is a process of detecting a time point at which a voice is terminated after a voice start interval is detected through a frequency energy based VAD. The energy value of each area is compared with a threshold value in units of frames as shown in the following equation, and if the number of frames in the non-speech signal interval is equal to the predetermined integer value, the speech can be terminated.

는 주파수 영역에서의 한 프레임 단위의 전력 스펙트럼 에너지를 나타낸다.

는 임계치보다 작은 에너지 값을 연속적으로 나타내는 프레임 개수이다.

은 음성 종료 시점을 결정하는데 사용되는 프레임의 개수를 의미하며, 이것은 정수 값일 수 있다.In Equation (5)

Represents the power spectral energy of one frame unit in the frequency domain.

Is the number of frames continuously representing an energy value smaller than the threshold value.

Means the number of frames used to determine the voice end time, which may be an integer value.

)와 한 프레임 단위의 샘플의 개수를 곱하여

으로 나누면

시간을 구할 수 있다. 비음성신호 구간에서 임계치보다 작은 프레임이

시간 동안 지속되었을 때 음성이 종료됨을 판별할 수 있다. Equation (6) represents the time of voice end described in Equation (5). The number of frames used to determine the end of speech (

) And the number of samples in one frame unit

Divided by

Time can be saved. In a non-speech signal section, a frame smaller than the threshold value

It is possible to judge that the voice is terminated when it lasts for a time period.

2 is a flowchart illustrating a multi-party wireless communication method using speech recognition according to an exemplary embodiment of the present invention.

A multi-party wireless communication method using speech recognition according to an exemplary embodiment of the present invention includes a step (S100) of receiving a voice signal from a user, a step of discriminating a voice signal and a non-voice signal from a voice signal S200), determining whether a predetermined command for performing a voice control function (VOX) is included in the detected voice signal (S300), and if the command is included in the detected voice signal, (S400) of transmitting a voice signal of another user to the wireless communication device of another user and receiving a voice signal of the voice from another user if the command is not included in the detected voice signal.

The following experiments were conducted to verify the effects of the present invention.

We used the voice signals input by 5 men and 1 woman in real time. The voice data used in the experiment consists of vocabulary such as "start of voice" and "microphone input" meaning commands for radio control of the radio. Each speaker utteres 2 words per word and tests 8 words per speaker Data were used. In the time domain sample energy and frequency domain power spectrum, the number of samples used in one frame is 320, and the sampling frequency is sampled at 16 kHz. Also, the FFT size is set to 512 for frequency domain analysis. In this experiment, the speech detection performance by the characteristics of the time domain and the frequency domain was evaluated. When the energy value of the speech signal inputted in real time was smaller than the threshold value, the ending point of the speech was determined when the non- We verified the post - processing algorithm. This experiment consists of experiments comparing the voice detection by the energy in the time domain and the frequency domain and the experiment using the post processing method according to the embodiment of the present invention. The error of False alarm and False reject was used as a performance evaluation scale. The threshold value used in the experiment was statistically estimated as a graph showing an inverse graph with a gentle slope. In order to evaluate fairly, a real-time input speech signal was stored considering a change in performance due to noise generated in a real environment, In the mix channel state (system sound: 70, speaker sound: 25).

Table 2 below shows False alarm and False reject results of the voice detection using the time-domain sample energy, Table 2 shows the false alarm and False reject results of the voice detection using the frequency spectrum power energy, Full False indicates alarm and False reject result.

As shown in Table 3, the False alarm error rate is lower when the frequency range is equal to the time-domain false rejection condition. This result shows that the frequency energy based method exhibits high voice detection performance. Referring to FIG. 6, the sample energy of FIG. 6 (a) is dispersed at the average point of the speech signal and the non-speech signal in comparison with the power spectrum energy of (b) have. That is, it is concentrated at the average point of the two sections, so that it is easy to distinguish between the voice signal and the non-voice signal by the threshold value.

)는 정수 값 15로 설정하여 실험에 사용하였다. 이는 위 수학식 6에 따라 0.3초 동안 비음성신호 구간이 지속되었을 때 음성 종료 시점을 판단함을 뜻한다. 이 같은 후처리를 통해 두 영역에서의 False alarm과 False reject가 눈에 띄게 감소한 것으로 나타났다. Table 4 shows the results of voice detection after applying the proposed post-processing according to each region. The number of frames that determine the end time of voice (

) Was set to an integer value of 15 and used in the experiment. This means that the speech end time is determined when the non-speech signal duration is continued for 0.3 second according to Equation (6) above. This post-processing showed that False alarms and False rejects in both areas decreased significantly.

The contents of the above-described apparatus can be applied in connection with the method according to an embodiment of the present invention. Therefore, the description of the same contents as those of the above-mentioned apparatus has been omitted in connection with the method.

One embodiment of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer readable medium may include both computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

1000: Multipurpose wireless communication device through speech recognition
1100: Voice collecting unit
1200:
1300: Voice recognition unit
1400:
1500:
1600: Output section

Claims (6)

A multi-party wireless communication apparatus using speech recognition,
A voice collector for receiving a voice signal from a user;
A voice detector for separating a voice signal and a non-voice signal from the voice signal and detecting the voice signal;
A voice recognition unit for determining whether a predetermined command for performing a voice control function (VOX) is included in the detected voice signal; And
If the command is included in the detected voice signal, causing the communication unit to transmit a signal of the user's uttered voice to a wireless communication apparatus of another user, and if the command is not included in the detected voice signal, And a controller for receiving a voice signal from the other user to input the voice signal.
The method according to claim 1,
Further comprising an output unit for outputting a sound signal of an uttered voice from the other user to the multi-user wireless communication apparatus through the voice recognition.
The method according to claim 1,
Wherein the voice detection section detects a voice signal section and a non-voice signal section from the voice signal, respectively, and the detection is performed based on an energy value in the frequency domain.
5. The method of claim 4,
Wherein the end point of the voice in the detection of the voice signal section is determined based on the number of frames whose energy value is smaller than the threshold value in the power spectrum of the voice signal in the frequency domain in frame units, Wireless communication device.
A multi-party wireless communication method using voice recognition,
Receiving a voice signal of a voice from a user;
Separating and detecting a voice signal and a non-voice signal from the voice signal;
Determining whether a predetermined command for performing a voice control function (VOX) is included in the detected voice signal; And
If the command is included in the detected voice signal, transmit the voice signal of the user to the wireless communication apparatus of another user, and if the command is not included in the detected voice signal, And receiving a voice signal from the voice recognition unit.
A computer-readable recording medium on which a program for implementing the method of claim 5 is recorded.

Images