KR20100086457A - Song searching method and device using voice recognition technology in karaoke environment - Google Patents

Abstract

PURPOSE: An apparatus and a method for using in searching a song title using a speech recognition technique in a karaoke environment are provided to enable to play the song stored in a machine after eliminating noise from the user voice on a microphone and detecting the keyword to a song title using a speech recognition technique. CONSTITUTION: A method for using in searching a song title using a speech recognition technique in a karaoke environment is as follows. Voice is input with micro(200). The signal input to the microphone is transmitted to a codec and converts an analog signal to a digital signal(220). A noise eliminating algorithm and a speech recognition program are executed in a microprocessor(250) and a main memory(270). A keyword is searched by comparing with the parameter of a stored song title database(280). Using a keyboard or a keypad, additionally necessary information is input(240). Desired information is displayed on a screen and the searching information of a remote control or a list book is transmitted to a machine and a corresponding accompanimnent is played(260).

Description

Song searching method and device using voice recognition technology in karaoke environment {Song searching method and device using voice recognition technology in karaoke environment}

The present invention uses a signal processing algorithm that extracts only the target signal in a noisy environment, and then using a speech recognition technology to search for the desired song in the karaoke by voice to find a songbook under dark lighting, or Or it relates to a half-ring cycle and a remote control device having a voice recognition function to eliminate the inconvenience of using the keyboard of the table of contents.

Songs contained in the title book of karaoke are not found in order as new songs are added, making it difficult to find, and environmentally speaking, the lighting is generally dark, so much time and effort is required to search for song titles.

According to the related art, by using a remote control table of contents or a remote control, the songs corresponding to the preceding measures are sequentially displayed on the screen to facilitate searching. However, this method also has the inconvenience of having to type letters one by one. In order to find the title in real time by vocalizing the title of the desired song in the microphone, the voice recognition rate of about 20,000 songs should be high. The recognition rate that can be commercialized should be guaranteed.

The present invention is designed to solve the above problems, the signal processing to remove the noise from the user's voice input to the microphone and to detect the keyword for the song name with a speech recognition technology, and then to execute the song stored in the half cycle The purpose of the present invention is to provide a method for developing an algorithm and implementing a hardware system capable of processing the same in real time.

By using the digital signal processing technique according to the present invention to achieve the above object, effectively removes the environment, half-cycle, human song noise, improve the speech recognition rate, search for the keyword of the desired song, half-cycle or table of contents Select and play the saved song. To remove the noise, convert the microphone input signal into the frequency domain, measure the statistical characteristics and power ratio of the signal, and determine the noise. Then, the gain adjuster is activated and subtract the spectrum of the estimated noise in the target signal section. do. In order to improve the speech recognition rate, the amount of change in the frequency and time axes of the input signal is measured to accurately extract the start and end points of the target signal, and the buffered effective speech data is transmitted to the speech recognition program. Compared with the pre-stored feature parameters of songs stored in the database of the semi-period or table of contents, the voice-recognized keyword is transmitted to the semi-period through the remote transceiver to start accompaniment.

According to the present invention, in a noisy karaoke environment, the user can use a microphone to search and play songs stored in a karaoke half-cycle or table of contents by voice, so that the user can search for a song book in a dark light or do not have to hit a keyboard of the remote controller. It can have the effect of searching for.

1 is a block diagram of a short channel noise cancellation algorithm.
2 is a flowchart illustrating a system for searching music pieces using a speech recognition technique after removing noise.
3 is a diagram illustrating a relationship between a half cycle and a remote controller using speech recognition.
4 is a block diagram of a hardware board for real-time processing of the example program modules described in FIG.

Hereinafter, the theory, configuration and operation of the present invention will be described in detail.

Signal processing method to increase speech recognition rate

Since the recognition rate of the speech recognition program depends on whether the input signal contains noise, an effective noise canceling technique is essential for using the speech recognition technique in a noisy environment such as karaoke.

A typical short channel noise cancellation system is performed in the frequency domain and estimates the loudness of speech by determining the attenuation or gain of each frequency component. This is a method to remove the ambient noise by using the characteristic that the noise is less change than the voice in the input signal mixed with voice and noise.

A block diagram of the proposed 1 mic noise cancellation system is shown in FIG. The 1 mic sound quality enhancement system of FIG. 1 estimates the power spectrum of noise D (k, l) from the magnitude information of the frequency component Y (k, l) of the input signal y (t) with noise added to the speech, After estimating the gain G (k, l), we multiply the magnitude signal spectrum of the input (noise spectral subtraction) and synthesize the speech using an Inverse Fast Fourier Transform (FFT).

If the noise section is estimated, the gain controller reduces the magnitude of the input signal. If the gain controller is not used, the residual component after subtracting the frequency component of the noise is output.

Among the statistical characteristics of the microphone input signal, the change in the frequency axis and the change in the time axis are calculated, and the noise and target signal sections are distinguished by investigating the change in the noise section and the target signal section.

,

의 평균을

, 해당 프레임의 전체 파워를

라 하면 주파수 영역에서의 정규화된 변화량, 즉 주파수 편평도는 수학식 1로 표현된다.
The power of each frequency component in the frequency domain

,

Average of

, The full power of that frame

In this case, the normalized change amount in the frequency domain, that is, the frequency flatness, is expressed by Equation 1.

[Equation 1]

는 실험적으로 얻어진 임계치이다.
here

Is an experimentally obtained threshold.

,

의 평균을

, 해당 프레임의 전체 파워를

라 하면 시간 영역에서의 정규화된 변화량은 수학식 2 로 표현된다.
The power of one frame in the time domain

,

Average of

, The full power of that frame

In this case, the normalized change amount in the time domain is represented by Equation 2.

[Equation 2]

는 실험적으로 얻어진 임계치이다.
here

Is an experimentally obtained threshold.

If the amount of change calculated in Equations 1 and 2 is larger than the experimentally obtained threshold, it can be regarded as the target signal.

Another parameter for estimating the target signal is to use the power of the input signal twice for a long time using the Infinite Impulse Response (IIR) average and compare it with the current frame power.

The IIR average power of the current frame is calculated as in Equation 3.

&Quot; (3) "

는 0과 1사이의 IIR 스무딩 계수이며,

는 현재 프레임의 IIR 평균 파워,

은 이전 프레임의 파워이다.here

Is the IIR smoothing coefficient between 0 and 1,

Is the IIR average power of the current frame,

Is the power of the previous frame.

가

보다 일정 배수 이하의 프레임에 해당하는 파워에 대해서 IIR(Infinite Impulse Response) 평균을 다시 계산하면 수학식 4와 같다.

end

Re-calculating the average impulse response (IIR) for a power corresponding to a frame less than a certain multiple, as shown in Equation 4.

&Quot; (4) "

는 0과 1사이의 IIR 스무딩 계수이며,

는 현재 프레임의 long-term IIR 평균 파워,

은 이전 프레임의 파워이다.here

Is the IIR smoothing coefficient between 0 and 1,

Is the long-term IIR average power of the current frame,

Is the power of the previous frame.

는 변화량이 급격하게 큰, 즉 큰 입력 신호에 대해서는 평균 계산에 참여하지 않게 되어 대체로 잡음 성분들의 프레임 파워를 보여 주게 된다.Calculated in Equation 4

For the large input signal, the amount of change does not participate in the average calculation, which shows the frame power of the noise components.

의 일정 배수보다 큰 경우에는 목적 신호로 간주할 수 있다.
Therefore, the power of the current frame of the microphone input signal

If it is larger than a certain multiple of, it can be regarded as an objective signal.

[Equation 5]

는 현재 프레임의 파워, c 는 실험적으로 목적 신호로 간주할 수 있는 배수로 임의의 상수이다.
here

Is the power of the current frame, c is an arbitrary constant that can be considered experimentally the target signal.

과 입력 신호의 프레임 파워인

와의 IIR 평균된 신호대잡음비를 수학식 6과 같이 계산한다.
Frame power of the noise section estimated using Equations 1 to 5

And the frame power of the input signal

Calculate the IIR averaged signal-to-noise ratio with

&Quot; (6) "

은 이전 프레임의 파워이며

는 0과 1사이의 IIR 스무딩 계수,

는 절대치 연산자이다.here

Is the power of the previous frame

Is the IIR smoothing coefficient between 0 and 1,

Is an absolute operator.

When using the Wiener filter commonly used among the noise reduction algorithms and applying Equation 6, Equation 7 is expressed.

[Equation 7]

In Figure 1, the gain multiplied by the frequency component of the input signal is represented by Equation 8 below.

[Equation 8]

은 잡음 성분의 감쇄 계수(Attenuation Level)이며 클수록 잡음 성분을 많이 감쇄시킨다.
here

Is an attenuation level of the noise component, and the larger it reduces the noise component.

를 곱하여 주파수 대역에서의 잡음 성분을 줄였으나, 목적 신호내의 잡음 성분을 좀 더 많이 제거하기 위해 수학식 8의

를 곱하여 얻은 목적 신호를 입력 신호

에서 뺀 추정된 잡음 신호

를 수학식 9와 같이 계산한다.
About input signal

Multiply by to reduce the noise component in the frequency band, but in order to remove more noise components in the target signal

Multiply the desired signal by the input signal

Estimated Noise Signal Subtracted from

Is calculated as in Equation (9).

[Equation 9]

를 갱신하는 방법으로 steepest descent 기반의 알고리즘을 적용하며 수학식 10과 같이 표현된다.
Coefficient of Adaptive Filter Using Normalized Complex Least Mean Square (LMS) Algorithm

As a method of updating, a steepest descent based algorithm is applied and is expressed as in Equation 10.

[Equation 10]

은 n-1 번째 입력신호,

은 n-1 번째까지 추정된 잡음 신호,

은 n-1번째 출력 신호,

는 수렴속도를 결정하는 상수,

은 노름(norm) 연산자이다.here

Is the n-1th input signal,

Is the noise signal estimated up to n-1th,

Is the n-1th output signal,

Is a constant that determines the speed of convergence,

Is the norm operator.

In addition, in the section estimated as noise, the gain adjuster is operated to reduce the level of the input signal and output the noise, thereby suppressing the speech recognizer from operating so as to prevent an error that the song is searched by noise instead of the user's voice.

Karaoke half cycle Table of Contents Data information between remote controllers

As shown in FIG. 2, the microphone input signal 200 passes only a signal of a voice band through a preamplifier 210 including a low pass filter, and is then passed to a codec to convert an analog signal into a digital signal (220). A noise reduction algorithm and a speech recognition program are executed in the microprocessor 250 and the main memory 270 to search for keywords by comparing the parameters of the pre-stored song title database (280), and supplementally necessary information using a keyboard or a keypad. (240) The desired information is displayed on the screen (230), the search information or keyword of the remote control or table of contents is transmitted to the semi-period through the remote transceiver (260) and the song accompaniment is performed (290) Noise reduction algorithm The software and the voice recognition program that implements the song are independently mounted in the table of contents or the half cycle, and accompanies songs by the searched keywords. Also, and there after to the remote transmitting the audio signal input to the microphone on the remote control to drive the half-period the noise reduction and speech recognition software with a half-period search the music name can also be a song accompaniment.

As shown in FIG. 3, after the voice input to the microphone is converted into a digital signal through a codec (300), the microprocessor and the main memory determine whether there is noise (310). The gain adjuster is removed (320) to extract the features and parameters of the desired signal (330), and to store the parameters of the pre-calculated song title (350) and compare them with the parameters extracted from the input signal (340). The candidates with short distances between the two parameters are displayed on the screen (360). If there is a desired song among the candidates, the song is selected by using a voice or an auxiliary button (370). 380)

Hardware Development for Real Time Processing

4 is a block diagram of an independent hardware system for operating a program implementing the flowchart illustrated in FIG. 2 in real time.

A real-time processing board based on an Arm 920T processor or a general-purpose processor and Philips CODEC UDA1341TS or AD (analog-to-digital) converter was developed based on the block diagram shown in FIG.

The input signal from the two microphones is amplified in the preamplifier 400 including the first low pass circuit of cutoff frequency 17KHz and converted into digital values by the sampling frequency of 44.1KHz in the UDA1341TS stereo codec 410. After the program implemented in the assembly language of C and Arm 920T is optimized, the algorithm shown in the flowchart of FIG. 2 is optimized and operated in the Arm 920T 430 to remove noise included in an input signal in real time, extract feature parameters, and perform voice recognition. Search for the desired song by the program. Frequently retrieved songs are automatically stored in the EEPROM (480) to shorten the search time for many called songs, and designed hardware for user convenience. In order to change the function in detail according to the user's environment, a button 420 and a connection device 440 for exchanging data can be exchanged by connecting to a PC or other device, and a flash memory ( 450) was used. Equipped with a copy protection security chip 460 for copyright protection of the software, and the LED on the front panel 470 to indicate the operating status of the program and whether the button is used in order to facilitate the user's use and control of the equipment. It was.

The above description of the algorithm for removing the noise contained in the microphone input signal in a noisy environment such as karaoke, searching for the name of a song using voice recognition technology, and then accompaniing the song in a half cycle and hardware for real-time processing Presented for. It is not intended to be exhaustive or to limit the invention to the precise form of equations or drawings. Many modifications and variations are possible in light of the above teaching, and some combinations of the mathematical formulas and embodiments may be used. It is intended that the scope of the invention be limited not by this detailed description or figures, or by equations, but by the claims appended hereto.

200: microphone signal input module
210: check the presence of noise module
220: noise cancellation and gain regulator using module
230: destination signal feature and parameter extraction module
240: comparison module with the existing stored parameters
250: precomputed parameter database storage module for music pieces
260: Module to display the retrieved candidate songs on the screen
270: module to select the desired song by voice or button
280: A module for accompaniment of songs with keywords received from the semi-period or the remote control device

Claims (7)

As a method to search for the title of a song by using a voice recognition technology, the equipment such as a semi-period, a table of contents, a remote control, etc. in a noisy environment such as karaoke. A data storage unit for storing the data storage unit; A preamplifier section including a microphone and a low pass filter; A flash memory unit including a microprocessor and a main memory unit for operating a program and a firmware program for driving electronic components and equipment when the power is turned on; A display unit for displaying the operating state of the program and equipment; In addition, a method of providing a user with a plurality of candidate songs as a result of speech recognition by operating a speech recognition program after removing noise on hardware including a data remote transmission / reception unit configured by frequency modulation, Bluetooth, and infrared circuits Karaoke speech recognition half cycle and table of contents, remote control device.
As a noise estimation method to remove the half-period accompaniment sound and the singer's voice from the voice recognition microphone input signal to improve the speech recognition rate in a noisy karaoke environment,
Equation 1 after converting a microphone input signal into a frequency domain

The change amount of the frequency component by

And the amount of change in the time domain by

By the IIR average power of the current frame,
Equation 4

By the long term IIR average power of the current frame,

> c

And estimating whether the input signal of the current frame is a noise or a target signal by comparing the power of the current frame with an experimentally obtained threshold and a speech recognition preprocessing method characterized by the above.
Equation 6 to subtract the noise component estimated in claim 2 from the target signal

Calculate the ratio of the input signal and the noise signal by

Equation 8 based on the Wiener filter of

Multiply the gain of the input signal by

After estimating only the noise component at, Equation 10

A noise reduction method for minimizing noise components in a target signal section by passing an input signal through an adaptive filter using a coefficient of and a speech recognition preprocessing method characterized by the same.
According to claim 2, buffering including past values in a method of measuring variation of an input signal according to Equation 1 and 2 for obtaining valid data required for speech recognition by designating a start point and an end point of a target signal mixed with noise. Speech recognition preprocessing method that adds a technique and transmits to the speech recognition engine including valid data that is easy to be lost such as consonant.
The average power of the frequency components and the formant frequency are measured with respect to the effective data obtained by the second, third and fourth terms, and the voices of the males or females are distinguished and compared with the male and female phoneme data on the database. Even if the speaking speed is significantly different, or long song titles (eg, one hundred meters before she meets her) are recognized as valid data by calculating the average impulse response (IIR) for the time and frequency axis changes in the input signal. A method of improving the speech recognition rate and a speech recognition song accompaniment device characterized by the same.
The microphone mounted on the remote controller transmits the user's voice to the semi-period through a remote transmission unit such as FM modulation, Bluetooth, or IR, or by a microphone connected to the semi-period. Displaying about 20 on the screen display, the user selects one of the songs and transmits the keyword of the corresponding song name in a semi-period, the semi-period to play the accompaniment of the song.
The microphone mounted on the table of contents transmits the user's voice to the pre-processing and speech recognition program mounted on the table of contents, and displays 5 to 20 song names similar to the words recognized in the display on the screen display. A method in which a user selects one of the songs and transmits the keyword of the corresponding song name to a half cycle through a remote transmission unit such as frequency modulation, Bluetooth, and infrared method, and the half cycle plays the accompaniment of the song.

Images