KR100391123B1 - speech recognition method and system using every single pitch-period data analysis - Google Patents

Abstract

The present invention relates to a speech recognition technique that improves the processing speed by processing pattern matching on a phoneme basis using pitch detection.

Such a system of the present invention includes a pitch detector for receiving a voice signal and detecting a pitch to output pitch position information; A pitch data analyzer configured to receive pitch position information from the pitch detector, standardize pitch data, extract feature vectors, and compare voice data with pre-registered standard pitch data to generate voiced phonetic strings; A silent section searcher which receives a voice signal and detects a silent section and outputs silent section information; An unvoiced sound discriminator that receives the pitch position information and the silence section information, determines an unvoiced sound section, and outputs an unvoiced phone string by receiving the sound signal; A syllable branch that receives the voiced phoneme string, the unvoiced phone string, and the silent section information to generate a single syllable corresponding to a phonetic character; And a language analyzer configured to receive a single syllable of the syllable branch and apply a grammar rule to generate a standard language string. Therefore, according to the present invention, after detecting the pitch of the voice signal, pattern matching is performed at the pitch period to determine the phonemes. Thus, the database and the calculation capacity are small, so that the voice can be accurately recognized by a relatively small computer.

Description

Speech recognition method and system using every single pitch-period data analysis}

The present invention relates to a speech recognition technology, and more particularly, to a speech recognition technology of improving processing speed by processing pattern matching on a phoneme basis using pitch detection.

In general, speech recognition refers to a process of expressing an acoustic signal produced by a speaker through a microphone or a telephone as words or phrases that can be understood by a human being. It is used for command or control on computer or machine, data input, document preparation, etc.

Speech recognition consists of AD (Acoustic Decoder) that recognizes phonemes / syllables or words from speech signals, and LD that recognizes sentences by combining the AD results and linguistic information. The field of AD is generally called speech recognition. Speech recognition is a process of recognizing a coefficient (W) of the nearest model by pattern matching with a recognition model when a pronounced speech pattern is given. The voice signal is input to the voice signal preprocessor via analog-to-digital conversion (ADC), and the voice signal preprocessor converts the voice signal of the time domain into a frequency domain so that the information inherent in the voice can be more effectively recognized by the next stage recognizer. do. The voice recognition algorithms used for speech recognition include DTD (Dynamic Time Warping), HMM (Hidden Markov Mode) and Neural Network (NN). The basic theory is based on pattern matching.

Among speech recognition techniques, a method of converting a speech signal into a string is called a speech-to-text (STT) technique. A conventional speech recognition technique for SST is a signal processing process (2) and a voice feature as shown in FIG. It consisted of extraction process (4), sound similarity analysis process (6), and linguistic similarity analysis process (8). That is, after digitally amplifying the voice signal input in the signal processing process (2), the feature is transformed in the voice feature extraction process (4), and the state using the sound DB 10 in the sound similarity analysis process (6). / After determining the phoneme, the sentence was judged using the language and pronunciation DB (12,14) in the linguistic similarity analysis process (8).

However, since the conventional technology recognizes speech based on frequency analysis based on a word or a buffer of a predetermined size, the recognition speed is slow due to a large amount of computation, and the size of a standard database has a big problem. In particular, in the case of recognizing word units, there is a problem that it is difficult to apply to a portable device such as a mobile phone or a PDA because a large computer is required to recognize a large vocabulary, except for a small vocabulary recognition.

The present invention provides a speech detector and a speech recognition method and system using a pitch unit data analysis and a pitch detector to analyze the speech signal in one cycle pitch data unit to solve the above problems. There is this.

1 is a block diagram showing a speech recognition system according to the present invention;

FIG. 2 is a detailed block diagram showing the pitch detector shown in FIG. 1;

3 is a functional block diagram showing the waveform simplification filter shown in FIG.

4 is a conceptual diagram of an impulse train filter according to the present invention;

5 is a flowchart illustrating a pitch selection method according to the present invention;

6 is a conceptual diagram of the position compensator shown in FIG. 2;

7 is a detailed block diagram illustrating the pitch data analyzer shown in FIG. 1;

8 is a detailed block diagram illustrating the unvoiced voice identifier shown in FIG. 1;

FIG. 9 is a detailed block diagram showing the silence section searcher shown in FIG. 1; FIG.

10 is a detailed block diagram illustrating a syllable separator shown in FIG. 1;

FIG. 11 is a detailed block diagram showing a language analyzer shown in FIG. 1;

12 is a conceptual diagram illustrating a concept of a pitch data normalization unit shown in FIG. 7;

13 is a view showing an example of a speech recognition process according to the present invention;

14 is a diagram illustrating a general voice recognition procedure.

* Explanation of symbols for main parts of the drawings

102: pitch detector 104: pitch data analyzer

106: unvoiced identifier 108: silent section searcher

110: syllable branch 112: language analyzer

202: waveform simplification filter 204: impulse train generator

206: pitch selector 208: position compensator

702: pitch extractor 704: pitch data normalization unit

706: Feature Extractor 708: Standard Pitch Database

710: feature vector comparator 802: unvoiced classifier

804: Unvoiced Feature Extractor 808: Unvoiced Feature Comparator

In order to achieve the above object, the voice recognition system of the present invention includes: a pitch detector for detecting a pitch by receiving a voice signal and outputting pitch position information; A pitch data analyzer configured to receive pitch position information from the pitch detector, standardize pitch data, extract feature vectors, and compare voice data with pre-registered standard pitch data to generate voiced phonetic strings; A silent section searcher which receives a voice signal and detects a silent section and outputs silent section information; An unvoiced sound discriminator that receives the pitch position information and the silence section information, determines an unvoiced sound section, and outputs an unvoiced phone string by receiving the sound signal; A syllable branch that receives the voiced phoneme string, the unvoiced phone string, and the silent section information to generate a single syllable corresponding to a phonetic character; And a language analyzer configured to receive a single syllable of the syllable branch and apply a grammar rule to generate a standard language string.

In order to achieve the above object, the speech recognition method of the present invention is a speech character conversion (STT) method for receiving a speech signal and generating a character string corresponding to the speech signal, wherein the pitch is detected from the input speech signal. step; Analyzing the detected pitch position information and a voice signal to set a pitch section, a silent section, and an unvoiced section; Discriminating voiced phonemes by comparing one period pitch pattern in the pitch period with predetermined standard data; Discriminating unvoiced phonemes by comparing one period pitch pattern in the unvoiced sound interval with predetermined standard data; And combining the determined voiced phones and unvoiced phones according to a predetermined rule to generate single syllables.

In addition, the pitch detector of the present invention to achieve the above object, a waveform simplification filter for receiving a voice signal and simplifying the waveform; An impulse train generator for generating an impulse train of a predetermined channel based on the output of the waveform simplification filter; A pitch selection unit detecting a pitch position by applying a predetermined pitch selection rule; And a position compensator for compensating the pitch position detected by the pitch selector by a position difference between the original input signal and the simplified signal.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing a speech recognition system according to the present invention, the speech recognition system 100 of the present invention is a pitch detector 102, pitch data analyzer 104, unvoiced identifier 106, silence interval searcher ( 108), the syllable separator 110, and the language analyzer 112 receives a voice signal and outputs a standard language string.

Referring to FIG. 1, the pitch detector 102 includes a waveform simplification filter 202, an impulse train generator 204, a pitch selector 206, and a position compensator 208 as shown in FIG. 2. Generates an impulse train based on the signal passed through the waveform simplification filter, filters it according to the specified rule, finds the pitch position by applying the pitch selection rule, and then calculates the position difference between the original input signal and the simplified signal. The pitch position information is output through the compensating process.

Referring to FIG. 2, the waveform simplification filter 202 is implemented as a five point arithmetic mean filter, as shown in FIG. According to FIG. 3, a five-point arithmetic mean filter is conceptually implemented with five delayers 302, 304, 306, 308, 310, four adders 312, 314, 316, 318, and a multiplier 320. Arithmetic mean of the input signal is output.

In addition, the impulse train generator 204 of FIG. 2 defines a set of impulses having values only at maximum and minimum points of the input signal. In particular, in the present invention, as shown in Fig. 4, the impulse train filter is not simply divided into a conventional signal disregarding section and an exponentially decreasing section. The pitch period can be detected more accurately by moving the pitch point. That is, in the present invention, in order to find the correct pitch point, even if there is an impulse larger than the magnitude proportional to the distance from the previous pitch point in the signal ignore section, a method of moving the pitch point is selected. At this time, the lower limit of the input signal is set to about 1.5 times the average noise value to define the effective section of the pitch detection process. In the embodiment of the present invention, the six-channel impulse train filter is output using the six-channel impulse train filter.

The pitch selector 206 of FIG. 2 selects the pitch according to the pitch selection rule as shown in FIG. The pitch selection rule consists of a selection based on the position where the impulse strain channels 1 to 3 match and a selection based on the position where the channels 4 to 6 match. There are three cases of selecting the pitch point as follows.

<Selection Rule 1>

If there is an impulse of channel 2 at the current data position, at least one impulse among channels 1 and 3, and at least three impulses of channels 4 to 6 exist between the previously selected pitch point and the current position, Select the current position as the pitch point.

<Selection Rule 2>

All the impulses of channels 4 to 6 appear at the current data position, there is an impulse of channel 2 between the previous pitch point and the current position, and an impulse of channels 4 to 6 between the channel 2 impulse and the previously selected pitch point. If more than one is present, the position of the channel 2 impulse is selected as the pitch point.

Rule 3

There is an impulse of channel 4 at the current data position, there is at least one impulse among channels 5 and 6, a pitch point candidate exists between the previously selected pitch point and the current position, the pitch point candidate and the previously selected When two or more impulses of channels 4 to 6 exist between pitch points, the position of the pitch point candidate is selected as the pitch point.

As shown in FIG. 5, the procedure for applying the pitch selection rule as described above receives the current data, determines whether the data is the last data, and ends the pitch selection rule if it is the last data (S1).

If it is not the last data, it is determined whether the number of peak impulses including channel 2 is 2 or more (S2), and if yes, it is determined whether three or more belly impulses exist between the previously selected pitch point and the current position (S3). . If three or more exist as a result of the determination, the current position is selected as the pitch point (S4).

If the determination result in step S2 is No, it is determined whether there are 3 belly impulses (S5). If yes, it is determined whether there is a channel 0 other than 0 between the previously selected pitch point and the current position (S6). If yes, the position of the previous channel 1 is selected as the pitch point (S8).

If the determination result in step S5 is No, it is determined whether there are two belly impulses and channel 4 is greater than zero (S9). If yes, a pitch point candidate exists between the previously selected pitch point and the current position. In the meantime, it is determined whether two or more belly impulses exist (S10), and if yes, the position of the previous peak candidate is selected as the pitch point (S11).

The concept of the position compensator (208 in FIG. 2) is as shown in FIG.

In general, the peak point position of the signal passed through the waveform simplification filter does not coincide with the peak point position of the original input signal, so that accurate results can be obtained when extracting and analyzing the pitch of the original input signal without compensating for this difference. none. The position compensator 208 compensates for the difference so that the position of the correct pitch can be found even when the waveform is extracted from the input signal. The position compensator 208 searches for a data area of the same size as the degree of median filter used for the arithmetic mean filter centering on the position of the pitch point obtained from the signal passed through the waveform simplifying filter and finds the maximum value. .

As described above, the method of predicting the pitch period using the impulse train according to the present invention provides not only accurate prediction of the pitch period but also a basis for analyzing the pitch data of one period and extracting and comparing the characteristics thereof, thereby providing a waveform for speech recognition. Effective for analyzing or extracting speaker characteristics.

Referring back to FIG. 1, the pitch data analyzer 104 includes a pitch extractor 702, a pitch data normalizer 704, a feature extractor 706, a standard pitch database 708, a feature vector, as shown in FIG. 7. Comprising a comparator 710 receives pitch position information, normalizes pitch data, extracts a feature vector, and generates a voiced phonetic string by comparing the registered feature pitch with standard pitch data based on the extracted feature vector and pitch length information. do. The structure of the standard pitch database used at this time is shown in Table 1 below.

Phoneme 70 80 90 ....... 340 350 360 370 Ah on this Five Ooh Ugh uh ....

Referring to Table 1, the standard pitch database distinguishes pitch lengths by phonemes in predetermined frequency bands (for example, 70 Hz to 370 Hz bands) at predetermined frequency intervals (for example, 10 Hz).

Referring to FIG. 7, the pitch extractor 702 receives pitch position information, extracts a pitch from an input signal, and outputs pitch length information to the feature vector comparator 710 and the pitch data standardizer 704.

The pitch data normalization unit 704 receives the pitch length information and normalizes it as shown in FIG. 12. A pitch data normalization process will be described below with reference to FIG. 12.

First, the slope A of the line connecting two adjacent pitch points in the entire syllable data is obtained. Next, the minimum position is found by searching a region for a predetermined time (eg, about 10 ms) based on each pitch point, and the slope B of the line connecting the two minimum points is obtained. Subsequently, the average C of two slopes is obtained according to Equation 1 below, and the standardized slope A 'is obtained by subtracting the average gradient C from the original slope A (A' = A-C). The n th pitch data normalized as above may be obtained as in Equation 2 below.

In Equation 2, X '(n) is normalized pitch data, and X (n) is original pitch data.

The feature extractor 706 extracts the feature vector from the normalized pitch data. The feature vector comparator 710 compares the feature vector input from the feature extractor 706 and the pitch length information input from the pitch extractor 702 with reference values of the standard pitch database 708 to determine the voiced phone string.

Referring to FIG. 1, the unvoiced identifier 106 is composed of an unvoiced classifier 802, an unvoiced feature extractor 804, an unvoiced standard database 806, and an unvoiced feature comparator 808, as shown in FIG. 8. It receives location information and voice signal as input and extracts the signal of predetermined time (for example, 125msec) before the start point of pitch section and decides as 'unvoiced section', and analyzes the voice signal of the section as loud noise, rupture sound, nasal sound After classifying, the unvoiced feature vector is extracted and compared with unvoiced standard data.

Referring to FIG. 8, the unvoiced classifier 802 receives a voice signal and pitch position information, extracts an unvoiced signal from an unvoiced section, and the unvoiced feature extractor 804 analyzes the unvoiced section of the unvoiced section, and generates unvoiced sounds, broken sounds, and the like. After extracting the unvoiced feature vector, the unvoiced feature comparator 808 compares the unvoiced feature vector with unvoiced standard data to determine the unvoiced character string.

Referring back to FIG. 1, the silence section finder 108 includes a zero crossing rate measuring unit 902, an average sound pressure measuring unit 904, and a silence section extracting unit 906 as shown in FIG. 9. After inputting, measure Zero Crossing Rate and Average Amplitude. If both magnitude and zero crossing are less than the reference value, set as silent period.

Referring to FIG. 9, the zero crossing rate measuring unit 902 measures a zero crossing rate in an input signal, the average sound pressure measuring unit 904 measures an average sound pressure in an input signal, and the silent section extraction unit 906 includes a zero crossing rate. And average sound pressure and pitch position information are received and silence section information is output.

Referring again to FIG. 1, the syllable separator 110 includes a string information synchronization unit 1002 and a syllable separator 1004, as shown in FIG. 10, for a voiced phonetic string, an unvoiced phonetic string, and a silent section. Set the boundary of single syllables by receiving the information, and set the most frequent voiced phonetic string among the voiced phonetic strings within the boundary as 'neutral', and the unvoiced lower case letter obtained from the unvoiced section preceding the pitch section is called 'first', and the pitch section The unvoiced lower case letters obtained from the following pitch intervals or silent intervals are set to 'jongseong' and combined to generate phonetic characters.

Referring to FIG. 10, the string information synchronization unit 1002 receives voiced phonetic strings, unvoiced strings, and silent section information, and divides them into initial, neutral, and final syllables, and the syllable division unit 1004 is pronounced according to syllable division rules. Print a string.

In FIG. 1, the language analyzer 112 is composed of a language / grammar database 1102 and a language model unit 1104 as shown in FIG. 11 and receives a pronunciation string obtained from a syllable branch 110. Compare with, and apply grammar rules to generate standard language strings.

13 is a diagram illustrating an example of a voice recognition process according to the present invention.

Referring to FIG. 13, when a single syllable “strong” is pronounced, a voice signal waveform is divided into a silent section, an unvoiced section (first section), a pitch section (neutral section), a weak pitch section (a final section), and a silent section in a time domain. Appear separately. According to the present invention, the unvoiced phone "a" is recognized by the waveform analysis of the unvoiced section, and the voiced phone is analyzed in the pitch section to detect a plurality of "ㅏ" features. Subsequently, the unvoiced phoneme “ㅇ” is detected in the weak pitch section to define a phoneme string. Then, the word "strong" is recognized by combining the lower and lower letters according to the syllable classification rule.

As described above, according to the present invention, since the pitch of the voice signal is detected and pattern matching is performed by pitch period, the phoneme is discriminated so that the database and the calculation capacity are small, so that the voice can be accurately recognized by a relatively small computer. . Therefore, the present invention can be widely used in the voice recognition means (STT) of a portable device such as a mobile phone or a PDA. In addition, there is no limit to the number of recognized words by performing the recognition process in phoneme units.

Claims (14)

A pitch detector for receiving a voice signal and detecting a pitch to output pitch position information; A pitch data analyzer configured to receive pitch position information from the pitch detector, standardize pitch data, extract feature vectors, and compare voice data with pre-registered standard pitch data to generate voiced phonetic strings; A silent section searcher which receives a voice signal and detects a silent section and outputs silent section information; An unvoiced sound discriminator that receives the pitch position information and the silence section information, determines an unvoiced sound section, and outputs an unvoiced phone string by receiving the sound signal; A syllable branch that receives the voiced phoneme string, the unvoiced phone string, and the silent section information to generate a single syllable corresponding to a phonetic character; And And a language analyzer configured to receive a single syllable of the syllable branch and apply a grammar rule to generate a standard language string. The method of claim 1, wherein the pitch detector A waveform simplification filter that receives a voice signal and simplifies the waveform; An impulse train generator for generating an impulse train of a predetermined channel based on an output of the waveform simplification filter; A pitch selection unit detecting a pitch position by applying a predetermined pitch selection rule; And And a position compensator for compensating the pitch position detected by the pitch selector by a position difference between the original input signal and the simplified signal. The speech recognition system of claim 2, wherein the waveform simplification filter is implemented with an arithmetic mean filter. The voice recognition system of claim 2, wherein the pitch selector selects a pitch point based on a position at which the impulse train channels 1 to 3 match, or selects a pitch point based on a position at which the channels 4 to 6 match. The method of claim 1, wherein the pitch data analyzer A pitch extractor for receiving pitch position information and extracting pitch from an input signal to output pitch length information; A pitch data normalizing unit configured to receive and standardize the pitch length information; A feature extractor for extracting feature vectors from the normalized pitch data; A standard pitch database storing predefined standard pitch data; And a feature vector comparator configured to determine a voiced phonetic string by comparing the feature vector input from the feature extractor and the pitch length information input from the pitch extractor with reference values of the standard pitch database. 6. The speech recognition system of claim 5, wherein the standard pitch database distinguishes pitch lengths by phonemes in predetermined frequency intervals in a predetermined frequency band. The method of claim 5, wherein the pitch data normalization unit Find the slope A of the line connecting two adjacent pitch points, search the predetermined time domain based on each pitch point, find the minimum point, and find the slope B of the line connecting the two minimum points. ), Then subtract the mean slope C from the original slope A to find the normalized slope A '(A' = AC), nth pitch data X (n) Voice recognition system, characterized in that the standardized operation. The method of claim 1, wherein the silent section searcher A zero crossing rate measuring unit measuring a zero crossing rate from an input signal; An average sound pressure measuring unit measuring an average sound pressure in the input signal; Speech recognition system, characterized in that composed of a silent section extraction unit for receiving the zero crossing rate, average sound pressure, pitch position information and outputs silent section information. The method of claim 1, wherein the unvoiced identifier An unvoiced sound classifier which receives an audio signal and pitch position information and extracts an unvoiced sound signal from the unvoiced sound section; An unvoiced feature extractor for extracting unvoiced feature vectors by analyzing voice signals of unvoiced sections, dividing unvoiced sounds into friction sounds, burst sounds, nasal sounds, etc .; A voice recognition system comprising: an unvoiced feature comparator for comparing unvoiced feature vectors with unvoiced standard data to determine unvoiced strings. A waveform simplification filter that receives a voice signal and simplifies the waveform; An impulse train generator for generating an impulse train of a predetermined channel based on the output of the waveform simplification filter; A pitch selection unit detecting a pitch position by applying a predetermined pitch selection rule; And And a position compensator for compensating the pitch position detected by the pitch selector by a position difference between the original input signal and the simplified signal. 11. The pitch detector of claim 10, wherein the waveform simplification filter is implemented as an arithmetic mean filter. The method of claim 10, wherein the pitch selector When the impulse train generator generates an impulse train of 6 channels, Selection Rule 1: Impulse of channel 2 exists at the current data position, at least one impulse exists between channel 1 and channel 3, and at least three impulses of channels 4 to 6 between the previously selected pitch point and the current position. If present, select the current position as the pitch point, Selection Rule 2: Impulses of channels 4 to 6 appear at the current data position, an impulse of channel 2 exists between the previous pitch point and the current position, and channels 4 to 6 between the channel 2 impulse and the previously selected pitch point. If there are two or more impulses, select the position of the channel 2 impulse as the pitch point, Selection Rule 3: There is an impulse of channel 4 at the current data position, at least one impulse among channels 5 and 6, a pitch point candidate exists between the previously selected pitch point and the current position, and the pitch point candidate And if more than two impulses of channels 4 to 6 exist between the previously selected pitch points, the pitch detector is selected as the pitch point. In the speech text conversion (STT) method of receiving a voice signal and generating a character string corresponding to the voice signal, Detecting a pitch from the input voice signal; Analyzing the detected pitch position information and a voice signal to set a pitch section, a silent section, and an unvoiced section; Discriminating voiced phonemes by comparing one period pitch pattern in the pitch period with predetermined standard data; Discriminating unvoiced phonemes by comparing one period pitch pattern in the unvoiced sound interval with predetermined standard data; And And combining the determined voiced phones and unvoiced phones according to a predetermined rule to generate single syllables. 15. The voice recognition method of claim 13, further comprising: setting a weak pitch section after the pitch section, and determining a voiced consonant coming into the final pitch from the weak pitch section.