KR100306205B1 - Text-to-speech processing method and continuous speech recognition method using pronunciation connection graph - Google Patents

Abstract

The present invention provides a method for efficiently expressing and processing in graph form by integrating morphological parts-of-speech, prototype, surface, pronunciation, and connection information for a sentence in a system for processing a speech such as a TTS system and a continuous speech recognition system. It is about.

A computer readable recording medium having recorded thereon a pronunciation connection node constituting a pronunciation connection graph, the pronunciation connection node comprising: a morpheme part-of-speech field for storing a part-of-speech part of speech; A morpheme pronunciation field for storing a string in which morphemes are pronounced; A probability information field for storing a probability that a morpheme is pronounced according to a string stored in the morpheme pronunciation field; A left phonological access information field for storing the spelling and pronunciation of the first phoneme of the morpheme and a right phonological access information field for storing the spelling and pronunciation of the last phoneme of the morpheme; And a pre-pronouncement node field that stores pointers for accessing one or more pronunciation-access nodes that stored a previous morpheme that is pronounced before morphemes, and one or more pronunciation-access nodes that store a later morpheme pronounced after the morphemes. After storing the pointers, the pointers have a pronunciation access node field.

According to the present invention, by expressing several candidate pronunciations appearing in one morpheme in the form of a pronunciation connection graph, overall performance can be improved while saving memory used in the TTS system and continuous speech recognition system.

Description

Text-to-speech processing method and continuous speech recognition method using pronunciation connection graph

The present invention integrates morphological parts-of-speech, prototype, surface, pronunciation, and connection information for a sentence in a speech processing system such as a text-to-speech (TTS) system and a continuous speech recognition system. It's about how you can handle it.

In general, a system for processing a speech such as a TTS system and a continuous speech recognition system is divided into several detailed modules according to its internal functions, and each detailed module operates the entire system while exchanging processed information. At this time, the format used to express the information to be sent and received has a great effect on the performance of the system.

The present invention has been created by the necessity of the above, a computer-readable recording medium recording a pronunciation connection node constituting a pronunciation connection graph for efficiently representing information processed in the TTS system and continuous speech recognition system and the pronunciation It is an object of the present invention to provide a TTS processing method and a continuous speech recognition method using a connection graph.

1 illustrates a data structure of a node used in a pronunciation connection graph according to the present invention.

2 illustrates a pronunciation connection graph used in a TTS system.

3 is a flowchart illustrating a TTS process using a pronunciation access node according to the present invention.

4 illustrates a pronunciation connection graph used for continuous speech recognition.

5 is a flowchart illustrating a continuous speech recognition process using a pronunciation access node according to the present invention.

In order to achieve the above object, in a computer-readable recording medium recording a pronunciation connection node constituting a pronunciation connection graph representing information processed in a TTS system for converting one text sentence into speech according to the present invention. The pronunciation access node comprises: a morpheme part-of-speech field for storing a part-of-speech part of speech; A morpheme pronunciation field for storing a string in which the morpheme is pronounced; A probability information field for storing a probability that the morpheme is pronounced according to a string stored in the morpheme pronunciation field; A left phonological access information field for storing the spelling and pronunciation of the first phoneme of the morpheme, and a right phonological access information field for storing the spelling and pronunciation of the last phoneme of the morpheme; And accessing a pre-pronunciation node field that stores pointers for accessing one or more pronunciation-access nodes that stored a previous morpheme that is pronounced before the morpheme and one or more pronunciation-access nodes that store a later morpheme that is pronounced after the morpheme. After storing the pointers for a phonetic access node characterized in that it comprises a field.

The pronunciation access node in the computer-readable recording medium includes: a morpheme circular field for storing the prototype of the morpheme; And a morpheme surface field for storing the character string expressed in the text sentence.

In order to achieve the above another object, a computer-readable recording recording a pronunciation connection node constituting a pronunciation connection graph representing information processed in a continuous speech recognition system for recognizing a speech signal of one sentence according to the present invention. 12. The medium of claim 1, wherein the pronunciation access node comprises: a morpheme surface field for storing a string in which a morpheme is expressed in a text sentence, for one morpheme; A morpheme pronunciation field for storing a string in which the morpheme is pronounced; A probability information field for storing the support to be recognized and extracted from the morphemes; And accessing a pre-pronunciation node field that stores pointers for accessing one or more pronunciation-access nodes that stored a previous morpheme that is pronounced before the morpheme and one or more pronunciation-access nodes that store a later morpheme that is pronounced after the morpheme. After storing the pointers for a phonetic access node characterized in that it comprises a field.

The pronunciation access node of the computer-readable recording medium includes: a morpheme part-of-speech field storing the morpheme part-of-speech; A morpheme circular field for storing the prototype of the morpheme; A start field for storing a start point of the pronunciation of the morpheme and an end field for storing a start point of the pronunciation of the morpheme; And a left phonological access information field storing spelling and pronunciation of the first phoneme of the morpheme, and a right phonological access information field storing spelling and pronunciation of the last phoneme of the morpheme.

In order to achieve the above another object, a method of converting a text sentence into speech using a pronunciation connection graph according to the present invention includes (a) morphological analysis and tagging of the text sentence to form a circular, surface layer of each morpheme And determining the part of speech; (b) extracting words between each of the morphemes determined in step (a); (c) normalizing morphemes by representing symbols not represented in Korean in Korean; (d) generating one or more pronunciation strings to be pronounced using dictionaries and CCV rules for each morpheme with normalization, and generating one pronunciation connection node for each pronunciation string to construct a pronunciation connection graph; (e) checking the morphological phonological connection of each pronunciation access node constituting the pronunciation access graph using left and right phonological variations, and deleting the unaccessible pronunciation access node from the pronunciation access graph; And (f) generating a voice according to an optimal pronunciation access string consisting of morphological pronunciation fields of pronunciation access nodes of a path having the highest probability information in the pronunciation access graph in which the phonological access test is performed.

In order to achieve the above object, a method of continuously recognizing a speech signal of one sentence by using the pronunciation connection graph according to the present invention comprises: (a) processing a speech signal of the one sentence to generate a feature vector; Obtaining; (b) generating a pronunciation access node by recognizing morphemes according to the feature vector; (c) checking phonological access information between the morpheme of the pronunciation access node and the morpheme of the previous node of the pronunciation access graph, storing the probability information if possible, and connecting the pronunciation access node to the pronunciation access graph; (d) repeating steps (b) to (c) for all the feature vectors obtained in step (a); (e) constructing an optimal pronunciation access graph composed of a predetermined number of paths having a high probability using probability information of each pronunciation access node constituting the pronunciation access graph.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

As shown in FIG. 1, the <partial speech>, <circular>, <surface>, <pronounce>, <left phonological connection information>, <right phonological connection information>, and the like for a morpheme are represented in one node structure. By expressing and connecting only node structures that can be connected in a phonological stage in a graph form, as shown in FIGS. 2 and 4, the entire sentence can be expressed in various speech language processing systems.

In order to express the morpheme pronunciation in the form of a graph, each pronunciation access node has a link to which the front and back pronunciation access nodes can be connected, that is, the <before pronunciation access node> and the <after pronunciation access node>.

When used in a TTS system, the <probability information> shown in FIG. 1 denotes a probability to be expressed by a corresponding pronunciation, and when used in continuous speech recognition, it is supported by a recognized morpheme.

When the pronunciation connection graph is used for continuous speech recognition, fields for indicating the start time and the end time of the pronunciation, i.e., <start> and <end>, are additionally used.

<Left / Right Phonological Connection Information> includes information on phonological changes that occur when the morpheme is pronounced. That is, the <left phonological access information> field is for storing the spelling and pronunciation of the first phoneme of the morpheme, and the <right phonological access information> field is for storing the spelling and pronunciation of the last phoneme of the morpheme. Using this <left / right phonological connection information>, it is possible to judge whether the adjacent <pronouncing access node> and <after pronunciation access node> are correctly connected.

Phonological Connection Checking Table Left stem stem right connection information Accessible Right stem stem left connection information D * [* | ㄴ] <==> e * [ㄱ = ㄲ | *] e * [ㄷ = ㄸ | *] e * [ㅅ = ㅆ | *] e * [ㅈ = ㅉ | *] D * [* | ㅁ] <==> e * [ㄱ = ㄲ | *] e * [ㄷ = ㄸ | *] e * [ㅅ = ㅆ | *] e * [ㅈ = ㅉ | *] H * [* | ㄴ] <==> e * [ㄱ = ㄲ | *] e * [ㄷ = ㄸ | *] e * [ㅅ = ㅆ | *] e * [ㅈ = ㅉ | *] H * [* | ㅁ] <==> e * [ㄱ = ㄲ | *] e * [ㄷ = ㄸ | *] e * [ㅅ = ㅆ | *] e * [ㅈ = ㅉ | *] D * [* | ㄵ: ㄴ] <==> e * [ㄱ = ㄲ | *] e * [ㄷ = ㄸ | *] e * [ㅅ = ㅆ | *] e * [ㅈ = ㅉ | *] D * [* | ㄻ: ㅁ] <==> e * [ㄱ = ㄲ | *] e * [ㄷ = ㄸ | *] e * [ㅅ = ㅆ | *] e * [ㅈ = ㅉ | *] H * [* | ㄵ: ㄴ] <==> e * [ㄱ = ㄲ | *] e * [ㄷ = ㄸ | *] e * [ㅅ = ㅆ | *] e * [ㅈ = ㅉ | *] H * [* | ㄻ: ㅁ] <==> e * [ㄱ = ㄲ | *] e * [ㄷ = ㄸ | *] e * [ㅅ = ㅆ | *] e * [ㅈ = ㅉ | *] D * [* | ㄴ] <==> b * [ㄱ = ㄲ | *] b * [ㄷ = ㄸ | *] b * [ㅅ = ㅆ | *] b * [ㅈ = ㅉ | *] D * [* | ㅁ] <==> b * [ㄱ = ㄲ | *] b * [ㄷ = ㄸ | *] b * [ㅅ = ㅆ | *] b * [ㅈ = ㅉ | *] H * [* | ㄴ] <==> b * [ㄱ = ㄲ | *] b * [ㄷ = ㄸ | *] b * [ㅅ = ㅆ | *] b * [ㅈ = ㅉ | *] H * [* | ㅁ] <==> b * [ㄱ = ㄲ | *] b * [ㄷ = ㄸ | *] b * [ㅅ = ㅆ | *] b * [ㅈ = ㅉ | *] D * [* | ㄵ: ㄴ] <==> b * [ㄱ = ㄲ | *] b * [ㄷ = ㄸ | *] b * [ㅅ = ㅆ | *] b * [ㅈ = ㅉ | *] D * [* | ㄻ: ㅁ] <==> b * [ㄱ = ㄲ | *] b * [ㄷ = ㄸ | *] b * [ㅅ = ㅆ | *] b * [ㅈ = ㅉ | *] H * [* | ㄵ: ㄴ] <==> b * [ㄱ = ㄲ | *] b * [ㄷ = ㄸ | *] b * [ㅅ = ㅆ | *] b * [ㅈ = ㅉ | *] H * [* | ㄻ: ㅁ] <==> b * [ㄱ = ㄲ | *] b * [ㄷ = ㄸ | *] b * [ㅅ = ㅆ | *] b * [ㅈ = ㅉ | *]

The morpheme parts of speech shown in the pronunciation access node and the left / right phonological access information may be used to represent access information in the following format.

Part-of-speech [left phoneme connection information | Right phoneme connection information]

Each pronunciation access node checks the accessibility according to the phonological access check table that can check whether the connection between the two phonological access information shown in Table 1 is possible using the access information of the above format. The morpheme parts D, H, e, and b in the connection information of the table represent verbs, adjectives, endings, and auxiliary verbs, respectively, and * indicates that subclassification of each part of speech is not considered. And, 'ㄵ: ㄴ' shows that the 'ㄵ' is pronounced as 'n' when pronounced, 'ㄱ = ㄲ' is pronounced 'ㄲ' when pronounced '경' Indicates. When checking the connection with the left phonetic access node, the right phonetic access information is not used, indicating that it is not considered using '*'.

According to this format, in the first column of Table 1, verbs ending in 'b' can be connected with endings pronounced by 'ㄱ', 'ㄷ', 'ㅅ', 'ㅈ', and the last column. When interpreted, the adjective 'ㄻ' ending in 'ㄻ' is pronounced as 'ㅁ', followed by the auxiliary verbs which are pronounced by consonant with 'ㄱ', 'ㄷ', 'ㅅ' and 'ㅈ'. By referring to the contents of this table, it is possible to check the integrity of the pronunciation access graph for the entire sentence by checking whether two adjacent invention access nodes are accessible.

2 shows a pronunciation connection graph used to generate a pronunciation of a text sentence 'how many are' in a TTS system. The pronunciation connection graph determines the pronunciation of the text sentence and can be used when generating a voice in the TTS system. Each pronunciation access node extracts candidates for parts of speech, circle, surface, pronunciation, left and right phonological access information, etc., from a result of morphological analysis and part-of-speech tagging of text sentences. Linked morphemes in the phonetic connection graph also appear linked on text sentences. Therefore, when all the pronunciations from the left first pronunciation access node to the right last pronunciation access node of the pronunciation access graph are connected, the pronunciation of the entire sentence is obtained.

3 shows a TTS process using a pronunciation connection graph. When a sentence of text is subjected to morphological analysis and part-of-speech tagging, the prototype, surface, and part-of-speech are determined.

From each of the morphemes determined here, a word boundary is extracted using probability information 30 that includes a probability that a boundary of a word of speech (a unit of rest when speaking) appears. In step 310, morphological normalization is performed to represent symbols not represented in Korean, such as numbers or alphabets, in Korean.

The dictionary 31 and the CCV (consonant + consonant + vowel) rule 32 for each morpheme expressed in Korean are converted to pronunciation, and the result is represented by a pronunciation connection graph (step 320). Here, the converted pronunciation represents a candidate of the pronunciation to be converted into a voice, so that several pronunciation candidates may appear for the same morpheme. This pronunciation, which appears as a candidate, has a large amount of memory to express as a list, and has a lot of complexity in processing. Thus, by expressing the candidate pronunciations appearing in one morpheme in the form of the pronunciation connection graph proposed by the present invention, overall performance can be improved while saving memory used in the TTS system.

For each pronunciation connection node constituting the pronunciation connection graph, the morphological phonological connection check is performed using the phonological connection check table 33 as illustrated in Table 1 (step 330). The pronunciation connection node determined to be inaccessible by the phonological connection check is deleted from the pronunciation connection graph. Speech is generated according to the optimal pronunciation access string consisting of the morphological pronunciation fields of the pronunciation access nodes of the path having the highest probability information in the pronunciation access graph in which the phonological access test is performed.

4 illustrates a pronunciation connection graph generated in a process of recognizing consecutively pronounced sentences. In continuous speech recognition, since the morpheme is recognized without knowing the exact temporal boundary of the morpheme in the speech sentence, the same morpheme is recognized differently according to the time difference. Accordingly, two morphemes (0-5, 1-6) having different parallaxes (0-5, 1-6) with respect to the morpheme 'reduction' shown in FIG. 4 may be recognized, and each morpheme becomes a morpheme candidate for recognizing the entire sentence. Although the entire phonetic sentence is pronounced 'must be reduced', the current technology does not satisfy the performance of the morpheme-aware speech recognizer, so misrecognized morphemes such as 'Juri' or 'Law' appear in the pronunciation access graph. do. These misrecognized morphemes are first filtered through a phonological connection test, and can be recognized as complete sentences by searching for probability values after the entire phonetic access graph is created.

5 shows a process of recognizing continuous speech using a pronunciation connection graph. A feature vector is obtained by signal processing on a speech sentence, and the morpheme is recognized from the dictionary 50 and the Hidden Markov Model Trie (HMM Trie) 51 according to the feature vector. Create a node (step 500).

Each time one morpheme is recognized, the phoneme connection check table 52 is used to determine whether the connection is possible by examining phonological access information between the new morpheme and the previously obtained morpheme. At this time, the previously obtained morphemes are represented by the pronunciation access graph, and if the new morphemes are accessible to the previous morphemes, the morphemes are included in the pronunciation access graph together with the probability value, the start time and the end time of the morphemes (step 520). After the processing up to the last feature vector of the speech sentence in this manner (step 530), an optimal pronunciation connection graph is constructed using the probability values for the entire pronunciation connection graph (step 540). You can get N-Best sentences.

On the other hand, the embodiments of the present invention described above can be written as a program that can be executed on a computer. And, it can be implemented in a general-purpose digital computer for operating the program using a medium used in the computer. The media may be stored such as magnetic storage media (e.g., ROM, floppy disk, hard disk, etc.), optical reading media (e.g., CD-ROM, DVD, etc.) and carrier waves (e.g., transmission over the Internet). Media.

According to the present invention, by expressing several candidate pronunciations appearing in one morpheme in the form of a pronunciation connection graph, overall performance can be improved while saving memory used in the TTS system and continuous speech recognition system.

Claims (6)

A computer-readable recording medium recording a pronunciation connection node constituting a pronunciation connection graph representing information processed by a TTS system for converting a text sentence into a voice, A morpheme part-of-speech field for storing a part-of-speech in one morpheme; A morpheme pronunciation field for storing a string in which the morpheme is pronounced; A probability information field for storing a probability that the morpheme is pronounced according to a string stored in the morpheme pronunciation field; A left phonological access information field for storing the spelling and pronunciation of the first phoneme of the morpheme, and a right phonological access information field for storing the spelling and pronunciation of the last phoneme of the morpheme; And For accessing one or more pronunciation access nodes for storing pointers for accessing one or more pronunciation access nodes for storing previous morphemes that are pronounced before the morpheme, and for one or more pronunciation access nodes for storing subsequent morphemes that are pronounced after the morphemes And a pronunciation access node field after storing the pointers. The method of claim 1, A morpheme circular field for storing the prototype of the morpheme; And And a morpheme surface field for storing the character string represented in the text sentence. A computer-readable recording medium recording a pronunciation connection node constituting a pronunciation connection graph representing information processed by a continuous speech recognition system for recognizing a speech signal of one sentence, A morpheme surface field, which stores, for one morpheme, a string in which the morpheme is represented in a text sentence; A morpheme pronunciation field for storing a string in which the morpheme is pronounced; A probability information field for storing the support to be recognized and extracted from the morphemes; And For accessing one or more pronunciation access nodes for storing pointers for accessing one or more pronunciation access nodes for storing previous morphemes that are pronounced before the morpheme, and for one or more pronunciation access nodes for storing subsequent morphemes that are pronounced after the morphemes And a pronunciation access node field after storing the pointers. The method of claim 3, A morpheme part-of-speech field for storing parts of speech of the morpheme; A morpheme circular field for storing the prototype of the morpheme; A start field for storing a start point of the pronunciation of the morpheme and an end field for storing a start point of the pronunciation of the morpheme; And And a left phonological access information field for storing the spelling and pronunciation of the first phoneme of the morpheme, and a right phonological access information field for storing the spelling and pronunciation of the last phoneme of the morpheme. Readable record carrier. In the method of converting a text sentence into speech using a pronunciation connection graph, (a) morphological analysis and part-of-speech tagging of the text sentence to determine the prototype, surface and part-of-speech of each morpheme; (b) extracting words between each of the morphemes determined in step (a); (c) normalizing morphemes by representing symbols not represented in Korean in Korean; (d) generating one or more pronunciation strings to be pronounced using dictionaries and CCV rules for each morpheme with normalization, and generating one pronunciation connection node for each pronunciation string to construct a pronunciation connection graph; (e) checking the morphological phonological connection of each pronunciation access node constituting the pronunciation access graph using left and right phonological variations, and deleting the unaccessible pronunciation access node from the pronunciation access graph; And (f) generating a speech according to an optimal pronunciation access string consisting of morphological pronunciation fields of pronunciation access nodes of a path having the highest probability information in the pronunciation access graph in which the phonological access test is performed. TTS processing method using. In the method of continuous speech recognition of the speech signal of one sentence using the pronunciation connection graph, (a) processing a speech signal of the one sentence to obtain a feature vector; (b) generating a pronunciation access node by recognizing morphemes according to the feature vector; (c) checking phonological access information between the morpheme of the pronunciation access node and the morpheme of the previous node of the pronunciation access graph, storing the probability information if possible, and connecting the pronunciation access node to the pronunciation access graph; (d) repeating steps (b) to (c) for all the feature vectors obtained in step (a); (e) constructing an optimal pronunciation connection graph consisting of a predetermined number of paths having a high probability using probability information of each pronunciation access node constituting the pronunciation connection graph. Continuous speech recognition method used.