KR20000014088A - Method of operation control in text/speech conversion system using hypertext mark-up language element - Google Patents

Abstract

PURPOSE: A method of operation control in text to speech conversion system(TTS) is provided to enable listening of a hypertext markup language(HTML) and to clarify a content transmittance using a synthetic by defining a HTML elements and an interface of the synthetic speech. CONSTITUTION: The TTS system includes a HTML document analyzer, a synthesizer control command former forming a control command into a structured data, and speech synthesizer. The method is comprising the steps of separating the markup language and interpreting structure of the sentence by inputting the document including the HTML element from the HTML document analyzer, reconstructing the HTML element with sentence unit by using the interpreted document structure transmitted from the HTML document analyzer, generating a synthetic speech by inputting the structured TTS data transmitted from the synthesizer control command former.

Description

Operation Control Method of Text / Speech Converter Using Hypertext Markup Language Element

According to the present invention, when a document written in a Hypertext Markup Language (HTML) document structure is input in a text-to-speech conversion system (TTS) implementation method, The present invention relates to a method of controlling an operation of a text / voice converter using HTML elements.

HTML is a hypertext feature that links to other Web pages and provides details on the form of text documents on the World Wide Web, such as the shape, size, and text of paragraphs. A language that has markup capabilities to define commands.

When using the text-to-speech conversion system (hereinafter referred to as TTS), HTML, a document form adopted and used by the WWW as a standard, the hypertext function is excluded. Elements of markup are used to control the operation of the TTS.

The function of the TTS is to provide a voice of various types of information to a human being who is a computer.

To this end, the TTS should be able to provide high quality synthesized speech services from the text given to the user.

High-quality synthesized sound should be clear in sound value and have high naturality by appropriately embodying rhyme elements such as cut-off, length, strength, and height.

This is the same as using the proper vocabulary, spacing, and punctuation to correctly convey meaning to the reader in the text.

However, these rhyme elements appearing in speech are the result of the combination of sentence semantic structure, syntax structure, words, articulation combination, speaker's intention, and speech rate.

Therefore, in order to generate high quality synthesized sound, the existing voice synthesizer synthesizes the sound selection, cut-off, sound length, strength and height by reading conversion, syntax structure analysis, rhyme processing, and signal processing from input text sentences. Create

That is, the conventional TTS generally goes through a three step process to generate a synthesized sound from the input text as shown in FIG.

In the first step, the language processing unit 1 separates only pure sentence text including punctuation marks from the input text.

And then, through the syntax structure analysis process from the separated sentences, the syntax structure information related to the boundary of phrases and clauses, and the grammatical function of phrases and phrases is estimated.

Finally, the sentence is converted into a phoneme string through the read conversion process.

The estimated syntax structure information and the converted phoneme string output are transmitted to a rhyme processing unit 2 which is two steps.

In the second step, the rhyme processing unit 2 inputs the syntax structure information and the phoneme strings transmitted from the language processing unit 1, and uses rules and rhyme tables, reading, cutting up and down the sound, the strength and weakness of the sound, and the length and length of the sound. Compute the prosody parameter value.

In other words, insert the appropriate rest period by applying the break reading rule according to the degree of separation at the boundary of phrase and clause, and calculate the pitch value related to intonation by using rules and tables according to the formula relation between words and assign the pitch value to the phoneme. .

And the duration of each phoneme is calculated by using the phoneme-specific duration value and change rules according to phoneme environment and syntax structure.

In addition, the energy contour of each phoneme is generated by considering the articulation characteristics and grammatical functions of each phoneme.

Rhythm parameter values estimated through the above process are transmitted to the signal processor 3 having three levels along with phoneme string information.

In the third step, the signal processing unit 3 selects a suitable piece from the synthesis unit database 4 using the phoneme sequence and the rhyme parameter value received from the rhythm processing unit 2, and processes the rhyme parameter value if necessary to join these pieces. This produces a desired synthesized sound.

Therefore, it can be seen that the existing synthesizer has to estimate the information related to the naturalness and clarity of the synthesized sound only by the input text in the language processor 1 and the rhyme processor 2.

In this case, errors in the rule application process cannot be avoided in the language processing unit and the prosody processing unit, and there is no way that the intention of the document author is reflected in the synthesized sound due to the uniform application of the rule.

As a supplementary method, a markup language that adds additional information related to prevention of analysis error and expression of user intention to a text has been proposed and attempted.

Examples include Speech Synthesis Markup Language (SSML), Oral Text Markup Language (STML), Java Speech Markup Language (JSML). And the Sable Consortion Markup Language.

The SSML is a markup language first proposed in 1995 by A. Isard of Edinburgh University using the existing standard generalized markup language (hereinafter referred to as SGML; ISO 8879: 1996) for synthesis.

The four components used in SSML are information such as phrase boundaries, highlighted words, feature phonetic notation, and inclusion of voice files.In order to improve the unnaturalness of synthesized sound by recording these additional information in text using SSML display method, it is used in synthesizer. It was.

In the meantime, Proc. “A Markup Language for Text-to-Speech Synthesis” in EUROSPEECH'97 and “SABLE: A Synthesis Markup” in tttp: //www.cst r.ed.ac.uk/projetcs/sable_spes.2.html Language (version 0.2) ”is a method of inserting text into a document, which can synthesize speech but cannot implement HTML.

In addition, the US patents "Authoring and use systems for sound synchronized animation [registration No. 5111409] and" Speech animation and infection system [registration No. 5278943] "as a prior patent is possible unlimited vocabulary speech synthesis determined by the lexical speech synthesis method, There was a problem that the HTML synthesizer could not be implemented.

Of course, in the past, each manufacturer of a synthesizer used a specific character string to input additional information and developed a method that can be used in the synthesizer independently. A joint study between Edinburgh University and Bell Laboratories was undertaken in 1996-1997.

STML uses the SSML of Edinburgh University as the basic framework, and the information related to the pitch variation range, accent pattern, and speaker characteristics is written in a form that accepts what Bell Lab was using.

Sun Microsystems also proposed JSML to add additional information to text to improve the quality and naturalness of synthesized sound.

JSML extends not only sentences but also paragraphs, and is not only a structural language that can accommodate detailed control of words, phrases, phonetic notation, accented words, boundary information, reading, pitch control, speech rate, etc. Control is also possible.

Recently Edinburgh University, Bell Labs, British Telecomm, AT & T and Sun Microsystems formed the Sable Consortium to unify the Markup Language for synthesis that can be used in any synthesis system. .1 The specification has been published.

However, the above method is to artificially add additional information related to prevention of analysis error and expression of user intention generated in the existing TTS to the text.

As described above, it can be seen that the existing synthesizer needs to estimate information related to the naturalness and clarity of the synthesized sound by only the input text in the language processor 1 and the rhyme processor 2.

In this case, errors in the rule application process cannot be avoided in the language processing unit and the prosody processing unit, and there is no way that the intention of the document author is reflected in the synthesized sound due to the uniform application of the rule.

In addition, when you want to read the text of the HTML form widely used in the Internet environment by using the existing TTS, the language processing unit 1 uses the parser to remove the additional information included in the input text and separate only pure sentence text. use.

In this case, all the additional information such as the letter size, color, shape, etc. in the document is ignored, and there is a problem that there is almost no way to reflect the intention of the document creator in the synthesized sound.

In addition, the method of using a markup language is also a level of artificially participating in the text of additional information related to the prevention of analysis errors occurring in the existing TTS and expression of user intention.

In order to solve the above problems, the present invention defines the interface between the HTML component that controls the appearance of the document, such as the type size, color, shape, etc. displayed on the screen and the speaker, tone, speech rate, intonation, intensity of the synthesized sound, and generates the synthesized sound This paper aims to achieve audible visualization of HTML document characteristics and clear differentiation of content delivery using synthesized sound.

1 is a block diagram of a conventional text-to-speech converter,

2 is a block diagram of a text / voice converter to which the present invention is applied.

* Explanation of symbols for main parts of the drawings

1: language processing unit 2: rhyme processing unit

3: signal processor 4: synthesis unit database

5: HTML Document Analyzer 6: Synthesizer Control Command Generator

7: language processing unit 8: rhyme processing unit

9: signal processing unit 10: synthesis unit database

In order to achieve the above object, the present invention provides a first method for receiving a document including an HTML component from a hypertext markup language (HTML) document analyzer, separating a markup language, and interpreting the structure of the document. A second process of reconstructing an HTML element using information of a tree structure of a document received from the HTML document analyzer, a third process of receiving a structured TTS data received from the synthesizer control command generator and generating a synthesis sound Characterized in that it comprises a.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram of a text / voice converter to which the present invention is applied, and an HTML document analyzer 5 for analyzing an HTML document, and based on the result of the analysis, a sentence and a sentence to be read, intonation, accent, A synthesizer control command generator 6 for creating a control command for the duration in the form of structured data and a voice synthesizer 7 for receiving the structured data and synthesizing the voice are provided.

The HTML document analyzer 5 receives a document containing HTML elements as input and first separates markup languages and interprets the structure of the document.

First, we check the <HTML>, </ HTML>, which mark the beginning and end of the HTML document, and read page information such as the title stored in the <HEAD>, </ HEAD> identifier section.

The body of the document, which is displayed on the actual screen, is read from the <BODY> and </ BODY> identifier sections and the parser is used to interpret the structure of the document.

Since markup language is written in a tree structure, the interpreted document has a tree structure, and each markup has one node, and the leaf of the tree has words, phrases, and clauses within sentences. Or a sentence.

And the markup element specified in the parent node is used effectively in the child node unless it is newly set in the child node.

The tree structure information of the analyzed document is passed to the synthesizer control command generator 6.

The synthesizer control command generator 6 detects the boundary of a sentence from the tree structure information received from the HTML document analyzer, and then uses the HTML elements of the nodes connected to the ends forming the best node and the sentence in units of sentences. Reconstruct the HTML element.

The elements of the text used in the reconstruction process include letter size, physical style, and logical style.

The element that represents the size of the font is <FONTSIZE = > And <H specifying a header level > Is used.

Elements representing the physical style include <B> which is a bold letter, <I> which is an italic letter, <U> which is an underlined letter, and <TT> which is a typeface letter.

Elements that represent logical styles include <EM> for highlights, <STRONG> for strong highlights, <CITE> for quoted strings, and <CODE> for computer code strings.

And an example of the mapping rule between the elements of the mapping between the HTML elements and the synthesizer control command and the synthesizer control command is shown.

Each coverage is words within the tag's identifier, as shown below.

TABLE 1

Classification tag Change way Heading <Hx> </ Hx> Insert a little pause after loud and slow vocalization. Font size <FONT SIZE = x> </ FONT> Change the sound intensity according to the font size. Font color <FONT color => </ FONT> Energy / pitch change. Physical style <B> </ B> emphasis <I> </ I> Energy / pitch change. U> </ U> Speed up and talk again <TT> </ TT> Lower the volume, increase the pitch. <STRIKE> </ STRIKE> Sound throbbing again and again. <BIG> </ BIG> Adjust the sound louder. <SMALL> </ SMALL> Adjust sound quietly. Logical style <EM> </ EM> Volume up, pitch up <STRONG> </ STRONG> Volume Up, Pitch Up <CITE> </ CITE> Volume up, pitch up <BLINK> / <BLINK> emphasis <CODE> </ CODE> Lower the volume, increase the pitch <KDB> </ KDB> Lower the volume, increase the pitch Specific text <BLOCKQUOTE> </ BLOCKQUOTE> Tone change, a little slow After talking again and again, break <A HREE=> </A> After talking again and again, break <A NAME=> </A> After talking again and again, break <A REL=> </A> After talking again and again, break A REV => </A> After talking again and again, break A TITLE> </A> After talking again and again, break Document Title <TITLE> </ TITLE> After accenting again and again, break

What was thus produced is the structured input data for TTS of FIG.

The generated input data for the structured TTS is passed to the structured data speech synthesizer 7.

Table 2 below shows an example of structured input data for TTS.

TABLE 2

Example of an HTML document Input data for structured TTS <html> <head> <title> Test Page> <title> </ head> <body background = "backround / ibori_back.gif"> <center> <b> This is an example. </ b> <p> < / center> <h2> Header 1 <br> <h2> The synthesizer creates <b> combinations of different voices </ b> based on the <i> letter shape. </ body> </ html> <EMPH> This is an example. </ EMPH> <VOLUME LEVEL = "+ 40%"> <RATE SPEED = "-20%"> Heading 1 </ RATE> </ VOLUME> <BREAK> The synthesizer is <EMPH> <PITCH BASE = "20%"> Creates a synthesized sound of <EMPH> different voices </ PITCH> </ EMPH> depending on the shape of the letters.

The structured data speech synthesizer 7 receives the structured TTS input data received from the synthesizer control command generator 6 and generates a synthesized sound for the sentence by adjusting the reading, intonation, stress, and duration in the sentence. Output to the outside.

As described above, the present invention can be applied in various fields, such as communication services such as web browsing reading, e-mail reading, and reading novels on-demand (fairy tale), by generating distinctive synthesized sounds according to the characteristics of visible HTML documents. Excellent effect

Claims (5)

In the text / voice conversion method, An HTML document analyzer that analyzes hypertext markup language (HTML) documents, and uses the results of the analysis to write structured data in the form of commands to control the reading, intonation, accent, and duration of sentences and sentences A synthesizer control command generator and a speech synthesizer for inputting structured data and synthesizing a speech, and separating a markup language by receiving a document including an HTML component as an input from the HTML document analyzer and dividing a markup language. A first process of interpreting; A second step of reconstructing the HTML elements in sentence units using the HTML elements of the structure of the parsed document received from the HTML document analyzer; And reconstructing the HTML element, and receiving a structured text / speech converter (TTS) data received from the synthesizer control command generator to generate synthesized sounds. Operation control method of text / voice converter using The method of claim 1, wherein the first process is A first step of identifying identifiers indicating the start and end of an HTML document and reading page information of a title stored in the start identifier section; A second step of reading the body, which is the content of the document appearing on the actual screen after performing the first step, from the body identifier section and using the parser to interpret the structure of the document; And a third step of transmitting the tree structure information in the structure of the parsed document to the synthesizer control command generator. The method of claim 1, wherein the second process Detecting a boundary of a sentence from tree structure information of a document received from a hypertext markup language (HTML) document analyzer; After performing the first step, a hypertext markup language element comprising a second step of reconstructing the HTML element by sentence unit using the HTML elements of nodes connected to the top node and the ends of the sentence unit in sentence units. Operation control method of text / voice converter using. The method of claim 1, wherein the second process An actual text size specified using an element for specifying a size of a character and an element for specifying a header level as an element of a body used in a reconstruction process when reconstructing the hypertext markup language (HTML); A physical style using bold text, italic text display, underline text display, and typographic text display as elements; A method of controlling the operation of a text / to-speech converter using a hypertext markup language element characterized by reconstruction using a logical style using highlighting, strong highlighting, quoted string representation, and computer code string representation as elements. The method of claim 1, wherein the third process is A first step of receiving input data for a structured text-to-speech converter (TTS) from a synthesizer control command generator; Using a hypertext markup language element, characterized in that the second step of generating a compound sound for the sentence by outputting the cut-out, intonation, stress, duration of the structured TTS input data and output it to the outside Method of controlling operation of text / voice converter.