KR20170090127A - Apparatus for comprehending speech - Google Patents

Abstract

Provided by the present invention is a robust apparatus to comprehend the speech language of a conversation processing system. The speech language comprehension apparatus comprises: a morpheme analyzing unit which analyzes morphemes of an inputted sentence; a vocabulary-based conversation behavior recognizing unit which recognizes conversation behavior by performing analysis based on vocabularies in the analyzed result of the morphemes; a semantic object recognizing unit which recognizes a semantic object in the analyzed result of the morphemes by performing a chart parsing and a fuzzy matching based on a semantic object dictionary; a slot tagging unit which tags the semantic object on a slot in order to recognize the function of the semantic object; a semantic-based conversation behavior recognizing unit which recognizes conversation behavior based on semantics by performing classification according to features including the semantic object information; and a user intention candidate generating unit which generates a user intention candidate by mixing the conversation behavior recognized based on semantics and the slot tagged with the semantic object.

Description

[0001] Apparatus for comprehending speech [

The present invention relates to a speech language understanding method, and more particularly, to a robust speech language understanding apparatus and method for a speech processing system.

Recently, with the emergence of services such as an intelligent secretary, an interactive navigation system, and an interactive language learning system, the importance of dialog processing technology and language understanding technology is increasing.

In the case of the semantic parsing-based language understanding system based on the conventional rule, there is a problem that the robustness against irregular speech input or speech recognition error is poor.

Recently, machine learning based method is being used. However, in the case of a system in which a large object dictionary such as a person name, a place name, and a title should be applied, there is a limitation in the ambiguity processing and object recognition as the existing learning based language understanding system.

In order to deal with this, semantic code should be assigned to the semantic object, and semantic code based processing should be performed. However, in the machine learning method based on the existing statistics, there is a limit to the implementation of robust speech language understanding device due to difficulty in handling ambiguity about semantic objects.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a speech language understanding apparatus and a speech recognition system which are robust against the case where an inscribed speech sentence, a speech recognition error, a user speech error, There is a way to provide ...

According to another aspect of the present invention, there is provided a speech language understanding apparatus comprising: a morpheme analysis unit for analyzing morphemes of input sentences; A vocabulary-based conversation behavior recognition unit for analyzing the result of analyzing the morpheme based on a vocabulary and recognizing a conversation activity; A semantic object recognition unit for recognizing a semantic object with respect to a result of analyzing the morpheme by performing chart parsing and fuzzy matching based on the semantic object dictionary; A slot tagging unit for recognizing a slot corresponding to a functional role of the semantic object; A semantic-based dialog action recognizing unit for recognizing a conversation action by performing classification on qualities including the semantic object information; And a user intention candidate generator for generating a user intention candidate by combining a conversation action recognized based on the meaning and a slot tagged with the semantic object.

According to the present invention, there is a need to use a large-capacity object dictionary such as a place name, a name, and a title in a dialog processing system. In the conventional machine learning-based language understanding apparatus, However, according to the present invention, object recognition and semantic class ambiguity can be solved by applying a large-capacity object dictionary as a semantic class. In addition, rule-based parsing and fuzzy matching consisting of lexical-semantic patterns have the advantage of applying an object dictionary to inaccurate utterances.

1 is a block diagram illustrating an internal configuration of a speech language understanding apparatus according to an embodiment of the present invention.
FIG. 2 is a block diagram showing the internal structure of the semantic object recognition unit shown in FIG. 1. FIG.
3 is a diagram illustrating a learning process for performing a language understanding function according to an embodiment of the present invention.
4 is a flowchart illustrating a procedure of a speech language understanding method according to an embodiment of the present invention.

The present invention relates to a robust speech language understanding apparatus and method in a speech processing system, which includes a morpheme analysis unit for performing morpheme analysis, a lexical-based conversation action recognition unit for recognizing a conversation action based on a lexical form on the morpheme tagging result, A semantic-based dialog action recognizing unit recognizing a conversation action including the recognized semantic object information, a slot tagging unit for recognizing a domain slot based on a machine learning, And a user intention candidate generating unit for generating a user intention candidate.

Language understanding means the process of translating the natural language sentences used by the speakers into intentional expressions that can be understood by machines and other machines.

Intentional expressions that can be understood by the machine can be configured to include Dialog Act (DA) and slots.

The conversation action refers to the original intention or purpose of the user, and the slot corresponds to the object (object) having meaning in the sentence.

For convenience, the intent expression is expressed in the form of "conversation action (slot name = slot) ".

For example, in a vehicle navigation system, if the natural language sentence used by the speaker is "Let's go to the post office near Seoul station", the intent expression that the machine can understand is expressed as "navigate (landmark =" Seoul station ", dest =" post office " .

In order to develop such a speech language understanding device, a learning corpus in which semantic expressions are tagged is automatically constructed to extract or learn knowledge.

<landmark = Seoul Station / poi> Near <dest = Post Office / poi> Go navigate (landmark = "Seoul Station", dest = "Post Office")

"Landmark", "dest" is the slot name, and poi (point of interest) is the semantic code name.

As mentioned above, even in the same poi sense, it can be used in various slots such as "landmark, dest".

Hereinafter, a robust speech language understanding device according to embodiments of the present invention will be described in detail with reference to the drawings.

Prior to the description of the embodiments, the following embodiments combine the elements and features of the present invention in a predetermined form. Each component or characteristic may be considered optional unless otherwise expressly stated. Each component or feature may be implemented in a form that is not combined with other components or features. In addition, some of the elements and / or features may be combined to form an embodiment of the present invention. The order of the operations described in the embodiments of the present invention may be changed. Some configurations or features of certain embodiments may be included in other embodiments, or may be replaced with corresponding configurations or features of other embodiments.

 The embodiments according to the present invention can be implemented by various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

 For a hardware implementation, the method according to embodiments of the present invention may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs) , Field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of an implementation by firmware or software, the method according to embodiments of the present invention may be implemented in the form of a module, a procedure or a function for performing the functions or operations described above. The software code can be stored in a memory unit and driven by the processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor by various well-known means.

The specific terminology used in the following description is provided to aid understanding of the present invention, and the use of such specific terminology may be changed into other forms without departing from the technical idea of the present invention.

1 is a block diagram illustrating an internal configuration of a speech language understanding apparatus according to an embodiment of the present invention.

1, a speech language understanding apparatus 100 according to an embodiment of the present invention includes a morpheme analysis unit 110, a vocabulary-based dialog action recognition unit 120, a semantic object recognition unit 130, A semantic-based dialog action recognizing unit 150, and a user intention candidate generating unit 160. [

In addition, the apparatus 100 for understanding a spoken language according to an embodiment of the present invention may further include a plurality of databases interlocked with the respective structures. 1, a speech language understanding device 100 according to an embodiment of the present invention includes a morpheme tagging learning model DB 11 interlocked with the morpheme analysis unit 110, A first dialogue activity learning model DB 12 interlocked with the recognition unit 120, a semantic object dictionary 13 interlocked with the semantic object recognition unit 130, a slot tagging unit 140 linked with the slot tagging unit 140, Model DB 14 and a second conversation activity learning model DB 15 interlocked with the meaning-based conversation behavior recognition unit 150. [ 2, the semantic object dictionary 13 includes a semantic object dictionary DB 31, a lexical-semantic pattern DB 33, a semantic object dictionary index DB 35, and a semantic object tagging learning model DB (37).

The morphological analysis unit 110 performs morphological analysis and part-of-speech tagging on the speech recognition result in which the speech (or user) has recognized the speech. Here, the speech recognition result may be an input sentence in which a speech uttered by a speaker (or a user) is converted into a text form.

When the speech recognition result is inputted in the form of text in the morphological analysis unit 110, the morphological analysis unit 110 refers to the morphological tagging learning model DB (or the morpheme dictionary) 11 to perform morphological analysis and morphological tagging Information of the vocabulary, parts of speech, and prototype of the objects (or words) in the input sentence can be known.

The lexical-based dialogue action recognition unit 120 recognizes a dialogue action based on information such as vocabulary, part-of-speech, prototype, etc., obtained from the morphological analysis and morphological tagging results of the morphological analysis unit 110. For example, the conversation action can be recognized from the verb information in the input sentence.

Example: An example of vocabulary-based conversation behavior recognition

If the input sentence contains a verb "to turn on", the conversation action can be recognized as "play" based on the verb vocabulary "to turn on." If the input sentence includes the verb "to go" Based on the verb vocabulary "Let's Go", the conversation action can be recognized as "navigate".

The semantic object recognition unit 130 recognizes the semantic object in the input sentence through fuzzy matching with the rule-based parsing and the semantic object dictionary 13, Based tagging recognizes semantic objects.

Parsing is a process of analyzing the structure of an input sentence. Rule-based parsing is a process of analyzing the structure of an input sentence based on a set rule and a set rule to analyze the structure of the input sentence. In one embodiment of the present invention, a chart parsing technique can be used as rule-based parsing. Here, the chart may be a table for recording the progress of parsing.

The fuzzy matching is a method of performing matching on results similar to search conditions as well as matching results matching the search conditions.

The slot tagging unit 140 recognizes a domain slot based on a machine learning in order to recognize a functional role of a semantic object. That is, the slot tagging unit 140 tags slots in the semantic objects recognized by the semantic object recognizing unit 130.

The semantic-based conversation-behavior recognizing unit 150 recognizes the conversation-based semantic based on information including the recognized semantic object.

The user intention candidate generating unit 160 may generate an intention candidate (n-best) according to a weight for all combinations of slots tagged with the semantic object and the semantic-based conversation action at the slot tagging unit 140 ), And outputs it.

In Slot and User Intent Recognition, a slot can be treated as a sequential labeling method, which is used for recognition of a common object name (Named Entity), and a classification problem for user intention recognition.

FIG. 2 is a block diagram showing the internal structure of the semantic object recognition unit shown in FIG. 1. FIG.

2, the semantic object recognizer 130 includes a semantic dictionary identifier matching unit 131, a chart parser 133, a semantic dictionary fuzzy matching unit 135, And a tagging unit 137.

The semantic dictionary object matching unit 131 refers to the semantic object dictionary DB 31 included in the semantic object dictionary 13 shown in FIG. 1 to perform an exact matching for the input sentence with the longest priority, To recognize the semantic object.

The semantic object dictionary DB 31 stores a plurality of semantic codes corresponding to morphological analysis results performed by the morphological analysis unit 110. Here, the meaning code may be, for example, "channel" and "poi". "channel" may be a semantic code associated with a broadcast channel, and "poi" may be a semantic code associated with a region.

The chart parsing unit 133 generates an object candidate by performing a chart parsing on an input sentence using a lexical-semantic pattern built in the lexical-semantic pattern DB 33 as a rule.

The semantic pre-fuzzy matching unit 135 refers to the semantic object dictionary index DB 35, Fuzzy matching is performed on the object candidates generated by the chart parsing unit 133 to select object candidates having the highest degree of similarity (or matching rate), and a weighting factor Select the final chart. Then, the semantic pre-fuzzy matching unit 135 recognizes the semantic object for the input sentence using the selected final chart.

If the chart parsing fails, the semantic object tagging unit 137 recognizes a semantic object in the corresponding domain by a statistical machine learning method.

3 is a diagram illustrating a learning process for performing a language understanding function according to an embodiment of the present invention.

Referring to FIG. 3, in step S201, morphological analysis (or morphological tagging) is performed on the original text using the domain learning corpus.

In step S202, the morphological lexical information and the semantic information of the slot are combined to extract a lexical-semantic pattern.

Vocabulary - Semantic Pattern Example:

<dbm_channel = embbi / channel> Turn on #play (dbm_channel = "embbi")

-> Turn on <channel>

<landmark = Yeouido / poi> <dest = kbs main / poi> Let's go #navigate (landmark = "Yeouido", dest = "kbs main")

-> <poi> <poi>

Then, in step S203, the lexical-based DA classification learning step 203 performs learning for classifying the conversation behavior using lexical information and parts-of-speech information as qualities.

For example, for learning to classify conversation behaviors,

Learning corpus: <dbm_channel = embbi / channel> Let's play #play (dbm_channel = "MBC")

Original text tagging: MBC / MBC / NN key / ON / VB + OWNER / OWNER / EV + EA / LA / EE.

Qualities and DA values to use in learning: MBC / NN ON / VB Alcohol / EV / EE.

In step S204, learning for semantic object tagging is performed. BIO tagging is used in sequential labeling for semantic objects and slots. The BIO tagging method is a method of tagging with the channel "B- channel" when the channel object is started, "I- channel" if the object is continuous, and "O" when the object is not a semantic object.

For example,

Learning corpus: <dbm_channel = embbi / channel> Let's play #play (dbm_channel = "MBC")

Significance Tagging Learning data: MBC / B-channel ON / O Algebra / O / O

Next, in step S205, learning is performed to tag the slot using vocabulary, part-of-speech, and semantic information as qualities.

For example,

Learning corpus: <dbm_channel = embbi / channel> Let's play #play (dbm_channel = "MBC")

Slot Tagging Learning Data: MBC / B-dbm_channel ON / O O / O / O

Next, in step S206, learning is performed to classify the conversation behavior using lexical information, part-of-speech information, and semantic information as qualities.

Learning corpus: <dbm_channel = embbi / channel> Let's play #play (dbm_channel = "MBC")

Original text tagging: MBC / MBC / NN key / ON / VB + OWNER / OWNER / EV + EA / LA / EE.

The qualities and dialogue values to use in learning Value: MBC / NN / channel on / VB Adverb / EV / EE.

In step S207, an inverse index file for semantic object fuzzy matching is generated.

A reverse index file is a commonly used method for information retrieval, in which each word stores a pointer to a document or sentence containing the word. For example,

[Meaning Object Dictionary]

1: future love / poi

2: future love apartment / poi

3: Daewoo Future Love Apartment / poi

4: Yangpyeong Station Daewoo Future Love Apartment / poi

[Meaning Object Dictionary Index]

Future: 1,2,3,4

Treatment: 3,4

Apartments: 2,3,4

Using the semantic object dictionary index, it is easy to find a semantic object entry that contains the most specific list of words.

Hereinafter, the operation will be described in detail based on two speech recognition results input to the speech language understanding apparatus 100 according to an embodiment of the present invention.

Input sentence 1) Voice recognition result: " Embassy Network  Let me see some football. "

1) Morpheme Analysis: The morpheme analysis unit 110 performs morphological analysis and part-of-speech tagging on the input sentence "Embassy Net Football Show" and analyzes tagging results of the input sentence.

The result of morphemic analysis: Embassy net / Embassy net / NN soccer / NN beaver / VB + crab / EE.

2) Recognition of Vocabulary Based Conversation Behavior: The lexicon based user's intention recognition unit 120 recognizes the morpheme analysis result of the morpheme analysis unit 110 (MBCNET / MBCNET / NN soccer / NN / VB / (12) to recognize the conversation action with the qualities of / AD key / ON / VB + week / week / EV + word / LA / EE.

Vocabulary based conversation recognition result: #play ()

3) Semantic object recognition

3.1) Semantic dictionary object matching unit 131: The semantic dictionary object matching unit 131 in the semantic object recognizing unit 130 refers to the semantic object dictionary DB 31 and performs the semantic object dictionary matching process on the " Embassy net " For example, "channel" and "poi" "Embassy net" has the ambiguity of Embassy net channel and embassy net station (poi).

Meaning Dictionary Eggjat Matching result: Embassy net / channel_poi Soccer / NNbo + Gemini key + Tagalog +

3.2) Chart parsing

Vocabulary - Meaning pattern: Turn on <channel> #play ()

Input vocabulary / meaning: embassy net / channel

In this case, since the phrase "soccer watching" is inserted in the input sentence, it will fail to perform the chart parsing because it does not match the extracted vocabulary-semantic pattern.

3.3) Semantic object tagging: In the semantic object tagging unit 137, the semantic object of the corresponding slot is recognized by the statistical machine learning method according to the failure of the chart parsing.

As a method of tagging semantic objects based on machine learning, BIO sequential labeling method used in general Named Entity tagging can be used.

The BIO method is a method of tagging B-object1 when object object1 is started, I-object1 when it is continuous, and "O" when it is not a semantic object.

When learning and applying the above object, we use vocabulary based conversation recognition result as a qualification. Therefore, in the above example, the meaning code of "Embassy" can be classified as a channel, not a poi, because it is classified by using "play ()" as its qualities.

Meaning Object Tagging Result: Embassy net / B-channel soccer / O beam / O + crab / O some / O key / O +

Meaning Object Recognition Result: Embassy net / channel soccer + key + key + word

4) Recognition of Slot and Conversation Behavior: The final user's intention recognition result can be generated by classifying the slot tagging and the conversation behavior using the semantic code of the recognized semantic object as a quality.

Slot and conversation recognition result: <dmb_channel = Embassy net> Let's watch football #play (dmb_channel = "Embassy net")

Input sentence 2) Voice recognition result: Yangpyeong station Daewoo Future Love Go

1) Morphological Analysis: The morphological analysis unit 110 performs morphological analysis and tagging on the input sentence "Yangpyeong Station Daewoo Future Love Guy ", and outputs the above tagging result.

Morphological Analysis Result: Yangpyeong Station / Yangpyeong Station / NN Daewoo / Daewoo / NN Future Love / Future Love / NN / Ga / VB +

2) Recognition of vocabulary-based conversation behavior: The vocabulary-based dialog action recognizing unit 120 recognizes the conversation action based on the morphological analysis and the tagging result of the morpheme analyzer 110 and generates the recognition result as a recognition code indicating a conversation action. A recognition code such as #navigate () may be generated if the conversation action is recognized based on the verb "Let's go".

Vocabulary-based conversation behavior recognition result: #navigate ()

3) Semantic object recognition

3.1) Semantic Dictionary Eggatch Matching: When the poi dictionary exists, it performs the longest match in the input sentence and allocates a semantic code for the range of matching input sentences as follows.

[poi object dictionary]

Yangpyeong Station / poi

Daewoo / poi

Love / poi

Future love / poi

Future love apartment / poi

Daewoo Future Love Apartment / poi

Yangpyeong Station Daewoo Future Love Apartment / poi

Meaning Dictionary Eggjat Matching Result: Yangpyeong Station / poi Daewoo / poi Future Love / poi +

3.2) Chart parsing

[Vocabulary - Meaning pattern]

<poi> is +

<poi> <poi> is +

The chart parsing unit 133 performs chart parsing using the lexical-semantic pattern as a rule. That is, candidates for semantic objects are recognized.

In the case of semantic object candidate recognition, if the matched node is the i-th morpheme node for the active chart and the next node to be matched is the semantic object node like poi, then the morphological analysis result . &Lt; / RTI &gt;

If the matching node is a jth morpheme, it creates a poi node in the range of i + 1 ~ j-1 and adds it to the inactive chart pool.

For example, when you apply the pattern "<poi> +" before you start parsing, the <poi> object node exists as the first node, and until the next node's vowel "a" Performs a search on the input morpheme list.

Then, in the third morpheme of "Yangpyeong Station / poi Daewoo / poi future love / poi + ja" morphological list, "a" comes out, so we create and add inactive chart corresponding to poi node range from 0 to 2.

poi0_2 is added and this chart matches the "<poi>" node in the pattern "<poi>", followed by the "+" character of the pattern and the "+" character of the input morpheme The entire pattern is matched, and the parsing is succeeded.

If there is a poi semantic code in the morpheme, an inactive chart corresponding to the poi node is automatically added.

Therefore, the inactive chart of poi1_2 is added, and "<poi>" is added because "<poi> <poi> is in the character" and the first <poi> and Yangpyeong station / poi match and <poi> The match for <poi> + character "also succeeds. In this way, all possible charts are generated.

3.3) Meaning pre-fuzzy matching

When the chart parsing is finished, the chart including the entire range is preferentially selected.

If such a chart does not exist, the chart range recognized in the active chart pool is used to generate a partial chart list with the longest matching priority. In the case of the above example, the following three patterns are selected as the pattern including the entire sentence range.

Yangpyeong station Daewoo future love / poi +

Yangpyeong station / poi Daewoo future love / poi +

Yangpyeong station Daewoo / poi future love / poi +

In the first place, "Yangpyeong Station Daewoo Future Love" and poi object dictionary (vocabulary - meaning pattern DB) "Yangpyeong Station Daewoo Future Love Apartment" are matched and "Yangpyeong Station Daewoo Future Love Apartment / poi + . Therefore, the result of recognizing the semantic object finally is Daewoo Daewoo Future Love Apartment / poi +.

3.4) Meaning Object tagging section: This is not performed because the parsing section succeeded in parsing.

3.5) Meaning Object Recognition Result: Yangpyeong Daewoo Future Love Apartment / poi +

4) Recognition Result of Slot and Conversation Behavior: <dest = Yangpyeong Station Daewoo Future Love Apartment> Let's go #navigate (dest = Yangpyeong Station Daewoo Future Love Apartment)

The above utterance is an inaccurate utterance, since the user originally said, "Let's go to Yangpyeong station Daewoo Future Love Apartment" and it is an accurate utterance. However, accurate slots are recognized through pre-fuzzy matching.

FIG. 4 is a flowchart illustrating a procedure of a speech language understanding method according to an embodiment of the present invention. In the description of each step below, the portions overlapping with those described with reference to FIGS. 1 and 2 are omitted or briefly described .

Referring to FIG. 4, in step S310, the morpheme of the input sentence (or speech recognition result) is analyzed.

Then, in step S320, the result of analysis of the morpheme is analyzed based on a vocabulary to recognize a conversation action. Here, the part of a vocabulary can be a verb. Recognized conversation actions can be represented (or generated) by recognition codes of a structure such as #navigate () or #play ().

In step S330, based on the semantic object dictionary 13, the semantic object is recognized using chart parsing and fuzzy matching. Here, the chart parsing is performed based on the lexical-semantic pattern DB 33, and the fuzzy matching can be performed based on the semantic object dictionary index DB 35. [

The step S330 may further include a semantic object tagging process of recognizing a semantic object by a statistical machine learning method when the chart parsing fails.

In step S340, a slot corresponding to the functional role of the semantic object is recognized. That is, the slot is tagged in the semantic object.

Then, in step S350, classification is performed based on the qualities including the semantic object, thereby recognizing the conversation action on a semantic basis.

Then, in step S360, a user intention candidate is generated by combining the conversation action recognized on the basis of the meaning in step S350 and the tag tagged to the meaning object in step S340.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood that various modifications and applications not illustrated in the drawings are possible. For example, each component specifically shown in the embodiments of the present invention can be modified and implemented. It is to be understood that all changes and modifications that come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

Claims (1)

In a robust speech language understanding device for a speech processing system,
A morpheme analysis unit for analyzing morphemes for input sentences;
A vocabulary-based conversation behavior recognition unit for analyzing the result of analyzing the morpheme based on a vocabulary and recognizing a conversation activity;
A semantic object recognition unit for recognizing a semantic object with respect to a result of analyzing the morpheme by performing chart parsing and fuzzy matching based on the semantic object dictionary;
A slot tagging unit for recognizing a slot corresponding to a functional role of the semantic object;
A semantic-based conversation behavior recognizing unit for classifying the semantic object information into qualities and recognizing a conversation action on a semantic basis; And
A user intention candidate generating unit for generating a user intention candidate by combining a dialogue action recognized based on the meaning and a slot tagged with the semantic object,
/ RTI &gt;

Images