KR20180092836A - System and method for character boundary recognition - Google Patents

Abstract

The present invention relates to a method of recognizing a character boundary by a character boundary recognition system. When a noun is found in a Korean sentence based on stored Korean postposition and noun words, a basic character boundary is marked on the found noun. If adjacent nouns in the Korean sentence marked with the basic character boundary are stored in a noun word as a single noun, or if multiple nouns are included in a single phrase or clause based on a sentence structure, the multiple nouns are merged into a single compound noun to extend the character boundary. It is possible to recognize a character boundary of nouns and compound nouns in a Korean sentence by using dictionary and syntactic analysis.

Description

System and method for character boundary recognition [

The present invention relates to a character boundary recognition system and a room.

An object appearing in a Korean sentence may be composed of plural words including a space. Conventionally, various methods have been used to identify the character boundaries of single or plural words and complex nouns in Korean sentences.

For example, it is possible to recognize an object name by using the back-off ENG grammatical property. In the case of generating a training corpus including an object name from a morpheme and a word in an input sentence, Extracts qualities, selects candidates corresponding to each word of the input sentence, and finally obtains the object name. And we learn the model to store the entropy model in the statistical database by learning the trained qualities of the backoff - yen gram extracted from the qualitative information extractor. This method has the disadvantage of extracting the qualities of sentences by using the back - off - element qualities and training the learning model accordingly.

Another method is an automatic method of constructing a corpus. In the proposed method, a method for constructing an object name dictionary storing irregular personal information, a method for constructing an object name dictionary containing at least one of a headword and a user- Extracting at least one snippet according to the data characteristic by checking the search result, acquiring the object name learning data by tagging the object name in the extracted snippet, acquiring the object name learning data based on the acquired object name learning data, And a step of determining a learning model for recognition. In this method, only one search should be executed using the dictionary and then used as the training data of the learning model.

Accordingly, the present invention provides a system and method for recognizing character boundaries of objects and compound nouns in Korean sentences using dictionary and syntax analysis.

According to another aspect of the present invention, there is provided a method of recognizing a character boundary,

Searching for a noun in the received Korean sentence based on the stored Korean search and the noun word, displaying a basic character boundary on the found noun, and determining whether the adjacent noun in the Korean sentence having the basic character boundary is a single noun And includes expanding a character boundary by merging a plurality of nouns into one compound noun if the noun is stored in a noun word or a plurality of nouns are contained in one phrase or phrase based on the sentence structure.

And expanding the noun range using the stored Korean search in a Korean sentence in which the character boundary is extended.

The step of searching for the noun may include the steps of extracting parts of speech information from the Korean sentence, dividing the Korean sentence into morpheme units including the extracted part-of-speech information, and dividing the Korean sentence into morpheme- Decomposing, and segmenting into segments.

The step of dividing the morpheme unit may include abbreviating the part-of-speech based on the extracted part-of-speech information and a predefined rule.

The step of displaying the basic character boundary may include displaying a basic character boundary in the Korean sentences separated by the phrase unit based on the noun found in the Korean sentence and the stored noun word.

Wherein the step of expanding the character boundary comprises the steps of: retrieving, in a Korean sentence in which the basic character boundary is displayed, whether an extended word including all the words between two adjacent nouns is stored as the noun word; And expanding the character boundaries by merging the boundaries between the two nouns if they are stored as words.

And expanding the noun range based on the Korean search in the Korean sentence with the character boundary extended after the step of adding to the storage module.

According to another aspect of the present invention, there is provided a system for recognizing a character boundary, the system comprising: a morpheme analysis module for receiving a Korean sentence and dividing the Korean sentence into morpheme units; And a character boundary is displayed on the sentence unit based on the stored noun words, and the character boundary detected by the noun detecting module is a character string And an adjacent noun merging module for merging the adjacent nouns to determine a character boundary after the noun unit sentence is re-searched using the stored noun words.

A first dictionary module for storing the Korean research, and a second dictionary module in which the noun words are stored, the noun words may include nouns, compound nouns, coined words, and unregistered entry words.

And a noun adding module for adding the new word and the unregistered entry word to the second dictionary module based on the sentence code indicated in the sentence when the sentence having the character boundary is received in the neighboring noun merging module.

And a noun range extension module for receiving a sentence in which the character boundary is determined in the neighbor noun merging module and expanding the boundary of the noun using the Korean search stored in the first dictionary module.

According to the present invention, the character boundaries of the general nouns and the complex nouns that are listed without boundaries in Korean sentences can be clearly grasped and applied to various natural language processing tasks.

1 is a structural diagram of a character boundary recognition system according to an embodiment of the present invention.
2 is a flowchart illustrating a method of recognizing a character boundary according to an embodiment of the present invention.
3 is a diagram illustrating an example of a result of intermediate processing for character boundary recognition according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as "comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise.

Hereinafter, a system and method for recognizing a character boundary according to an embodiment of the present invention will be described in detail with reference to the drawings. First, using the large-sized dictionary entries and POS (Part of Speech) tagging collected from the web, the character boundary of an object composed of a single word or a plurality of words in a Korean sentence and a compound noun The structure of the character boundary recognition system will be described with reference to Fig.

1 is a structural diagram of a character boundary recognition system according to an embodiment of the present invention.

1, a character boundary recognition system 100 driven by at least one processor includes a morphological analysis module 110, a noun detection module 120, a neighbor noun merging module 130, a noun adding module 140, a noun range extension module 150, a Korean inquiry dictionary module 160, and a compound noun dictionary module 170.

The morpheme analysis module 110 divides a Korean sentence input from the outside into a plurality of morpheme units including part-of-speech information. At this time, the morpheme analysis module 110 divides the unnecessary idioms or common phrases into morpheme units after eliminating candidates from the candidates through a process of reducing the list of parts of speech.

In the embodiment of the present invention, the execution of the part shrinking process using the heuristic rule will be described as an example, but the present invention is not limited thereto. Since the morpheme analysis module 110 can reduce the parts of speech or divide morpheme units into various parts, it is not limited to any one method in the embodiment of the present invention.

The noun detection module 120 receives a plurality of morpheme-unit sentences including part-of-speech information output from the morpheme analysis module 110, and outputs the sentences in units of phrases. That is, the noun detection module 120 decomposes the sentence of the morpheme unit into the unit of the word by using the Korean search stored in the Korean inquiry dictionary module 160, and outputs it as a sentence of the phrase unit. At this time, the noun detection module 120 can perform the method of decomposing the sentence into the unit of the word by using the Korean language search can be performed by various methods. Therefore, the present invention is not limited to any one method.

The noun detection module 120 firstly delimits a sentence in a phrase unit based on the noun words stored in the compound noun dictionary module 170 in sentence units.

The neighboring noun merging module 130 may determine that the noun phrase is not registered in the compound noun dictionary module 170 or that the noun is in a phrase / The nouns stored in the compound noun dictionary module 170 are re-searched in the case where nouns are not detected in the case of omissions or other foreign language notation.

That is, nouns stored in the compound noun dictionary module 170 are rediscovered by including words between neighboring detection results, in a sentence unit sentence boundary formed by the noun detection module 120 and bounded. The noun detection module 120 merges the nouns using the re-searched result.

For example, if the noun detection module 120 detects' {love} 'and' {motion} 'in a sentence of' {Love} The complex noun dictionary module 170 is re-searched with the phrase " building of love ", and it is confirmed whether or not the corresponding word is registered in the compound noun dictionary module 170. [ If the word is registered in the compound noun dictionary module 170, the neighbors are merged by merging the boundaries.

Then, the neighboring noun merging module 130 merges the neighboring detection results to determine the character boundary, assuming that no more than two nouns are contained in one phrase or phrase. For example, if multiple partial results are detected in one phrase, such as '{}' in {}, the neighboring noun merging module 130 merges neighboring nouns into {{Acksamosk}} .

When the neighboring nouns are merged in the neighboring noun merging module 130 and the sentence in which the character boundary is confirmed is received, the noun adding module 140 preliminarily adds the coined word or the unregistered vocabulary written in the sentence code. For example, if a phrase is used in quotation marks in a sentence, such as "" the term '' lavabo '' is used, it is already established that the boundaries of the noun are fixed. Therefore, if the noun phrase 'honey leprechaun' is not registered in the compound noun dictionary module 170, the new noun phrase is newly registered to update the compound noun dictionary module 170.

The noun range extension module 150 receives a sentence in which the character boundary is determined in the neighbor noun merging module 130 and expands the boundary of the noun using the search stored in the Korean search dictionary module 160. That is, based on the fact that the Korean search is used to distinguish phrases that are always attached at the end of the phrase, the noun range extension module 150 extends and confirms the noun boundary.

For example, if only 'obo' is detected as' concept 'in the process of playing' {obo} ', it is possible to detect another oboe' oboe 'by expanding the scope of noun based on' .

A method of recognizing a character boundary using the character boundary recognition system 100 will be described with reference to FIGS. 2 and 3. FIG.

2 is a flowchart illustrating a method of recognizing a character boundary according to an embodiment of the present invention.

As shown in FIG. 2, when the character boundary recognition system 100 receives a Korean sentence from the outside (S100), the sentence is divided into morpheme units and parts-of-speech information is extracted (S101). And generates a sentence of a morpheme unit that includes part-of-speech information.

The character boundary recognition system 100 performs a lexical deconstruction procedure for deleting a phonetic word or a common phrase in a sentence using a heuristic rule in a sentence of a morpheme unit (S102). If the sentence of the morpheme unit in which the part-of-speech is abbreviated is generated, the character boundary recognition system 100 decomposes the sentence into units of phrases using the previously stored Korean search, and then divides the sentence into phrases (S103).

The character boundary recognition system 100 detects a noun range by using a compound noun dictionary in which phrases classified in units of phr are stored in advance (S104). When the noun range is detected in step S104, the character boundary recognition system 100 performs a first neighbors merging procedure for merging neighboring nouns by re-searching the dictionary including words between neighboring detection results (S105 ). This will be described first with reference to FIG.

3 is a diagram illustrating an example of a result of intermediate processing for character boundary recognition according to an embodiment of the present invention.

As shown in FIG. 3, if the inputted sentence is' Maria Salomea Scorpio Skye 'and the French name is Mari Curie', as shown in the first step, the part-of-speech information is extracted, To create a sentence. In the fourth step, we can see that the sentence is divided into three phrases, {French}, {Expression}, {Name}

Thus, as shown in the seventh step, a compound noun such as {French name} is generated after merging neighboring nouns using Korean surveys. At this time, the portion indicated by "{" and "}" means the boundary of the entity (for example, name, place name, work title, etc.) to be detected by the character boundary recognition system 100.

After performing the first neighboring noun merging procedure in step S105 of FIG. 2, the character boundary recognizing system 100 executes a second neighboring noun merging procedure (S106). To perform the second neighboring noun merging procedure, the character boundary recognition system 100 uses the assumption that no more than two nouns in a phrase or phrase will be included.

For example, in step 15 shown in FIG. 3, when a plurality of partial results are detected in one phrase as in '{Maria Salome}' {Scowdowska}, as shown in step 16, Salome Asschewdowska} merges neighbors' nouns.

If the second neighbors are merged in this way, the character boundary recognition system 100 preliminarily adds new words or unregistered lemmas or the like marked with a punctuation mark (S107). Then, the noun range is expanded using the pre-stored search (S108). Through the above procedure, it is possible to apply various natural language processing tasks by clearly recognizing the character boundaries of general nouns and various compound nouns which are listed without boundaries in Korean sentences.

For example, after the establishment of the Carter Foundation after the retirement of a sentence containing a general compound noun in the character boundary recognition system (100), the Third World election surveillance activities for the realization of democracy, I tried for the disaster prevention of the disease '. Then, the character boundary recognition system 100 will result in {Carter Foundation} after {withdrawal}, and then {Carried out {election surveillance activity} and {Guinea Bug} in {Third World} De la Kunks disease} struggled for disaster.

Also, in the character boundary recognition system 100, there are a plurality of words including a plurality of words, 'foreign exchange transactions in Europe', a foreign currency financial market in which a euro dollar is developed, a new financial system such as syndicated loans and credit derivative products It is assumed that the product has been continuously developed and successfully commercialized. As a result, the character boundary recognition system 100 has resulted in {foreign exchange transactions} of {Europe} being concentrated, {foreign money market} of {Euro dollar} being developed, {syndicated loan} And other new {financial products} have been continuously developed and successfully commercialized.

Also, in the character boundary recognition system 100, there are a long-length object including a foreign language or a sentence including a business name 'Grand Intercontinental Seoul Parnass, and COEX Intercontinental 2 in Samseong-dong, Seoul, ) Is operated. As a result, the character boundary recognition system 100 operates as {{Grand Intercontinental Seoul Seoul Parnas}, {COEX Intercontinental}} in {{Samsung Seoul} and {Parnas Hotel Co., Ltd.} And "

In addition, after succeeding the throne, which is a sentence containing a long-length foreign language name not previously registered in the character boundary recognition system 100, Bhumi Phong is appointed as a regent and substitute cleaner by his uncle Chanthan Rangsitpuraya Saddy And returned to Switzerland in 1947 to finish his studies. " Then, the character boundary recognition system 100 results in the succession of {throne}, then {Bumiphone} appoints {Uncle} {Chain Nancy Situ Pra Yura Saddi} to {Regent} and { I returned to {Switzerland} in 1947 to finish my studies. "

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

Claims (14)

A method for a character boundary recognition system to recognize a character boundary,
Searching for a noun in the received Korean sentence based on the stored Korean search and noun words,
Displaying a basic character boundary on the found noun, and
If adjacent nouns in the Korean sentence with the basic character boundaries are stored in a noun word as a single noun, or a plurality of nouns are contained in one phrase or phrase based on the sentence structure, a plurality of nouns may be regarded as a single compound noun Merging to extend the character boundary
/ RTI > The method according to claim 1,
Expanding a noun range by using the stored Korean search in a Korean sentence in which the character boundary is extended,
The character boundary recognition method further comprising: 3. The method of claim 2,
Wherein the step of searching for the noun comprises:
Extracting part of speech information from the Korean sentence, and dividing the Korean sentence into morpheme units including the extracted part-of-speech information, and
Dividing Korean sentences divided into morpheme units into units of phrases based on the Korean research and dividing them into phrases
/ RTI > The method of claim 3,
Wherein dividing into morpheme units comprises:
A step of reducing the part-of-speech based on the extracted part-of-speech information and a predefined rule
The character boundary recognition method further comprising: 5. The method of claim 4,
Wherein the step of displaying the basic character boundary comprises:
A step of displaying a basic character boundary in the Korean sentences separated by the sphere unit based on the noun found in the Korean sentence and the stored noun word,
/ RTI > 3. The method of claim 2,
Wherein expanding the character boundary comprises:
Retrieving whether or not an extended word including all the words between two adjacent nouns in the Korean sentence in which the basic character boundary is displayed is stored as the noun word; and
If the expanded word is stored as the noun word, expanding the character boundary by merging the boundary between the two nouns
/ RTI > The method according to claim 6,
After the step of extending the character boundary,
If one phrase includes a plurality of character boundaries, merging the plurality of character bounds to extend the character boundary
The character boundary recognition method further comprising: 8. The method of claim 7,
Wherein the step of outputting the determined sentence includes:
Extracting a predefined sentence code in the Korean sentence in which the character boundary is extended, and
Adding the extracted punctuation word to a storage module for storing the noun phrase
/ RTI > 9. The method of claim 8,
After the step of adding to the storage module,
Expanding a noun phrase range based on the Korean search in the Korean sentence in which the character boundary is extended
The character boundary recognition method further comprising: A system for recognizing a character boundary,
A morpheme analysis module for receiving the Korean sentence and dividing the Korean sentence into morpheme units,
A noun detection module for generating a Korean sentence divided into morpheme units in the morpheme analysis module as a phrase sentence based on the Korean sentence stored in the Korean sentence and displaying a character boundary on the sentence unit sentence based on the stored noun words,
A noun unit sentence including the character boundaries displayed by the noun detecting module is re-searched using the stored noun words and an adjacent noun merge module
The character boundary recognition system comprising: 11. The method of claim 10,
A first dictionary module for storing the Korean search, and
A second dictionary module in which the noun word is stored
/ RTI >
Wherein the noun word comprises a noun, a compound noun, a coined word, and an unregistered entry word. 12. The method of claim 11,
And a noun adding module for adding the new coined word and the unregistered entry word to the second pre-module based on the sentence code indicated in the sentence when receiving the sentence in which the character boundary is confirmed in the neighbor noun merging module,
The character boundary recognition system further comprising: 13. The method of claim 12,
A noun expansion module for expanding a boundary of a noun using the Korean search stored in the first dictionary module,
The character boundary recognition system further comprising: 11. The method of claim 10,
Wherein the morpheme analysis module comprises:
Extracting part of speech information from the Korean sentence, and executing a part-of-speech reduction process based on the extracted part-of-speech information.

Images