KR100289332B1 - Automatic Word Construction System for Electronic Documents and Method - Google Patents

Abstract

The present invention relates to an apparatus and method for constructing a database composed only of configuration information of a specific word from an electronic document. The present invention relates to a database that is divided into sentence units and word units to be particularly suitable for searching for necessary information or extracting usage of a specific word. An apparatus and method for automatically constructing a word database of electronic documents are provided.

The present invention is a location information management device for storing the position information of the sentence and the word separated from the electronic document, the frequency of occurrence of the specific word extracted by the location information management device and the left and right word information of the specific word And a frequency information management device which separates the words extracted by the location information management device into morpheme units and gives tag information, and constructs the location information, frequency information, and analysis information of the words in a database format. An automatic word database construction apparatus for electronicized documents, including an information management device for managing the device, is provided.

Description

Automatic Word Construction System for Electronic Documents and Method

The present invention relates to an apparatus and method for constructing a database composed only of configuration information of a specific word from an electronic document. The present invention relates to a database that is divided into sentence units and word units to be particularly suitable for searching for necessary information or extracting usage of a specific word. An apparatus and method for automatically constructing a word database of electronic documents are provided.

The conventional Korean information processing technology could not process a large amount of data, and it was difficult to construct a large amount of electronicized Korean documents. In addition, these small data sets were only separate resources and were not used in the sense of linking or sharing with each other. Therefore, unnecessary work or additional work had to be executed for storage or operation.

The part-of-speech analysis technique in English can have a perfect analysis rate because it is a technique for analyzing fixed words, but it can be solved by using the analyzed information for the complete analysis of Korean, which is a deadlock with a large change in morphology. However, the above-described conventional technology does not utilize the previously analyzed information, and there is a problem in that each Korean document is analyzed separately.

Accordingly, the present invention has been made to solve the problems of the prior art as described above, by constructing a language information warehouse that provides information in a form that can be reprocessed by separating and storing the electronic document in sentence units and word units The purpose of the present invention is to provide a word database automatic construction apparatus and method for electronic documents.

1 is a block diagram of a word database automatic construction device according to an embodiment of the present invention,

2 is a flow chart of a word database automatic construction method according to an embodiment of the present invention;

3 is an exemplary diagram showing the contents of word database data according to the present invention.

Explanation of symbols on the main parts of the drawings

10: information management device 20: encoder

30: location information management device 30a: sentence separation device

30b: word separator 30c: electronic document

40: frequency information management device 50: word management device

60: analysis information management device 60a: morphological information analysis device

60b: tagger information providing device 60c: stemming dictionary search device

60d: Rules for the construction of words 60e: Korean morpheme electronic dictionary

In order to achieve the above object, a word database automatic construction device of an electronic document of the present invention comprises: a position information management device for storing position information of a sentence and a word separated from the electronic document;

A frequency information management device for storing the appearance frequency of the specific word extracted by the location information management device and the left and right word information of the specific word,

An analysis information management device that separates words extracted by the location information management device into morpheme units and gives tag information; and

And an information management device for constructing and managing the word location information, frequency information, and analysis information in a database format.

In addition, the automatic word database construction method of the electronic document according to the present invention, the first step of inputting the electronic document to be built into the database,

A second step of dividing the electronic document into word units to obtain location information;

A third step of obtaining frequency information of a specific word in the document separated by the word unit,

A fourth step of analyzing the morphemes constituting the word to obtain analysis information, and

And a fifth step of constructing a database using the location information, frequency information, and analysis information.

In addition, the present invention, in a computer,

When the electronic document to be built into the database is input, the first step of dividing the electronic document in word units to obtain location information;

A second step of converting the location information of the word into a simple numeric pointer and storing the converted location information;

A third step of obtaining frequency information of a specific word in the document separated by the word unit,

A fourth step of obtaining analysis information by analyzing the morphemes constituting the word,

A fifth step of eliminating the neutrality of the morphological analysis information of the word using the word position information and the frequency information; and

A computer-readable recording medium having recorded thereon a program for executing the sixth step of building a database using the location information, frequency information and analysis information is provided.

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings.

The present invention is a part of the language information warehouse for the Korean information processing, the separation device, analysis device, encoder, storage, management device for the construction of large-scale information is a technology with the Korean characteristics.

1 is a block diagram illustrating an automatic word database construction apparatus of an electronic document according to an embodiment of the present invention.

Referring to FIG. 1, the sentence separator 30a assigns an automatic document number from a large-scale electronicized document, and divides the document into sentence units and assigns a unique number.

The word separator 30b receives a sentence separated by the sentence separator 30a through an input means, separates a word based on a blank, and assigns a unique number of a word unit.

The location information management device 30 is a device for managing the unique number of a word given through the sentence separator 30a and the word separator 30b, and manages the unique number so that the location information of the word preserves uniqueness. do. That is, since the location information where a word appears is the same as the frequency of the word, it is stored in a separate table and assigned a unique number to access this table.

The location encoder 20 converts the unique number determined by the location information management device 30 into a special number through an encoding technique to store the location information indicated by the word in the document.

The frequency assigning device 40 is an index indicating how many words appear in a large-scale electronicized document, and is a device for giving a frequency of occurrence of word units stored by the word managing device 50. Rather than assigning frequency of simple words, the information on left and right words is also kept to reflect the characteristics of left and right words in order to give analysis information of later words.

The analysis information management device 60 uses the Korean morpheme electronic dictionary 60e and the word composition rule 60d, the morpheme information analysis device 60a and the tagger information provision device 60b, and the morpheme dictionary search device 60c. It is a device that separates the word received from the word separator device 30b into morpheme units and gives tag information.

The Korean morpheme electronic dictionary 60e registers morphemes in headword units, and stores them in a modified tree structure format as electronic dictionaries with parts-of-speech information and feature information of the headwords. This is a method for speeding up compression and retrieval speed of a dictionary by storing using syllable information characteristics of Korean.

The word composition rule 60d describes a composition rule between word stems that can form a word by a set of tags.

The morpheme information analyzing apparatus 60d is a device that gives parts of speech to morphemes using the Korean morpheme electronic dictionary 60e and the word composition rule 60d.

The tagger information assigning device 60b is a word position and frequency provided by the word frequency information management device 40 and the location information management device 30 to solve the neutrality of the morpheme information given by the morphological analysis device 60d. It is a device that determines the exact result value using information.

The information management apparatus 10 is an apparatus which builds up each information provided through the encoder 20, the analysis information management apparatus 60, and the frequency information management apparatus 40 in the database of the record form of a word unit. Since the words extracted from the large electronic documents and the size of the built database are large, they are stored using Korean syllable characteristics. The primary key used to search a stored database is word.

The present invention configured as described above is a method for constructing a large amount of Korean information processing language information warehouse, and a large word database construction can be widely used for analyzing Korean information and searching for information.

2 is a flowchart illustrating a method of automatically constructing a word database of an electronic document according to an embodiment of the present invention. The database building operation continues until a building end key is input (step S21). When the large-scale electronicized document 30c is input, the position information management device 30 first constructs data on the position information that needs to be constructed (step S23). That is, the electronic document 30c is separated into sentence units using the sentence separator 30a and the word separator 30b (step S24), and each sentence is separated into word units (step S25).

Next, the frequency information management device 40 calculates the frequency of appearance in units of words (step S26). That is, the frequency of appearance of the word is calculated through the word management apparatus 50 (step S27). Next, the separated word and the left and right words centered on the word are provided to the analysis information management device 60 (step S28). The analysis information management device includes the word composition rule 60d and the Korean morpheme electronic dictionary 60e. ), The morpheme unit is analyzed through the morpheme information analyzing apparatus 60a, the tagger information providing apparatus 60b, and the morpheme dictionary search apparatus 60c (steps S29 and S30).

Finally, the information management apparatus 10 focuses on the morphological analysis information obtained by the analysis information management apparatus 60, the position information obtained by the location information management apparatus 30, and the frequency information obtained by the frequency information management apparatus 40. A tabular database is constructed (step S22). This database can be built in real time regardless of the size of the document, and new documents that are added can be added to the existing database.

3 is an exemplary diagram illustrating data construction of a language information database constructed by a method and apparatus for automatically constructing a word database of electronic documents according to the present invention.

The information stored here is composed of words, location information, frequency information, and analysis information. The categories are divided into the humanities, social sciences, natural sciences, engineering, arts, and other fields. Where humanities include philosophy, psychology, mechanics, religion, literature, history / geography / greatness, language / language, and social sciences, sociology, folklore, economics, management, politics, administration, defense, law, pedagogy, It includes the broadcast field. In addition, natural science includes mathematics, statistics, physics, chemistry, astronomy, earth science, biology, fauna and flora, oceanography, medicine, pharmacy, and home sciences. Engineering includes engineering, mechanical, electrical, electronic, chemical, civil, Includes architectural engineering, environmental science, urban engineering, agriculture, agriculture, and computer science. Art includes general arts, including sculpture and crafts, art history and sports, leisure, tourism and entertainment.

Here, the location detail information according to the location information identifier is represented by 13 digits, and the meaning thereof is as follows.

That is, as an example of the location detail information, when analyzing '98F0110001010', the first two digits '98' denotes the year in which the document is electronicized, and the next letter 'F' indicates a sector classification. Here, the field division is represented by the humanities (A), social sciences (B), natural sciences (C), engineering (D), art (E), other (F) and the like.

Next, the five-digit number '01, 100 'is the file name, the first two digits are the internal divisions of the field, and the remaining three digits are the file division elements. The next three digits '010' are the sentence position, and the next two digits '10' are the word position.

The meaning of the analysis information is divided into morphological units, and standard tag information is written horizontally. The meaning of the symbol used is shown in Table 1.

As described above, according to the present invention, the electronic document is stored in a sentence unit and a word unit, and the location information of the original text and the analysis information about the word are stored as unique information, thereby utilizing the information previously analyzed in the preceding document. It is effective to build a language information warehouse that enables complete analysis of Korean language.

Claims (8)

A location information management device for storing location information of sentences and words separated from the electronic document; A frequency information management device for storing the appearance frequency of the specific word extracted by the location information management device and the left and right word information of the specific word, An analysis information management device that separates words extracted by the location information management device into morpheme units and gives tag information; and And an information management device for constructing and managing the word location information, frequency information, and analysis information in a database format. The word database automatic construction device of electronic document according to claim 1, further comprising an encoder for converting the location information of the word output by the location information management device into a special number through an encoding technique. . The method according to claim 1 or 2, A sentence separator for assigning an automatic document number to an electronic document and dividing the document into sentence units to give a unique number in sentence units; The word of the electronic document characterized in that it further comprises a word separation device that receives the separated sentences from the sentence separation device to separate the word centered around the blank, give a unique number of the word unit and provide it to the location information management device. Database auto build device. The method according to claim 1 or 2, A Korean morpheme electronic dictionary that registers morphemes by heading unit and records parts of speech information and feature information of the headings, A word composition rule, in which information on the composition rule between word stems is recorded, A morpheme information analyzing apparatus for assigning parts of speech to morphemes using the Korean morpheme electronic dictionary and word composition rules; and Automatically constructing word database of electronic documents, characterized in that it further comprises a tagger information assigning device for determining a result value by using the location information and frequency information of words provided by the location information management device and the frequency information management device. . A first step of inputting an electronic document to be built into a database; A second step of dividing the electronic document into word units to obtain location information; A third step of obtaining frequency information of a specific word in the document separated by the word unit, A fourth step of analyzing the morphemes constituting the word to obtain analysis information, and And a fifth step of constructing a database using the location information, frequency information, and analysis information. The method of claim 5, further comprising a sixth step of converting and storing the position information of the word obtained in the second step into a simple numeric pointer. The method of claim 5, further comprising a seventh step of resolving the neutrality of the morphological analysis information of the word by using the position information and the frequency information of the word. On your computer, When the electronic document to be built into the database is input, the first step of dividing the electronic document in word units to obtain location information; A second step of converting the location information of the word into a simple numeric pointer and storing the converted location information; A third step of obtaining frequency information of a specific word in the document separated by the word unit, A fourth step of obtaining analysis information by analyzing the morphemes constituting the word, A fifth step of eliminating the neutrality of the morphological analysis information of the word using the word position information and the frequency information; and A computer-readable recording medium having recorded thereon a program for executing a sixth step of constructing a database using the location information, frequency information, and analysis information.