JPH05128159A - Key word extraction and its device - Google Patents

Abstract

PURPOSE:To improve accuracy of key word extraction by identifying the kind of document information whose content character string is divided into plural logical elements in advance, detecting the logical element conforming to prestored logical element for each kind and extracting a key word from among content character string in the detected logical element. CONSTITUTION:A key word extracting means 3 is a part of a document retrieving unit 2. The document retrieving unit 2 is connected to a data base, etc., which are not shown in the Figure. Document information has a structure in which a content character string is divided into plural logical elements in advance. A document content identifying means 5 divides the document information into logical element and content character string by analyzing with a context free grammar, etc., for detecting correspondence relation. The key word detecting means 3 sequentially reads the logical element stored in a specific logical element name table 4 and then detects coincidence with the logical content separated by the means 5 for extracting a key word out of the content character string in the detected logical element.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a keyword extracting method and apparatus used in a document filing system or the like.

[0002]

2. Description of the Related Art In a conventional document search apparatus, an operator selects a keyword that is considered appropriate at the time of registration of document information and classifies the keyword according to a thesaurus. However, the thesaurus is updated when such a keyword is added or deleted. The burden on the operator is increasing because the work is complicated.

Therefore, in order to solve such a problem,
A keyword extraction device has been developed which extracts a keyword from a content character string of document information by mechanical information processing such as morphological analysis. Then, such a keyword extracting device has, for example, a structure provided with keyword extracting means for extracting a keyword from the content character string of the input document information, and the keyword is extracted from the document information formed as a text file. It is designed to be extracted. By doing this, for example, by registering the correspondence between the extracted keyword and the document information in the transposed file of the document retrieval device, the predetermined document information is later retrieved from the keyword in the transposed file. be able to.

Here, as such a keyword extraction method, it has been proposed that the type of document information from which a keyword is extracted is not assumed in advance. For example, the type of document information is not assumed in advance. In the keyword extracting method, the keyword can be extracted if the document information is formed in the text file format. On the other hand, assuming the type of document information in advance, for example, Haruo Kimoto “Automatic keyword extraction using language processing”
The keyword extraction method proposed at the 1st National Conference of the Japanese Society for Artificial Intelligence (1987) assumes that document information is a newspaper article and extracts keywords depending on the characteristics of the document structure.

[0005]

As described above, according to the keyword extraction method which does not assume the type of document information in advance, the keyword can be extracted from the document information in the text file format. It is difficult to improve. Further, in the keyword extraction method that assumes the type of document information in advance, the keyword extraction accuracy is good, but this is less versatile because the document information from which the keyword is extracted is limited to a specific type. ..

[0006]

The invention according to claim 1 is
In the keyword extracting method in which the keyword extracting means extracts the keyword from the input document information,
The type identifying means identifies the type of document information in which the content character string is divided into a plurality of logical elements in advance, and a logical element that matches the logical element stored in advance in the element storage means for each identified type of document information. Is detected from the document information by the element detecting means, and the keyword extracting means extracts the keyword from the content character string in the detected logical element.

According to a second aspect of the invention, in the keyword extracting method in which the keyword extracting means extracts the keyword from the input document information, the type of the document information in which the content character string is divided into a plurality of logical elements in advance. Is identified by the type identifying means, and the element detecting means detects, from the document information, a logical element that does not match the logical element stored in advance in the element storing means for each type of the identified document information, and the detected logical element The keyword extracting means extracts the keyword from the content character string in the.

According to the third aspect of the invention, the importance degree calculating means calculates the importance degree of the keyword extracted from the content character string of the document information by the keyword extracting means, and the document information in the logical element detected by the element detecting means is used. The importance updating means updates the importance of the extracted keywords.

According to a fourth aspect of the present invention, in a keyword extracting device provided with keyword extracting means for extracting a keyword from a content character string of input document information, a document in which the content character string is divided into a plurality of logical elements in advance. A type identification means for identifying the type of information is provided, and element storage means for storing a specific logical element in advance for each type of document information identified by the type identification means is provided, and a logical element stored in the element storage means is provided. Element detecting means for detecting the matching logical element from the document information identified by the type identifying means, and keyword extracting means for extracting a keyword from the content character string in the logical element detected by the element detecting means are provided.

According to a fifth aspect of the present invention, in a keyword extracting device provided with keyword extracting means for extracting a keyword from a content character string of input document information, a document in which the content character string is divided into a plurality of logical elements in advance. A type identification means for identifying the type of information is provided, and element storage means for storing a specific logical element in advance for each type of document information identified by the type identification means is provided, and a logical element stored in the element storage means is provided. Element detecting means is provided for detecting a non-matching logical element from the document information identified by the type identifying means, and keyword extracting means is provided for extracting a keyword from the content character string in the logical element detected by the element detecting means.

According to a sixth aspect of the present invention, there is provided importance calculating means for calculating the importance of the keyword extracted from the content character string of the document information by the keyword extracting means, and the document information in the logical element detected by the element detecting means. An importance updating means for updating the importance of the keywords extracted from is provided.

[0012]

According to the inventions described in claims 1 and 4, it is possible to extract a keyword from a content character string which is logically structurally important in correspondence with the type of document information, so that the keyword extraction accuracy is improved. Since a plurality of types can be set as the document information for extracting such a keyword, the versatility of the document information for which the keyword is extracted can be improved.

According to the second and fifth aspects of the present invention, the keyword can be extracted from the content character string other than the logically unimportant portion corresponding to the type of document information. Since a plurality of types can be set as the document information for extracting such a keyword, the versatility of the document information from which the keyword is extracted can be improved.

According to the third and sixth aspects of the present invention, the importance of the keyword can be corrected in correspondence with the importance of the logical structure of the document information. Therefore, the document information can be stored based on the importance of the keyword. It is possible to improve the search accuracy when performing a search.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the invention described in claims 1 and 4 will be described with reference to the drawings. First, as illustrated in the block diagram of FIG. 2, the keyword extracting device 1 illustrated here is formed as a part of the document searching device 2, and the document searching device 2 stores a database ( (Not shown) or the like. Then, the document information from which the keyword extracting device 1 extracts the keyword has a structure in which the content character string is divided into a plurality of logical elements in advance as illustrated in FIG. Logical elements indicating start and end are <title> and </ titl>
It is represented by e> etc., and sandwiches the content character string. further,
In the document information formed in this way, logical elements are hierarchically set, and here the document type, which is the type of document information, is expressed by the name in <report>, which is the highest logical element. There is. Note that, as described above, a concrete means for giving a logical element to the content character string of the document information is, for example, ISO 8879 Information processing-Text and office syst.
ems-Standard Generalized Markup Language
(SGML) and the like.

Therefore, in the document retrieval apparatus 2 exemplified in this embodiment, the keyword extraction means 3 is connected to the specific logical element name table 4, the document content identification means 5, the type identification means (not shown), etc. The extraction device 1 has a structure in which the transposed file updating means 6 and the like are connected.

More specifically, the document content identification means 5
Is configured to separate the document information into logical elements and content character strings by a context-free grammar analysis utilizing the form of the logical elements illustrated in FIG. Identifies the document type, which is the identification of document information, from the name of the highest logical element. Further, as illustrated in FIG. 5, the specific logical element name table 4, which is an element storage unit, stores in advance a specific logical element with a name for each document type which is a type of document information. Note that such a specific logical element name table 4 may be created in advance, and it is not necessary to create it every time the document information is registered.

The keyword extracting means 3, which is also an element detecting means, sequentially reads out the specific logical elements stored in the specific logical element name table 4, and the document content identifying means 5 separates the logical content of the document information. Is detected, and the keyword is extracted from the content character string of the document information in the logical element thus detected. Note that the keyword extracting operation by the keyword extracting unit 3 is easily executed by a known technique such as detecting a noun by morphological analysis.

Further, the transposed file updating means 6 connected to the keyword extracting device 1 having the above-mentioned structure,
The storage information of the transposed file (not shown) connected to this is updated with the extracted keyword.The storage information of this transposed file is an index that can uniquely detect the document information corresponding to the keyword. Etc.

With such a configuration, the processing operation of the keyword extracting device 1 will be described in detail below together with the processing operation of the document searching device 2 based on the flowchart illustrated in FIG. First, when the document information having the above-described logical structure is input to the keyword extracting device 1 of the document searching device 2, the type identifying unit detects the highest logical element and reads the name to determine the document type. Since they are identified, the keyword extracting means 3 sequentially reads the corresponding logical elements by name from the specific logical element name table 4 according to the identified document type. On the other hand, since the document content identification means 5 extracts a logical element from the document information, the keyword extraction means 3 matches the logical element read from the specific logical element name table 4 with the logical element extracted by the document content identification means 5. It will be detected, and this will extract a keyword from the content character string in the detected logic element.

At this time, actually, the keyword extracting unit 3 selects a keyword candidate in advance by extracting a noun by morphological analysis from all the content character strings separated from the document information by the document content identifying unit 5, and the specific logical element Only the keyword candidates in the logical element having the same name as the logical element name read from the name table 4 are selected as keywords.

In the document searching device 2, the transposed file updating means 6 updates the storage information of the transposed file with the keyword extracted by the keyword extracting device 1 as described above.

By doing so, the keyword extracting device 1 of the document searching device 2 can logically define the portion for extracting the keyword corresponding to the type of the document information, so that the keyword extraction accuracy can be improved. Is very good, and a plurality of types can be set as the document information for extracting such a keyword, so that versatility can be improved. Since the transposed file is updated with the keyword extracted in this way, the document retrieval apparatus 2 has extremely good retrieval accuracy of document information.

In this embodiment, by extracting a keyword from the logical element stored in the specific logical element name table 4 as an embodiment of the invention described in claims 1 and 4, the optimum portion for extracting the keyword is defined. However, the keyword extraction is performed by extracting keywords from logical elements not stored in the specific logical element name table 4 as in the inventions according to claims 2 and 5. It is also possible to prevent the extraction of unnecessary keywords by excluding the portions that are not suitable for.

Further, as in the invention according to claims 3 and 6, the importance degree calculating means for calculating the importance degree of the keyword extracted from the content character string of the document information by the keyword extracting means.
It is also possible to provide (not shown) and provide importance updating means (not shown) for updating the importance of the keyword extracted from the document information in the logical element detected by the element detecting means. In this case, the calculation of the importance degree of the keyword by the importance degree calculation means is performed by a mathematical expression such as (the number of the keywords) / (the number of all the extracted keywords), and the importance degree calculation means calculates the importance degree of the keyword. The update is realized by, for example, multiplying a preset constant. Note that such updating of the importance level is performed by, for example, increasing the importance level of the keyword extracted from the logical element stored in the specific logical element name table 4 so that the importance level of the keyword appropriate for searching the document information is increased. By reducing the importance of keywords extracted from logical elements that are not stored in the specific logical element name table 4 to reduce the importance of keywords unsuitable for searching document information. This is possible, and by doing so, it is possible to improve the search accuracy when searching the document information based on the importance of the keyword.

[0026]

According to the invention described in claim 1, in the keyword extracting method in which the keyword extracting means extracts the keyword from the input document information, the document information in which the content character string is divided into a plurality of logical elements in advance. The type identification unit identifies the type of the document information, and the element detection unit detects, from the document information, a logical element that matches the logical element stored in advance in the element storage unit for each type of the identified document information. Since the keyword extracting means extracts the keyword from the content character string in the logical element, it is possible to extract the keyword from the content character string which is logically important in correspondence with the type of document information. Therefore, it is possible to improve the extraction accuracy of keywords, and it is possible to set multiple types of document information for extracting such keywords. Since kill those having effects such can improve the versatility of the document information as a keyword to be extracted.

According to a second aspect of the invention, in the keyword extracting method in which the keyword extracting means extracts the keyword from the input document information, the type of the document information in which the content character string is divided into a plurality of logical elements in advance. Is identified by the type identifying means, and the element detecting means detects, from the document information, a logical element that does not match the logical element stored in advance in the element storing means for each type of the identified document information, and the detected logical element Since the keyword extracting means extracts the keyword from the content character string in the inside, it is possible to extract the keyword from the content character string other than the portion that is not logically important in correspondence with the type of the document information. Since it is possible to improve the keyword extraction accuracy, multiple types can be set as document information for extracting such keywords. It is possible keep, those having the effect of such can improve the versatility of the document information as a keyword to be extracted.

According to a third aspect of the invention, the importance degree calculating means calculates the importance degree of the keyword extracted from the content character string of the document information by the keyword extracting means, and the document information in the logical element detected by the element detecting means is used. Since the importance updating means updates the importance of the extracted keyword, the importance of the keyword can be corrected according to the logical structural importance of the document information. This has the effect of improving the search accuracy when searching for document information based on importance.

According to a fourth aspect of the present invention, in a keyword extracting device provided with keyword extracting means for extracting a keyword from a content character string of input document information, a document in which the content character string is divided into a plurality of logical elements in advance. A type identification means for identifying the type of information is provided, and element storage means for storing a specific logical element in advance for each type of document information identified by the type identification means is provided, and a logical element stored in the element storage means is provided. By providing the element detecting means for detecting the matching logical element from the document information identified by the type identifying means, and by providing the keyword extracting means for extracting the keyword from the content character string in the logical element detected by the element detecting means. , It is possible to extract the keyword from the content string that is logically important in correspondence with the type of document information. It is possible to improve the extraction accuracy,
Since a plurality of types can be set as the document information for extracting such a keyword, the versatility of the document information from which the keyword is extracted can be improved.

According to a fifth aspect of the present invention, in a keyword extracting device provided with keyword extracting means for extracting a keyword from a content character string of input document information, a document in which the content character string is divided into a plurality of logical elements in advance. A type identification means for identifying the type of information is provided, and element storage means for storing a specific logical element in advance for each type of document information identified by the type identification means is provided, and a logical element stored in the element storage means is provided. By providing the element detecting means for detecting the unmatched logical element from the document information identified by the type identifying means, and by providing the keyword extracting means for extracting the keyword from the content character string in the logical element detected by the element detecting means. It is possible to extract a keyword from a content character string other than a part that is not logically important in correspondence with the type of document information. With this, it is possible to improve the keyword extraction accuracy, and it is possible to set a plurality of types as the document information for extracting such a keyword, so that the versatility of the document information from which the keyword is extracted is improved. It is possible to obtain such effects.

According to a sixth aspect of the present invention, the keyword extracting means is provided with importance calculating means for calculating the importance of the keyword extracted from the content character string of the document information, and the document information in the logical element detected by the element detecting means. Since the importance updating means for updating the importance of the keyword extracted from is provided, it is possible to correct the importance of the keyword in correspondence with the logical structural importance of the document information. This has the effect of improving the search accuracy when searching for document information based on importance.

[Brief description of drawings]

FIG. 1 is a flow chart showing an embodiment of the present invention.

FIG. 2 is a block diagram showing a document search device including a keyword extraction device as a part.

FIG. 3 is a conceptual explanatory diagram showing a logical structure of document information.

FIG. 4 is a conceptual explanatory diagram showing a logical structure of context-free grammar.

FIG. 5 is a conceptual explanatory diagram showing the structure of a specific logical element name table.

[Explanation of symbols]

 1 keyword extracting device 3 keyword extracting means and element detecting means 4 element storing means

Claims (6)

[Claims] 1. In a keyword extracting method in which a keyword extracting unit extracts a keyword from input document information, a type identifying unit identifies a type of document information in which a content character string is divided into a plurality of logical elements in advance. Then, the element detecting means detects, from the document information, a logical element that matches the logical element stored in advance in the element storing means for each type of the identified document information, and from the content character string in the detected logical element. A keyword extracting method, wherein the keyword extracting means extracts a keyword. 2. In a keyword extracting method in which a keyword extracting unit extracts a keyword from input document information, a type identifying unit identifies the type of document information in which a content character string is divided into a plurality of logical elements in advance. Then, the element detecting unit detects, from the document information, a logical element that does not match the logical element stored in advance in the element storing unit for each type of the identified document information, and from the content character string in the detected logical element. A keyword extracting method, wherein the keyword extracting means extracts a keyword. 3. The importance of the keyword extracted from the document information in the logical element detected by the element detecting means by the importance calculating means calculating the importance of the keyword extracted from the content character string of the document information by the keyword extracting means. The degree updating means updates the degree by the importance degree updating means.
The keyword extraction method described. 4. A keyword extracting device provided with a keyword extracting means for extracting a keyword from a content character string of input document information, wherein the type of document information in which the content character string is divided into a plurality of logical elements in advance is identified. A type identification unit is provided, and an element storage unit that stores a specific logical element in advance for each type of document information identified by the type identification unit is provided, and a logical element that matches the logical element stored in the element storage unit is described above. A keyword extracting device provided with an element detecting means for detecting from the document information identified by the type identifying means, and a keyword extracting means for extracting a keyword from a content character string in a logical element detected by the element detecting means. .. 5. In a keyword extracting device provided with a keyword extracting means for extracting a keyword from a content character string of input document information, the type of document information in which the content character string is divided into a plurality of logical elements in advance is identified. A type identification unit is provided, and an element storage unit that stores a specific logical element in advance for each type of document information identified by the type identification unit is provided, and a logical element that does not match the logical element stored in the element storage unit is described above. A keyword extracting device provided with an element detecting means for detecting from the document information identified by the type identifying means, and a keyword extracting means for extracting a keyword from a content character string in a logical element detected by the element detecting means. .. 6. An importance degree calculating means for calculating the importance degree of the keyword extracted from the content character string of the document information by the keyword extracting means is provided, and the keyword extracting means extracts the keyword extracted from the document information in the logical element detected by the element detecting means. 6. The keyword extracting device according to claim 4, further comprising an importance updating means for updating the importance.