JP2003108569A - Classifying processing unit, control method of classifying processing unit, control program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To execute optimal classification by extracting proper classification ontology from a sentence of a classification object. SOLUTION: A database part 11 prestores a word or a compound word included in a prescribed keyword original document as the classification ontology in relation to a specific field. A classification renewing processing part 12 analyzes text data corresponding to the document of the classification object, extracts the word or the compound word included in the document of the classification object as classification object words and phrases, compares the classification ontology and the classification object words and phrases, and discriminates classification to which the document of the classification object belongs.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a classification processing device, a classification processing device control method, a control program, and a recording medium, and more particularly to a technique for classifying a document by determining the classification to which a target document belongs.

[0002]

2. Description of the Related Art With the development of information technology in recent years, a vast amount of information has become available, and various documents are no exception. However, there is a contradictory relationship between the amount of information that can be used and the ease of use, and as the amount of information, that is, the amount of documents increases, the problem that the documents cannot be used more easily arises. In order to avoid the above problems, it has been proposed to classify the documents to be used and use the documents that belong to the necessary classification.When classifying the documents, extract the keywords included in the documents. Generally, it is classified based on the extracted keywords.

In such a case, as an example of a technique for extracting a keyword from a document, Japanese Patent Laid-Open No. 6-2825 is available.
The keyword automatic extraction device described in No. 72 is mentioned.
The automatic keyword extraction device described in Japanese Patent Application Laid-Open No. 6-282572 extracts a part-of-speech information by performing morphological analysis on a document, and extracts a noun phrase and a sahen noun from the document. Then, the importance of the extracted noun phrase and sahen noun in the document is determined, and the keyword of high importance is automatically extracted as the keyword corresponding to the classification of the document.

[0004]

By the way, in the above-mentioned conventional keyword automatic extraction device, the extracted keyword includes, for example, a compound word in which a plurality of noun phrases are combined. In this case, the compound word and the words forming the compound word are also to be extracted as keywords. However, when classifying documents, the compound words and the words that make up the compound words do not necessarily belong to the same field, and the classification of documents is performed based on both the compound words and the words that make up the compound words. When you do
There was a problem that the classification would be different from the original classification.

For example, consider a case where a sentence to be classified contains a compound word "music CD". In this case, in the conventional keyword extraction device, three of "music", "CD" and "music CD" are extracted as keywords. By the way, the word "CD"
Generally means "CD-ROM" which is a recording medium, and the category to which the word "CD" belongs also includes "computer field". Therefore, it can be seen that the word “CD” is not suitable as a keyword for discriminating the classification (hereinafter referred to as the classification ontology). Therefore, an object of the present invention is to extract an appropriate classification ontology from a sentence to be classified, and to perform optimum classification, a classification processing device control method, a classification processing device control program, and It is to provide a recording medium having a control program recorded therein.

[0006]

In order to solve the above-mentioned problems, the classification processing device stores a word or a compound word included in a predetermined classification standard document in association with a specific field in advance as a classification standard phrase. Section, a document to be classified, and a phrase extraction unit that extracts a word or a compound word included in the document to be classified as a classification target phrase, compares the classification criterion phrase and the classification target phrase, and classifies And a classification determination unit that determines the classification to which the target document belongs. According to the above configuration, the classification database stores words or compound words included in a predetermined classification standard document in advance as classification standard phrases in association with a specific field. The word / phrase extraction unit analyzes a document to be classified and extracts a word or a compound word included in the document to be classified as a classification target phrase. The classification determination unit compares the classification reference phrase and the classification target phrase to determine the classification to which the document to be classified belongs.

In this case, a morphological analysis unit for performing morphological analysis of the classification standard document to extract the word or the compound word, and calculating the importance of the extracted word or compound word in the classification standard document. And a reference word registration unit that registers the extracted word or the compound word in the classification database unit as the classification reference word in association with the importance and the specific field. You may

Further, the classification discriminating section may discriminate preferentially as a classification to which the document to be classified belongs, a classification in which the number of words included in the classification reference word is large among the classification objects. . Furthermore, when the classification target phrase includes N words and the compound word, the classification determination unit represents the classification target phrase by a vector in an N-dimensional vector space, and the classification reference phrase is the vector. It may be represented by a vector in space, and the determination may be performed based on the distance between the two vectors.

The classification discriminating unit represents the importance of the classification target phrase by a vector in an N-dimensional vector space when the classification target phrase includes N words and the compound word. The importance of the classification criterion phrase may be represented by a vector in the vector space, and the determination may be performed based on the distance between the two vectors.

Further, the classification discriminating unit sets the vector X corresponding to the importance of the classification target phrase as X = (X1, X2, ..., XN), and corresponds to the importance of the classification reference phrase. The vector Y is represented by Y = (Y1, Y2, ..., YN), and the distance D is D = Σ (Xi-Yi) * (Xi-Yi) i = 1,
2, ..., N, if the distance D is smaller than a predetermined threshold value, it is determined that the classification target document belongs to a classification close to the classification to which the classification reference document belongs. Good.

Furthermore, the morpheme analysis unit is a combination of a plurality of words of the compound word, a combination of a word and a compound word having a smaller number of characters than the extracted compound word,
Alternatively, only one of the compound words having a smaller number of characters than the extracted compound word may be extracted. Further, the morphological analysis unit may extract, in the morphological analysis, words belonging to a noun including at least a noun phrase and a sahen noun and a part of speech that can be regarded as a predetermined noun phrase as a classification target phrase to be extracted.

Furthermore, the part of speech that can be regarded as the predetermined noun phrase may include a noun form of an adjective verb and a continuous form of a one-stage verb. A morphological analysis reverse lookup dictionary for registering the classification target words may be provided, and the morphological analysis unit may perform the morphological analysis based on the morphological analysis reverse lookup dictionary. Further, the morphological analysis unit may register a word or a compound not registered in the morphological analysis reverse lookup dictionary as an indefinite term in the morphological analysis reverse lookup dictionary. Furthermore, the morphological analysis unit, in the morphological analysis, when the extracted word or compound word includes a predetermined symbol,
The extracted word or compound may be obtained after removing the symbol from the word or compound.

Further, the importance degree calculation unit may perform the importance degree calculation by excluding words that are inappropriate as a predetermined classification target phrase from the extracted words or the compound words. Further, the importance degree calculation unit may perform the importance degree calculation by excluding a word or phrase not suitable for discriminating the predetermined classification from the extracted word or the compound word. Furthermore, the extracted word or the compound word is provided with a standardization unit that performs a predetermined standardization process, and the importance calculation unit determines the importance level of the word or the compound word after the standardization process. It may be calculated.

Further, the control method of the classification processing apparatus provided with the classification database unit for preliminarily storing a word or a compound word included in a predetermined classification standard document in association with a specific field as a classification standard phrase is a document to be classified. And a word extraction process of extracting a word or a compound word included in the document to be classified as a classification target phrase, comparing the classification reference phrase and the classification target phrase, and determining the classification to which the classification target document belongs. It is characterized by having a classification discrimination process for discriminating. According to the above configuration, in the phrase extraction process, the document to be classified is analyzed, and the word or compound word included in the document to be classified is extracted as the classification target phrase. In the classification determination process, the classification reference phrase and the classification target phrase are compared to determine the classification to which the document to be classified belongs.

In this case, a morphological analysis process of performing morphological analysis of the classification standard document to extract the word or the compound word, and calculating the importance of the extracted word or compound word in the classification standard document. And a reference word registration step of registering the extracted word or the compound word in the classification database unit as the classification reference word in association with the importance and the specific field. You may

Further, in the classification determination process, when the classification target phrase includes N words and the compound word,
The importance of the classification target phrase is represented by a vector in an N-dimensional vector space, the importance of the classification criterion phrase is represented by a vector in the vector space, and the determination is performed based on the distance between the two vectors. Good. Further, in the classification determination process, a vector X corresponding to the importance of the classification target phrase is set to X = (X1, X2, ..., XN), and a vector Y corresponding to the importance of the classification criterion phrase is set. , Y = (Y1, Y2, ..., YN), and the distance D is D = Σ (Xi−Yi) * (Xi−Yi) i = 1,
2, ..., N, if the distance D is smaller than a predetermined threshold value, it is determined that the classification target document belongs to a classification close to the classification to which the classification reference document belongs. Good.

Furthermore, in the morphological analysis step, the compound word is a combination of a plurality of words, a combination of a word and a compound word having a smaller number of characters than the extracted compound word, or a combination of the extracted compound word. In the case of any combination of compound words having a small number of characters, only the compound word may be extracted. Further, in the morpheme analysis process, the morpheme analysis may be performed based on the registered content of the reverse morphological analysis dictionary for registering the classification target phrase. Further, in the morphological analysis step, a word or a compound not registered in the morphological analysis reverse lookup dictionary may be registered as an indefinite word in the morphological analysis reverse lookup dictionary.

Further, in the morphological analysis step, in the morphological analysis, when the extracted word or compound word includes a predetermined symbol, the extraction is performed after removing the symbol from the word or compound word. It may be a word or a compound word. Further, in the importance calculation step, the importance may be calculated by excluding words that are inappropriate as a predetermined classification target word from the extracted words or the compound words. Furthermore, the extracted word or the compound word is provided with a standardization process for performing a predetermined standardization process, and the importance degree calculation process calculates the importance level for the word or the compound word after the standardization process. It may be calculated.

Further, in a control program for causing a computer to function as a classification processing device using a classification database unit that stores a word or a compound word included in a predetermined classification standard document in advance as a classification standard phrase in association with a specific field, A document to be classified is analyzed, a word or a compound word included in the document to be classified is extracted as a classification target phrase, the classification criterion phrase and the classification target phrase are compared, and the classification to which the classification target document belongs is determined. The feature is that they are distinguished. In this case, the word or the compound word is extracted by performing a morphological analysis of the classification criterion document, the importance of the extracted word or the compound word in the classification criterion document is calculated, and the extracted word is extracted. The word or the compound word may be registered in the classification database unit as the classification reference word in association with the importance and the specific field.

The classification may be discriminated based on the words included in the classification standard words and the classification standard words among the classification target words. Further, in the classification determination, a classification in which the number of phrases included in the classification reference phrase is large among the classification target phrases may be preferentially determined as a classification to which the document to be classified belongs. Furthermore, when the classification target phrase includes N words and the compound word, the classification target phrase is represented by a vector in an N-dimensional vector space, and the classification criterion phrase is represented by a vector in the vector space. Alternatively, the determination may be performed based on the distance between the two vectors.

When the classification target phrase includes N words and the compound word, the importance of the classification target phrase is represented by a vector in an N-dimensional vector space, and the importance of the classification criterion phrase is determined. The degree may be represented by a vector in the vector space, and the determination may be performed based on the distance between the two vectors. Further, a vector X corresponding to the importance of the classification target phrase is represented by X = (X1, X2, ..., XN), and a vector Y corresponding to the importance of the classification target phrase is expressed as Y = ( Y1, Y2, ..., YN) and the distance D is D = Σ (Xi-Yi) * (Xi-Yi) i = 1,
2, ..., N, if the distance D is smaller than a predetermined threshold value, the classification target document is determined to belong to a classification close to the classification to which the classification reference document belongs. Good.

Furthermore, the compound word is a combination of a plurality of words, a combination of a word and a compound word having a smaller number of characters than the extracted compound word, or a compound word having a smaller number of characters than the extracted compound word. In the case of any of the combinations, only the compound word may be extracted. Further, in the morphological analysis, a word belonging to a noun and a part of speech that can be regarded as a predetermined noun phrase may be extracted as a classification target phrase to be extracted. Further, the noun includes a noun phrase and a sahen noun. Furthermore, the part of speech that can be regarded as the predetermined noun phrase may include the noun form of the adjective verb and the continuous form of the one-stage verb. Further, the morphological analysis may be performed based on the registered contents of the reverse lookup dictionary for morphological analysis for registering the classification target phrase. Further, a word or a compound word not registered in the morphological analysis reverse lookup dictionary may be registered as an indefinite word in the morphological analysis reverse lookup dictionary.

Furthermore, in the morphological analysis, when the extracted word or compound word includes a predetermined symbol, the extracted word or compound word is removed after the symbol is removed from the word or compound word. You may allow it. Further, the importance calculation may be performed by excluding an unsuitable word as a predetermined classification target word from the extracted word or the compound word. Furthermore, a predetermined standardization process may be performed on the extracted word or the compound word, and the importance of the word or the compound word after the standardization process may be calculated. Further, each of the above control programs may be recorded in a recording medium.

[0024]

BEST MODE FOR CARRYING OUT THE INVENTION Next, preferred embodiments of the present invention will be described with reference to the drawings. [1] Schematic Configuration of Classification Processing System FIG. 1 shows a schematic configuration block diagram of the classification processing system.
The classification processing system 10 is roughly classified into a database unit 11 that accumulates various data as a database, a classification process based on each database accumulated in the database unit 11, and a database unit 11 based on the result of the classification process. Update processing unit 12 for updating each database of the above, and display unit 13 for displaying various information
And an input unit 14 for inputting various data. Here, the classification processing system 10 can be realized by a computer system, and the function of the classification update processing unit 12 is realized by a program executable by a microprocessor corresponding to each unit. Further, such a program may be directly executed from a recording medium such as a semiconductor memory or a CD-ROM. It is also possible to install the program in an external storage device in advance and execute it. Furthermore, the program may be installed each time it is executed prior to execution or only once at the beginning via a network such as the Internet.

The database unit 11 is roughly classified into a classification database unit 15 and a morphological analysis reverse lookup dictionary 16.
And a text database unit 17. Here, the database unit 11 is built in an external storage device such as a hard disk. Classification database unit 15
Stores a word or a compound word included in a keyword source document (classification reference document) as a classification ontology (classification reference phrase) in association with a predetermined specific classification (field). The morphological analysis reverse lookup dictionary 16 stores words or compound words (morphological analysis results) obtained by performing morphological analysis of text data as dictionary data used for morphological analysis. The text database unit 17 stores the results (words and compound words) of morphological analysis of text data corresponding to documents to be classified. The classification update processing unit 12 roughly includes a morpheme analysis unit 21, an importance degree calculation unit 22, a standardization unit 23, and a classification addition unit 24.

The morpheme analysis unit 21 performs a morpheme analysis of a text document to be classified or a keyword source document for extracting a keyword to generate a morpheme analysis result. If the morphological analysis target is a text document to be classified, the word or compound word that is the morphological analysis result is output to the text database unit 17. If the target of the morphological analysis is the keyword source document, the word or the compound word that is the morphological analysis result is output to the classification database unit 15 in order to generate the classification ontology.
The importance calculator 22 calculates the importance of a word or a compound word that is a morphological analysis result of a keyword source document. For example, the TFIDF value is calculated by the TFIDF method as the degree of importance in the keyword source document. Then, the importance level is output to the classification database unit 15 in association with the word or the compound word that is the morphological analysis result of the keyword source document.

The standardization unit 23 corrects the fluctuation of the notation of the word or the compound word which is the result of the morpheme analysis, and outputs the corrected word or the compound word to the morpheme analysis unit 21 as the morpheme analysis result. For example, "PC",
The notation of “personal computer” and “personal computer” is unified to “personal computer” by the standardization unit 23, and the result is output to the morpheme analysis unit 21 as a morpheme analysis result. Similarly, the notation of "George Washington" and "George-Washington" is unified to "George-Washington", and the result is output to the morphological analysis unit 21 as the morphological analysis result. Classification addition unit 24
Is a result of morphological analysis (words and compound words) of the text data corresponding to the document to be classified stored in the text database unit 17 and the classification database unit 15.
The classification of the document to be classified is discriminated by referring to the classification ontology stored in, and the classification discrimination result is added to the morphological analysis result (word and compound word) in the text database unit 17 and stored.

[2] Overall Processing Next, the operation of the classification processing apparatus according to the embodiment will be described with reference to FIGS. 2 to 7. FIG. 2 shows an overall processing flowchart of the classification processing device. The user is prompted to confirm on the display screen of the display 13 the morphological analysis dictionary (reverse lookup dictionary) 17 and the text data of the classification target document to be used for classification processing (step S1). Morphological analysis dictionary (reverse lookup dictionary) 17 to be used for classification processing by the user 17
When the text data of the classification target document is confirmed, the morphological analysis unit performs a morphological analysis process on the text data of the classification target document and registers the extracted words and compound words that are the morphological analysis results in the text database (step S2).

Here, the processing of the morphological analysis unit in the processing of step S2 will be described. FIG. 3 shows a processing flowchart of the morphological analysis unit. First, the user is prompted to confirm the number of unanalyzed documents and the morphological analysis dictionary on the display screen (step S11). Next, the morphological analysis unit determines whether or not there is an unanalyzed document (step S1).
2). If there is no unanalyzed document in the determination of step S12 (step S12; No), there is no need to perform morphological analysis, and the process ends. If there is an unanalyzed document in the determination of step S12 (step S12;
Yes), morphological analysis processing is performed (step S13).
In this morphological analysis processing, part-of-speech processing is performed to extract nouns (noun phrases, sahen nouns) and words and compound words that belong to parts of speech regarded as noun phrases. As the part of speech of the word and compound word to be extracted, first, as in the past, the adjective verb, Sahen noun, common noun, number, proper noun, adjunct, idiomatic phrase, idiomatic kanji (excluding symbols), rendaku (noun rendaku) , Continuous rendaku), indefinite terms. Here, the phrase is
A phrase such as "amen" or "condolences".
In addition, rendaku means that when two words are combined into one new phrase, the pure sound at the beginning of the word behind is changed to dakuon. For example, the "living" part in the phrase "country living", the "street" part in the phrase "desired street", and the like can be mentioned.
Furthermore, an indefinite term means a word or compound that is not included in the reverse dictionary for morphological analysis.

The newly added word to be extracted and the part of speech of the compound word include the noun form of the adjective verb and the continuous form of the one-stage verb. Here, the noun form of the adjective verb and the continuous form of the one-stage verb will be specifically described. For example, "Beautiful" in "Beautiful flowers" is a continuous form of adjective verbs and is not subject to extraction. However, when "Beautiful" is the noun form of adjective verbs such as "Flower is pretty", it is subject to extraction. And Also, "Aomuku" is not subject to extraction,
"Aomuke" is a combination of single-stage verbs, so it is targeted for extraction. On the contrary, it is a part of speech that was conventionally extracted,
In the present embodiment, the part-of-speech removed from the extraction target includes the final form of the Sahen noun and the adnominal. Here, the ending forms and adnominals of the Sahen nouns will be specifically described. For example, in “to behave together”, “behavior” is a sahenun and is an extraction target, but “behavior” is an end form of a sahenun and is not an extraction target. In addition, “Akirakuru” in “Akarikuru Asahi” and “Evil” in “Awkward Habit” are adjuncts and therefore are not subject to extraction.

Specifically, a morphological analysis dictionary (reverse lookup dictionary)
When a morpheme is extracted based on, and a word (indefinite word) not registered in the morpheme analysis dictionary (reverse lookup dictionary) is extracted, the indefinite word is output as a noun phrase. Examples of the word extracted as the indefinite word thus extracted include the model number of the product. In addition, regarding a compound word registered in the dictionary, a plurality of words forming the compound word are not analyzed. Next, the morphological analysis unit determines whether or not a noun (noun phrase, sahen noun) and a phrase according to the noun phrase have been extracted (step S14). In the determination of step S14, when the noun phrase and the word equivalent to the noun phrase are not extracted (step S14; No), the process proceeds to step S12, and the same process is performed thereafter. In the determination of step S14, when a noun phrase and a phrase according to the noun phrase are extracted, the morphological analysis unit 21 performs symbol processing.

In the symbol processing, when an unsuitable character is included at the beginning of a word such as a midpoint, the processing is not performed as a word / phrase extracted from a character string including the unsuitable character. This is because. In addition, even when the character string is different depending on the symbol, the words may be substantially the same, for example, “● HDD” and “HDD” although the cutout positions are different. In addition, it is an empty symbol which is inappropriate as a model number of a product or the like. Characters that are inappropriate as the beginning of this word include the following. In the following description, the code in <> is the corresponding shift JIS code. Half-width characters: ".", "'", "` ","! ","? ",
"-", "(),""Double-bytecharacters:"!<8149>"," ・ <8145>","?<8148
> ”,“ ○ <819b> ”,“ ● <819c> ”,“ ＊ <8196
> ”,“. <8144> ”,“ '<8165> ”to“ ”<8178>”,
“/ <815e>”, “\ <815f>”, “= <8181>” to “≧ <818
6> ”,“, <8141> ”to“, <8143> ”, and shift JI
Two characters corresponding to S code 8179, 817a

Therefore, the morphological analysis unit 21 determines whether or not the leading character of the extracted word or phrase is a symbol (step S15). In the determination in step S15, if the leading character is a symbol, the leading symbol may be used to represent a heading or the like. Therefore, in order to remove the leading symbol from the extracted phrase, the beginning of the extracted phrase is set to the next character excluding the symbol, and is regarded as the newly extracted phrase (step S16).

Next, the morpheme analysis unit 21 determines the word length of the newly extracted word and determines whether the word length is greater than 0, that is, the newly extracted word is constructed. It is determined whether a character exists (step S1)
7). In the determination in step S17, the word length is 0
If it is (step S17; No), it means that the word does not exist, and therefore the process proceeds to step S14 again. In the determination of step S17, when the word length is greater than 0 (step S17; Yes),
The morphological analysis unit 21 will make the determination again in step S15.

If the first character is not a symbol in the determination of step S15, the newly extracted word is considered to be a word to be registered in the text database 16, so the standardization unit 23 performs standardization processing. (Step S18). Then, the classification update processing unit 12 registers the standardized word in the text database 16 (step S19). Next, the classification update processing unit 12 registers the words registered in the text database 16 in the morphological analysis reverse lookup dictionary 17 (step S3). Next, the classification update processing unit 12 prompts the user to confirm the document from which the keyword is acquired and the classification on the screen of the display 13 (step S4). The morphological analysis unit again performs morphological analysis processing on the document from which the keyword was acquired (step S
5). The importance calculator 22 calculates the importance (step S6).

Here, the processing of the importance calculation section 22 in the processing of step S6 will be described. FIG. 4 shows a processing flowchart of the importance calculation section. First, the importance calculation unit 22 acquires the morphological analysis processing data of the document from which the keyword is acquired, obtained in step S5 (step S21). Next, the importance calculation section 22 determines whether or not there is an unprocessed document (step S22). Step S22
If there is no unprocessed keyword acquisition source document for which importance calculation has not been completed in the determination of (step S
22; No), since there is no need to calculate the importance, the process ends. If there is an unprocessed keyword acquisition source document in the determination of step S22 (step S22;
Yes), the importance calculation unit 22 calculates a TFIDF value for a word or a compound word included in the unprocessed keyword acquisition source document based on the morphological analysis processing data corresponding to the unprocessed keyword acquisition source document, A word or compound word having a TFIDF value equal to or greater than a certain threshold is extracted (step S23).

Next, the importance calculation section 22 performs a restriction process on the extracted words or compound words, and determines whether or not the extracted words or compound words are target words of the restriction process (should be excluded from the processing target in the restriction process). It is determined (step S24).
Specifically, the following cases (1) to (4) are listed as the words or compound words to be subjected to the restriction process. When the morphological analysis fails, the first character of the word or compound word is "a", "i", "u",
"E", "o", "tsu", "ya", "yu", "yo",
When "ヮ", "N", "K", "K", etc. Two-byte full-width katakana In case of including half-width characters such as "%", "&", ";", ":", "+" in the middle of the character string that constitutes a word or compound word. When a double-byte character such as "~", "x", "+" is included in the middle of the character string that constitutes a word or compound word. If it is a single kanji.

Since these words or compound words are obviously unsuitable as noun phrases (proper nouns) or sahen nouns, they will be removed in the restriction process. In the processing of step S24, when the word or compound word is a target of the restriction processing (should be excluded from the processing target in the restriction processing) (step S25; Yes), the importance calculation section 22 determines that the word or the compound word. Is discarded (step S27). Then, the process shifts to step S22 again, and the same process is repeated thereafter.

In the processing of step S24, if it is not a word or a compound word that should be subjected to restriction processing (should be excluded from processing by restriction processing) (step S24; N).
o), the importance calculator 22 is a target of stopword processing (should be excluded from processing targets in stopword processing)
Then, it is determined whether or not the word is a word to be flipped or a compound word (step S25). Here, the stop word is a word or compound word used in a plurality of fields, that is, a very general word or compound word, and is an inappropriate word or compound word for estimating the classification. For example, "TEL", "FAX", "OK",
Examples include “NG”.

When it is determined in step S25 that the word or the compound is the target of the stopword processing (should be excluded from the processing target in the stopword processing) (step S25; Yes), the importance calculation section 22 The word or compound word is discarded (step S28).
Then, the process moves to step S22 again, and the same process is repeated thereafter. In the determination in step S25, if the word is not a target word of stopword processing (which should be excluded from the processing target in stopword processing) or a compound word (step S25; No), the word or the word is classified as a classification ontology of the corresponding classification. The compound word is registered, the process proceeds to step S22 again, and the same process is repeated thereafter. As a result, the classification adding unit 24 allocates the classification based on the classification keyword (step S7).

Here, a specific method of classification will be described. FIG. 5 shows a conceptual diagram when classification is performed using the distance (similarity) between vectors in a two-dimensional (N = 2) vector space. A vector X corresponding to the degree of importance of a word or a compound word (classification target phrase) included in the text data corresponding to the document to be classified and extracted by the morphological analysis process is expressed as follows: X = (X1, X2, ..., XN ) Is a vector Y corresponding to the importance of the classification ontology that is a word or a compound word (classification reference phrase) that is included in the document from which the keyword is acquired and is extracted by the morphological analysis process, Y = (Y1, Y2, ... , YN), and the distance D is D = Σ (Xi-Yi) * (Xi-Yi) i = 1,
2, ..., N, if the distance D is smaller than a predetermined threshold value, it is determined that the classification target document belongs to a classification close to the classification to which the keyword acquisition source document (classification reference document) belongs. .
In this case, the predetermined threshold value may be appropriately determined based on various classification results. Originally, the distance D is D = √ {Σ (Xi-Yi) * (Xi-Yi)} i =
It should be 1, 2, ..., N, but the calculation time is shortened by omitting the square root calculation.

By the way, as shown in FIG. 5, the vector area A is the keyword acquisition of the words or compound words (classification target phrases) included in the text data corresponding to the document to be classified and extracted by the morphological analysis process. This is an area where vectors corresponding to words or compound words that are not included in the original document exist. Further, the vector area B is a word or a compound word (classification target phrase) included in the text data corresponding to the document to be classified and extracted by the morphological analysis process, and is included in the document from which the keyword is acquired,
This is an area where vectors corresponding to words or compound words (classification criteria words) extracted by the morphological analysis process exist. Further, the vector region C is a word or compound word included in the document from which the keyword is acquired and extracted by the morphological analysis process (classification reference phrase) that is not included in the text data corresponding to the document to be classified. This is an area where vectors corresponding to words exist.

Normally, in all the vector areas A, B and C, if the distance D between the document to be classified and the document from which the keyword is acquired is smaller than a predetermined threshold value, It can be recognized that the document to be classified belongs to the classification to which the document from which the keyword is acquired belongs. The concept of the distance D will be described more specifically with reference to FIG. In FIG. 6, a document D1: a document document including four words "mountain, mountain, river, river" D2: a document document including three words "mountain, river, river" D3: "mountain, mountain, mountain" Document Document Containing Three Words Document D4: It is assumed that there is a document containing one word "kawa".

In the actual calculation, the distance D is calculated by weighting the words or compound words that are the elements of the vector. This weighting generally uses the TFIDF value. In this case, if a word or compound that is a vector element appears in many documents,
This means that the TFIDF value will be a small value and that the word or compound need not be weighted to determine the classification. On the other hand, when the same word or compound word appears many times in the same document, the TFIDF value becomes large and the word or compound word is an important word for determining the classification. That is. However, for simplification of explanation, consider a vector space in which the vertical axis represents the number of word occurrences of “mountain” and the horizontal axis represents the number of word occurrences of “river”. In this case, as shown in FIG. 7, the vectors corresponding to the documents D1 to D4 are vectors V1 to V, respectively.
It becomes 4. Therefore, the distance D between the document D1 and the document D2
Is considered to be the closest, and it can be determined that the documents D1 and D2 belong to the same classification.

When determining the classification, if the number of vectors belonging to the vector regions A and B is small, the distance D is calculated using only the vectors belonging to the vector regions B and C ( Even if (approximate calculation) is performed, it is possible to obtain the same result as when the distance D is calculated using all the vectors belonging to all the vector regions A, B, and C. Become. However, when the approximate calculation is performed, even when the dimension of the vector area B is one-dimensional (N = 1), that is, the word or compound word corresponding to the vector area B is one word, and the vector area B When the distance D in is 0 (the words match), the result that the document to be classified belongs to the classification to which the document from which the keyword was acquired belongs is obtained.

That is, since the total number of vectors (the number of elements) belonging to the vector area B and the vector belonging to the vector area C is different for each classification, the keyword is used for the classification having a small total number of vectors and a small dimension (small N). The probability that the result that the document to be classified belongs to the classification to which the document of the acquisition source belongs is increased. Therefore, in order to avoid this, the vector area B
The one having a larger dimension (larger N) is preferentially used to calculate the distance D and used for the discrimination of the classification, and if the dimensions are the same, the one having the smaller distance D may be determined to be the closer classification. As a result, it is possible to select a closer classification while reducing the processing time.

[3] Effects of Embodiment As described above, according to this embodiment, when performing morpheme analysis, the above-mentioned part-of-speech processing and symbol processing are performed. The efficiency can be improved and more accurate classification can be performed. Moreover, when the importance of each word or compound word is calculated when the classification ontology is generated, the above-described restriction processing and stop word processing are performed, so that the importance can be calculated more accurately. While reducing the number
A more effective classification ontology can be generated.
Furthermore, by registering a word or compound word obtained by morphological analysis from the text data that composes the document to be classified into the reverse dictionary for morphological analysis, a learning-accurate classification ontology can be obtained by repeatedly performing morphological analysis. Extractable morphological analysis can be realized. Furthermore, the number of words or compound words to be registered in the database can be reduced, and the capacity of the database can be reduced.

[4] Modification of Embodiment [4.1] First Modification In the above description, the database unit 11 was configured integrally with the classification update processing unit 12, but both are connected via a network. It can also be configured as a distributed processing system. In this case, the database unit 11
Each of the databases 15 and 16 and the reverse morphological analysis reverse lookup dictionary 17 that are configured to be stored in another database server via a network, and can be used by a computer system that functions as a plurality of classification update processing units 12. It is also possible to [4.2] Second Modification In the above description, the standardization unit 23 has been described as an indispensable configuration, but there is a demerit that the capacity of the database increases a little even if the standardization unit 23 is not necessarily provided, but it is almost the same. It is possible to obtain the same effect.

[0049]

According to the present invention, the word / phrase extraction unit analyzes a document to be classified and extracts a word or a compound word included in the document to be classified as a classification target phrase. Since the classification criterion words and the classification target words stored in the database are compared and the classification to which the classification target document belongs is determined, accurate classification of documents can be easily performed.
In addition, the morphological analysis unit registers a word or compound that is not registered in the morphological analysis reverse lookup dictionary as an indefinite term in the morphological analysis reverse lookup dictionary, so that learning is performed accurately by repeatedly performing morphological analysis. A morphological analysis capable of extracting a classification ontology can be realized.

Further, in the morphological analysis, when the extracted word or compound word includes a predetermined symbol,
The word or compound word is extracted after removing the symbol from the word or compound word, and the importance calculation is performed by removing the inappropriate words or phrases as the predetermined classification target words or phrases from the extracted words or compound words. The extracted word or the compound word so that the importance calculation is performed by removing the words or phrases that are inappropriate for discriminating the predetermined classification from the extracted word or the compound word, or the extracted word or the compound word By performing a predetermined standardization process on words and calculating the importance of the standardized word or compound word, the processing speed is improved and accurate classification is performed while reducing the database capacity. Can be done.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of a classification processing system according to an embodiment.

FIG. 2 is an overall processing flowchart of a classification processing device.

FIG. 3 is a processing flowchart of a morphological analysis unit.

FIG. 4 is a processing flowchart of an importance calculation section.

FIG. 5 is a conceptual diagram when classification is performed using a distance (similarity) between vectors in a two-dimensional (N = 2) vector space.

FIG. 6 is a conceptual explanatory diagram of a distance D.

FIG. 7 is a specific explanatory diagram of a distance D.

[Explanation of symbols]

10 ... Classification processing system 11 ... Database part (classification database) 12 ... Classification update processing unit (word extraction unit, classification determination unit) 13 ... Display section 14 ... Input section 15 ... Classification database section 16 ... Reverse dictionary for morphological analysis 17 ... Text database section

Claims (42)

[Claims] 1. A classification database unit for preliminarily storing a word or a compound word included in a predetermined classification standard document as a classification standard phrase in association with a specific field, and a document to be classified, and a document to be classified. A word extraction unit that extracts a word or a compound word included in the item as a classification target phrase; and a classification determination unit that compares the classification criterion phrase and the classification target phrase and determines the classification to which the document to be classified belongs. A classification processing device characterized by the above. 2. The classification processing apparatus according to claim 1, wherein a morphological analysis unit that performs morphological analysis of the classification standard document to extract the word or the compound word, and a morphological analysis unit for the extracted word or the compound word. An importance calculation unit for calculating the importance in the classification criterion document, and a criterion for registering the extracted word or the compound word in the classification database unit as the classification criterion phrase in association with the importance and the specific field. A classification processing device comprising: a term registration unit; 3. The classification processing device according to claim 1, wherein in the classification determination, the classification determination unit gives priority to a classification having a large number of words / phrases included in the classification reference words / phrases among the classification target words / phrases. A classification processing device, characterized in that it is determined as a classification to which the document to be classified belongs. 4. The classification processing device according to claim 1, wherein the classification determination unit sets the classification target phrase to N when the classification target phrase includes N words and the compound word.
Represented by a vector in a dimensional vector space, the classification criterion word represented by a vector in the vector space,
A classification processing device, characterized in that the discrimination is performed based on a distance between both vectors. 5. The classification processing apparatus according to claim 2, wherein the classification determination unit sets the importance of the classification target word to N when the classification target word includes N words and the compound word. A classification processing device, wherein the classification reference phrase is represented by a vector in a vector space, the importance of the classification criterion is represented by a vector in the vector space, and the determination is performed based on a distance between the two vectors. 6. The classification processing device according to claim 5, wherein the classification determination unit sets a vector X corresponding to the importance of the classification target phrase to X = (X1, X2, ..., XN), A vector Y corresponding to the importance of the classification criterion word is represented by Y = (Y1, Y2, ..., YN), and a distance D is D = Σ (Xi-Yi) * (Xi-Yi) i = 1,
2, ..., N, it is determined that the document to be classified belongs to a classification close to the classification to which the classification reference document belongs if the distance D is smaller than a predetermined threshold value. Classification processing device. 7. The classification processing device according to claim 2, wherein the morpheme analysis unit is a combination of a plurality of words in which the compound word is a combination of a word and a compound word having a smaller number of characters than the extracted compound word, or A classification processing device, wherein when any of the combinations of compound words having a smaller number of characters than the extracted compound words is extracted, only the compound word is extracted. 8. The classification processing device according to claim 2, wherein the morpheme analysis unit considers a noun including at least a noun phrase and a sahen noun and a predetermined noun phrase as a classification target phrase to be extracted in the morphological analysis. A classification processing device characterized by extracting words belonging to. 9. The classification processing device according to claim 8, wherein the part of speech that can be regarded as the predetermined noun phrase includes a noun form of an adjective verb and a continuous form of a one-stage verb. 10. The classification processing device according to claim 8, further comprising a morphological analysis reverse lookup dictionary for registering the classification target phrase, wherein the morphological analysis unit includes the morphological analysis reverse lookup dictionary. A classification processing device, characterized in that the morphological analysis is performed based on. 11. The classification processing device according to claim 10, wherein the morpheme analysis unit registers a word or a compound word not registered in the morpheme analysis reverse lookup dictionary as an indefinite term in the morpheme analysis reverse lookup dictionary. A classification processing device characterized by: 12. The classification processing device according to claim 2, wherein the morpheme analysis unit, in the morpheme analysis, if the extracted word or compound word includes a predetermined symbol, the word or compound word. A classification processing device, wherein the extracted word or compound word is obtained after removing the symbol from the. 13. The classification processing device according to claim 2, wherein the importance degree calculation unit removes an inappropriate word or phrase from the extracted words or the compound words as a predetermined classification target word or phrase. A classification processing device characterized by performing calculation. 14. The classification processing device according to claim 2, further comprising a standardization unit that performs a predetermined standardization process on the extracted word or the compound word, and the importance degree calculation unit is configured to perform the standardization process after the standardization process. A classification processing device, wherein the importance is calculated for the word or the compound word. 15. A method of controlling a classification processing device, comprising a classification database unit for preliminarily storing a word or a compound word included in a predetermined classification standard document as a classification standard phrase in association with a specific field, the document to be classified. And a word extraction process of extracting a word or a compound word included in the document to be classified as a classification target phrase, comparing the classification criterion phrase and the classification target phrase, and determining the classification to which the classification target document belongs. A method for controlling a classification processing device, comprising: a classification judgment process for judging; 16. The method of controlling a classification processing device according to claim 15, wherein a morphological analysis process of performing morphological analysis of the classification criterion document to extract the word or the compound word, and the extracted word or the compound An importance calculation process of calculating the importance of a word in the classification standard document, and associating the extracted word or the compound word with the importance and the specific field, as the classification standard phrase in the classification database unit. A method of controlling a classification processing device, comprising: a reference word registration process for registering; 17. The control method of the classification processing device according to claim 16, wherein the classification determination step is performed when the classification target phrase includes the N number of words and the compound word, the importance of the classification target phrase. The degree is represented by a vector in an N-dimensional vector space, the importance of the classification criterion is represented by a vector in the vector space, and the determination is performed based on the distance between the two vectors. . 18. The control method of the classification processing apparatus according to claim 17, wherein the classification determination process calculates a vector X corresponding to the importance of the classification target phrase as X = (X1, X2, ..., XN). ), And a vector Y corresponding to the importance of the classification criterion word is represented by Y = (Y1, Y2, ..., YN), and a distance D is D = Σ (Xi-Yi) * (Xi-Yi) ) I = 1,
2, ..., N, it is determined that the document to be classified belongs to a classification close to the classification to which the classification reference document belongs if the distance D is smaller than a predetermined threshold value. Method of controlling classification processing device. 19. The method of controlling a classification processing device according to claim 16, wherein the morphological analysis step is performed on a combination of a plurality of words, the word, and a compound word having a smaller number of characters than the extracted compound word. A control method of a classification processing device, wherein when the combination word is either a combination or a combination of compound words having a smaller number of characters than the extracted compound word, only the compound word is extracted. 20. The method of controlling a classification processing device according to claim 16, wherein the morphological analysis step performs the morphological analysis based on registered contents of a morphological analysis reverse lookup dictionary for registering the classification target words and phrases. A method for controlling a classification processing device, comprising: 21. The method for controlling a classification processing apparatus according to claim 20, wherein in the morphological analysis step, a word or compound not registered in the morphological analysis reverse lookup dictionary is used as an indeterminate word to perform the morphological analysis reverse lookup. A method for controlling a classification processing device, characterized by registering in a dictionary. 22. The method of controlling a classification processing device according to claim 16, wherein in the morphological analysis step, when the extracted word or compound word in the morphological analysis includes a predetermined symbol, the word or A method of controlling a classification processing device, characterized in that the extracted word or compound word is obtained after removing the symbol from the compound word. 23. The method of controlling a classification processing device according to claim 16, wherein the importance calculation step excludes words that are inappropriate as a predetermined classification target word from the extracted words or the compound words. A method of controlling a classification processing device, characterized in that the importance calculation is performed. 24. The control method of a classification processing apparatus according to claim 16, wherein said importance calculation step is an unsuitable word or phrase for discriminating a predetermined classification of said extracted words or said compound words. A method of controlling a classification processing device, characterized in that the importance calculation is performed except for. 25. The control method of the classification processing apparatus according to claim 16, further comprising a standardization process for performing a predetermined standardization process on the extracted word or the compound word, and the importance calculation process includes the standardization process. A method of controlling a classification processing device, comprising: calculating the degree of importance of the processed word or the compound word. 26. A control program for causing a computer to function as a classification processing device using a classification database unit that stores a word or a compound word included in a predetermined classification standard document in association with a specific field in advance as a classification standard phrase. The document to be classified is analyzed, a word or a compound word included in the document to be classified is extracted as a classification target phrase, and the classification criterion phrase and the classification target phrase are compared,
A control program for determining a classification to which the document to be classified belongs. 27. The control program according to claim 26, wherein the morphological analysis of the classification criterion document is performed to extract the word or the compound word, and the extracted word or compound word in the classification criterion document is extracted. A control program for calculating a degree of importance, and registering the extracted word or the compound word in the classification database unit as the classification reference phrase in association with the importance and the specific field. 28. The control program according to claim 26, wherein the classification is discriminated based on a phrase included in the classification criterion phrase among the classification target phrases and the classification criterion phrase. 29. The control program according to claim 28, wherein in the classification determination, the classification target document belongs to the classification target phrase having a larger number of words included in the classification reference phrase, to which the classification target document belongs. A control program characterized by being classified as a classification. 30. The control program according to claim 26, wherein, when the classification target phrase includes N words and the compound word, the classification target phrase is represented by a vector in an N-dimensional vector space, A control program, wherein a classification criterion word is represented by a vector in the vector space, and the discrimination is performed based on a distance between the two vectors. 31. The control program according to claim 27, wherein when the classification target phrase includes N words and the compound word, the importance of the classification target phrase is a vector in an N-dimensional vector space. A control program, characterized in that the importance of the classification criterion phrase is expressed by a vector in the vector space, and the determination is performed based on the distance between the two vectors. 32. The control program according to claim 31, wherein a vector X corresponding to the degree of importance of the classification target phrase.
Is represented by X = (X1, X2, ..., XN), and the vector Y corresponding to the importance of the classification criterion is represented by Y = (Y1, Y2 ,. D, D = Σ (Xi-Yi) * (Xi-Yi) i = 1,
2, ..., N, when the distance D is smaller than a predetermined threshold value, it is determined that the classification target document belongs to a classification close to the classification to which the classification reference document belongs. Control program 33. The control program according to claim 27, wherein the compound word is a combination of a plurality of words, a combination of a compound word having a smaller number of characters than the word and the extracted compound word, or the extracted compound word. A control program for extracting only a compound word if it is one of the combinations of compound words having a smaller number of characters. 34. The control program according to claim 27, wherein in the morphological analysis, a word belonging to a noun and a part of speech regarded as a predetermined noun phrase is extracted as a classification target phrase to be extracted. 35. The control program according to claim 34, wherein the noun includes a noun phrase and a sahen noun. 36. The control program according to claim 34, wherein the part of speech that can be regarded as the predetermined noun phrase includes a noun form of an adjective verb and a continuous form of a one-stage verb. 37. The control program according to claim 34, wherein the morphological analysis is performed based on the registered content of the morphological analysis reverse lookup dictionary for registering the classification target word / phrase. A control program characterized by. 38. The control program according to claim 37, wherein a word or a compound word that is not registered in the morphological analysis reverse lookup dictionary is registered in the morphological analysis reverse lookup dictionary as an indefinite term. program. 39. The control program according to claim 37, wherein, in the morphological analysis, when the extracted word or compound word includes a predetermined symbol, after removing the symbol from the word or compound word. A control program, characterized in that the extracted word or compound word is used. 40. The control program according to claim 37, wherein the importance calculation is performed by removing words that are inappropriate as a predetermined classification target word from the extracted words or the compound words. Control program 41. The control program according to claim 37, wherein a predetermined standardization process is performed on the extracted word or the compound word, and the importance degree is applied to the word or the compound word after the standardization process. A control program characterized by: 42. A recording medium on which the control program according to any one of claims 26 to 41 is recorded.