JP2008176489A - Text discrimination device and text discrimination method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To highly precisely and quickly discriminate the class of a text by quickly learning a discrimination model, and using the discrimination model. <P>SOLUTION: This text discrimination device is provided with a text storage part 1; a text class storage part 2; a text analyzing part 3 for creating the structure information and meaning information of the text; a featured word extraction part 4 for extracting a featured word based on the structure information and frequency information with respect to each word included in the text; a word vector generation part 5 for generating a featured word vector corresponding to the featured word based on the meaning information of the featured word; a text vector generation part 6 for generating a text vector featuring the text based on the featured word vector; a discrimination model learning part 7 for learning a discrimination model by inputting the text vector and a text class as a set; a discrimination model storage part 8 for storing a discrimination model; and a text class discrimination part 9 for deciding the text class corresponding to the text by applying the text vector to the discrimination model. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an apparatus and a method for determining whether text described in a natural language is included in a specific text class using structural information.

  With the development of computer environment, texts can be collected and stored easily. In these texts, useful knowledge for users is buried, and techniques for analyzing texts are being researched. As one of the methods, a method for determining whether text is included in a desired class is known.

  In the method proposed in Non-Patent Document 1, morphological analysis of text is performed, key phrases that are word strings including a specific pattern are extracted, and the text is characterized to perform text classification learning. In this method, characterization based on morphological analysis is performed, so it is not possible to analyze the dependency relationship between words, and it is possible to learn a discrimination model that takes into account the semantic connection of words. There wasn't. For this reason, when text is discriminated using the discriminating method, there is a limit to increasing the classification accuracy.

  On the other hand, in the “text analysis device” proposed in Patent Document 1, the semantic attribute is given to the words constituting the text to be classified by performing syntax analysis. In addition, the text is classified into a specific class by using the relationship between the specified structure including the semantic attribute and the class as a rule. However, in this method, the user needs to specify the relationship between a structure including a semantic attribute and a specific class as a rule. Therefore, this technique could not be used when it is not clear what structure is associated with a particular class. Further, since it is necessary to compare tree structures in order to determine whether a text includes a specific structure, it takes a lot of time for the comparison.

On the other hand, in the “data processing method, information processing system and program using the same” proposed in Patent Document 2, classification learning using a labeled ordered tree is performed. Since the result of parsing the text can be regarded as a kind of this ordered tree with labels, by using this method, a discriminant model for discriminating whether the text is included in the desired class is learned. Can do. However, in this method, it is necessary to count and characterize the partial structures included in the labeled ordered tree, and enormous time is required to count the partial structures. It took a lot of time to discriminate classes using.
JP 2003-16079 A JP 2003-271599 A Shigeaki Sakurai and Akihiro Suyama: “Rule Discovery from Textual Data based on Key Phrase Patterns”, Proceedings of the 19th Annual ACM Symposium on Applied Computing, 606-612 (2004-03) Hirakawa, Amano: Optimal solution search in Japanese analysis, Natural Language Processing Study Group, NL74-2, pp. 9-16, (1989.9) VN Vapnik: “The Nature of Statistical Learning Theory”, Springer, (1995).

  As described above, according to the conventional method, it is impossible to learn a discrimination model at high speed, and to use the discrimination model to determine the text class with high accuracy and high speed.

  The present invention has been made in view of the above circumstances, and provides a text discrimination device and a text discrimination method for learning a discrimination model at high speed and discriminating a text class with high accuracy and high speed using the discrimination model. The purpose is to provide.

  In order to achieve the above object, a text discriminating apparatus according to the present invention is a text discriminating apparatus that discriminates whether or not a text described in a natural language is included in a specific text class. By analyzing the text storage unit that stores text, the text class storage unit that stores the text class corresponding to the text, and the set of text stored in the text storage unit, the structure information and semantic information of each text can be obtained. A text analysis unit to be extracted, a feature word extraction unit for extracting a feature word based on structure information and frequency information in each text for each word included in the set of the text, and the feature for each text A word that generates a feature word vector corresponding to the feature word based on the semantic information and structure information of the word A text generator and a text vector generator that generates a text vector that characterizes the text based on feature word vectors corresponding to all feature words, and a text vector and text generated by the text vector generator. A discriminant model learning unit that learns a discriminant model by inputting a text class stored in the class storage unit as a set, a discriminant model storage unit that stores the discriminated model, and a text vector generator for text A text class determination unit that determines a text class corresponding to the text by applying the text vector generated by the above to the discrimination model stored in the discrimination model storage unit.

  The text discrimination method according to the present invention is a text discrimination method for discriminating whether or not a text described in a natural language is included in a specific text class, and storing the text written in the natural language in a text storage unit. A text storage step, a text class storage step for storing the text class corresponding to the text in the text class storage unit, and analyzing the set of text stored in the text storage unit, A text analysis step for extracting semantic information, a feature word extraction step for extracting feature words based on structure information and frequency information in individual texts, and individual texts for each word included in the set of texts Based on the semantic information and structure information of the feature word for A word vector generation step for generating a feature word vector corresponding to the word, a text vector generation step for generating a text vector characterizing the text from the feature word vectors corresponding to all the feature words, and a text vector generation for the text A discriminant model learning step for learning a discriminant model by inputting the text vector generated by the unit and a text class stored in the text class storage unit as a set; and a discriminant model storage step for storing the discriminated model Text class determination that determines the text class corresponding to the text by applying the text vector generated by the text vector generation step to the text to the discrimination model stored by the discrimination model storage step And having a step, the.

  Further, the text discrimination method according to the present invention is a text discrimination method for discriminating whether or not a text described in a natural language is included in a specific text class. In the text discrimination method, unprocessed from a plurality of texts stored in a text storage unit A first text reading step for reading the text of the text, and each text read in the first text reading step is parsed to analyze a dependency structure between words and a semantic attribute of the word to obtain a syntax. A first text analysis step for generating a tree, and a word extracted from the text stored in the text storage unit, based on the appearance position in the syntax tree generated in the first text analysis step , A word evaluation value calculating step for calculating a word evaluation value for each syntax tree, and the word evaluation value calculating step Feature word selection that calculates the overall evaluation value of words for all texts based on the word evaluation value for each syntax tree that is issued, and selects feature words from the words based on the overall evaluation value A text extraction step for extracting one unprocessed one of the syntax trees corresponding to the text stored in the text storage unit, and a word extracted from the text stored in the text storage unit A first word extraction step for extracting one unprocessed word from the inside, and whether or not the word extracted in the first word extraction step is included in the syntax tree generated in the first text analysis step If it is determined in the first word presence determination step that determines whether the word is included in the syntax tree in the first word presence determination step, A search is made for a position where a word appears in the syntax tree, and an attribute vector corresponding to the feature word is generated based on a semantic attribute attached to the searched word and a structural attribute based on the appearance position of the searched word. An attribute vector indicating that the word does not exist in the text when the feature word vector generation step and the first word presence determination step determine that the word is not included in the syntax tree. Corresponding to the text based on the first non-feature word vector generation step to be generated, and the attribute vector of each word generated by the first feature word vector generation step and the first non-feature word vector generation step A first text vector generation step for generating an attribute vector, and an attribute generated in the first text vector generation step A discriminant model learning step for generating a discriminant model corresponding to all texts by combining a vector and a text class corresponding to the text, and a new data stored in the text storage unit after the discriminant model learning step A second text reading step for reading unprocessed text from the text, and the text read in the second text reading step are parsed to analyze the dependency structure between words and the semantic attributes of the words. A second text analysis step of generating a syntax tree, a second word extraction step of extracting one unprocessed word from words extracted from the text stored in the text storage unit, The word extracted in the second word extraction step is the second text analysis step. A second word existence determination step for determining whether or not the word is included in the syntax tree generated in step (b), and the second word presence determination step determines that the word is included in the syntax tree A second feature that searches for the position of the appearance and generates an attribute vector corresponding to the feature word based on a semantic attribute associated with the searched word and a structure attribute based on the appearance position of the searched word. A second step of generating an attribute vector indicating that the word does not exist in the text when it is determined that the word is not included in the syntax tree in the word vector generation step and the second word presence determination step; Non-feature word vector generation step, and the attribute vector of each word generated by the second feature word vector generation step and the second non-feature word vector generation step. Therefore, a second text vector generation step for generating an attribute vector corresponding to the text, and an attribute vector corresponding to the new text generated in the second text vector generation step are determined in the discrimination model learning step. And a discriminating step for discriminating a text class corresponding to the text by applying to the generated discriminant model.

  According to the present invention, syntax analysis that outputs more information than morphological analysis is used, while the result of syntax analysis is simplified and used. As a result, the discriminant model can be learned at a higher speed than when the discriminant model is learned by directly using the structural information obtained by the syntax analysis. In addition, since it is possible to perform characterization that preserves structural information to some extent, it is possible to learn a discrimination model with high discrimination accuracy.

  Embodiments of a text discrimination device and a text discrimination method according to the present invention will be described below with reference to the drawings. First, an embodiment of a text discrimination device according to the present invention will be described with reference to FIG.

  As shown in FIG. 1, this text discrimination device has a text storage unit 1 for storing text written in a natural language, and a text class storage unit 2 for storing a text class corresponding to the text. Further, by analyzing a set of text stored in the text storage unit 1, a text analysis unit 3 that extracts structure information and semantic information of each text, and for each word included in the set of text The feature word extraction unit 4 extracts feature words based on structure information and frequency information in individual texts.

  The text discrimination device further includes a word vector generation unit 5 that generates a feature word vector corresponding to the feature word based on the semantic information and structure information of the feature word for each text, A text vector generation unit 6 that generates a text vector that characterizes the text based on the feature word vector corresponding to the feature word.

  The text discrimination device further includes a discrimination model learning unit 7 for learning a discrimination model by inputting the text vector generated by the text vector generation unit 6 and the text class stored in the text class storage unit 2 as a set. By applying the discriminant model storage unit 8 for storing the discriminated model learned and the text vector generated by the text vector generator 6 for the text to the discriminant model stored in the discriminant model storage unit 8, A text class determination unit 9 that determines a text class corresponding to the text.

  Next, the details of the embodiment of the text discrimination method according to the present invention will be described with reference to the flowcharts of FIGS. FIG. 2 is a flowchart showing the flow of discriminant model learning in this embodiment, and FIG. 3 is a flowchart showing the flow of the text discriminating procedure using the discriminant model in this embodiment.

  In step Sa1 of FIG. 2, one unprocessed text is read from the text stored in the text storage unit 1. At this time, if there is no text to be read, the process proceeds to step Sa4. On the other hand, if there is a text to be read, the process proceeds to step Sa2.

  In step Sa2, the text analysis unit 3 analyzes the dependency structure between words and the semantic attribute of the word by applying the syntax analysis described in Non-Patent Document 2, for example, to the read text. Thus, the syntax tree illustrated in FIGS. 4 and 5 is generated. That is, by referring to a dictionary in which a word and its attributes are described, the given text is grouped into phrases and decomposed into a column accompanied by the word and attributes related to the word. In the syntax tree, this attribute is expressed as a label (part of speech, negation, etc.) attached to the right side of the word.

  Next, a dependency relationship is established between a noun and the verb closest to the noun by using surface features such as case particles, which are particles that show the relationship of body words and words that are similar to body words to other words. And generate a simple syntax tree.

  Next, combinations of nouns and verbs that can be modified are extracted from the simple syntax tree, and a plurality of dependency candidates are generated for the combination if necessary. At this time, a score calculated based on the settable semantic dependency relationship and its certainty is given to the dependency candidate by referring to a dictionary describing the semantic dependency relationship. Yes. In the syntax tree, this semantic dependency relationship is expressed by a label (eg, case, etc.) given on the link.

  Finally, after considering the semantic constraints between clauses, an optimum dependency candidate is determined from the dependency candidate group, and a syntax tree having the highest score is generated. Further, each clause is expanded to generate a syntax tree in which each node is composed of words.

For example,
Example 1: “I bought a computer the other day, but I ’m satisfied with its processing speed.”
4 is obtained, the syntax tree shown in FIG. 4 can be obtained. In this syntax tree, semantic attributes are given in parentheses on the right side of words, and the relationship between words is connected by a line. Assume that the symbols indicating the relationship between the words are labeled. In this example, it is assumed that only some semantic attributes corresponding to words are described.

  On the other hand, if a Japanese sentence such as “Example 2:“ I bought a computer the other day but I am not satisfied with its processing speed ”is given, the syntax tree shown in FIG. 5 is obtained. Can do.

  Next, in step Sa3, the text analysis unit 3 calculates word evaluation values for words extracted from all the texts stored in the text storage unit 1 based on the appearance positions in the target syntax tree. Then, the process returns to step Sa1.

For example, it is assumed that the word evaluation value of each word is calculated by Expression (1) and Expression (2).

Here, w _i is the i-th word extracted from the text storage unit 1, and T _j is the word constituting the j-th text. Also, let t _ij be the number that contains the word w _i in the j th text, and match dept (i, j, k) to the i th word that appears in the syntax tree corresponding to the j th text. A function for obtaining the depth of the kth word. However, the depth at the root node of the syntax tree is 0. Therefore, since the word evaluation values of the word “satisfaction” and the word “computer” in the syntax tree of FIG. 4 appear only once in the syntax tree, 1 (= 1 / (0 + 1)), 0.25 (= 1 / (3 + 1)).

  In step Sa4, based on the word evaluation value of each word calculated by the feature word extraction unit 4 for each syntax tree, a total evaluation value of words for all texts is calculated, and based on the total evaluation value. Select a feature word from the words.

For example, a conventionally proposed tf-idf (term frequency-inverse document frequency) value (the second term of the expression (3)) is corrected in consideration of the appearance position of the word in the syntax tree. ) To calculate the overall evaluation value of the word.

Here, D is the total number of texts, d _i is the number of texts with the i-th word, and n _j is the number of words contained in the j-th text. By this comprehensive evaluation value, a high comprehensive evaluation value can be given to a word that appears in association with a specific text and appears at the top of the syntax tree.

  In this step, the feature word extraction unit 4 extracts, as a feature word, a word that is equal to or higher than a threshold value specified for the word.

  In step Sa <b> 5, the word vector generation unit 5 extracts one unprocessed syntax tree corresponding to the text stored in the text storage unit 1. At this time, if there is no syntax tree to be extracted, the process proceeds to step Sa11. On the other hand, if there is a syntax tree to be extracted, the process proceeds to step Sa6.

  In step Sa6, the word vector generation unit 5 extracts one unprocessed word from the words extracted from the text stored in the text storage unit 1. At this time, if there is no extracted word, the process proceeds to step Sa10. On the other hand, if there is a word to be extracted, the process proceeds to step Sa7.

  In step Sa7, the word vector generation unit 5 determines whether or not the extracted word is included in the target syntax tree. At this time, if the extracted word is included, the process proceeds to step Sa8. On the other hand, if the extracted word is not included, the process proceeds to step Sa9.

  In step Sa8, the word vector generation unit 5 searches for a position where the word appears in the syntax tree. Further, an attribute vector corresponding to the feature word is generated based on the semantic attribute attached to the searched word and the structure attribute based on the appearance position of the searched word, and the process returns to step Sa5.

  For example, the word part of speech (value of the part of speech attribute in the syntax tree) and the word negation (value of the negative attribute in the syntax tree) are the semantic attributes of the word. The number of branches is extracted as a structural attribute of the word. At this time, if the word “satisfied” is given in Example 1, the attribute vector shown in FIG. 6 is generated. However, since the meaning attribute indicating whether the word is negated is not given to “satisfied” in Example 1, the value of this meaning attribute is set to “none”. In Example 2, when a word such as “satisfied” is given, the attribute vector shown in FIG. 7 is generated.

  In step Sa9, the word vector generation unit 5 generates an attribute vector indicating that the word does not exist in the text, and returns the process to step Sa5.

  For example, “un-extracted” is set for a discrete attribute, and a value outside the range of the numeric attribute (−1 for depth and the number of branches) is set for a numerical attribute. At this time, in the example 1, when a word such as “dissatisfied” is given, the attribute vector shown in FIG. 8 is generated.

  In step Sa10, the text vector generation unit 6 generates an attribute vector corresponding to the text from the attribute vector of each word generated by the word vector generation unit 5 for the target text.

  For example, a text attribute vector is generated by concatenating word attribute vectors. At this time, for example 1 and example 2, an attribute vector (text vector) shown in FIG. 9 is generated.

  In step Sa11, the text vector generated by the text vector generation unit 6 and the text class stored in the text class storage unit 2 are paired with the text vector, thereby corresponding to all texts. Generate learning cases.

  For example, if the text classes corresponding to Example 1 and Example 2 are “no dissatisfaction” and “no dissatisfaction”, respectively, learning examples shown in FIG. 10 are generated for Example 1 and Example 2.

  In addition, for example, the learning example is learned using a support vector machine (SVM) described in Non-Patent Document 3 to learn a discrimination model for discriminating a text class, and the learned discrimination model is stored in the discrimination model storage unit 8. Store.

  As described above, if the discrimination model is learned, when a new text is given, the text class corresponding to the text can be discriminated by applying the text to the discrimination model. . That is, in step Sb1 of FIG. 3, the text analysis unit 3 reads one unprocessed text among the newly stored text from the text storage unit 1. At this time, if there is a text to be read, the process proceeds to step Sb2. On the other hand, if there is no text to be read, the process ends.

  In step Sb2, the text analysis unit 3 analyzes the dependency structure between words and the semantic attributes of the words for the read text to generate a syntax tree.

  In step Sb3, the word vector generation unit 5 extracts one unprocessed word from the words extracted from the text stored in the text storage unit 1. At this time, if there is no word to be extracted, the process proceeds to step Sb7. On the other hand, if there is a word to be extracted, the process proceeds to step Sb4.

  In step Sb4, it is determined whether the word extracted by the word vector generation unit 5 is included in the target syntax tree. At this time, if the extracted word is not included, the process proceeds to step Sb5. On the other hand, if the extracted word is included, the process proceeds to step Sb6.

  In step Sb5, the word vector generation unit 5 searches for a position where the word appears in the syntax tree. Further, an attribute vector corresponding to the feature word is generated based on the semantic attribute attached to the searched word and the structural attribute based on the appearance position of the searched word, and the process returns to step Sb3.

  In step Sb6, the word vector generation unit 5 generates an attribute vector indicating that the word does not exist in the text, and returns the process to step Sb3.

  In step Sb7, the text vector generation unit 6 generates an attribute vector corresponding to the text from the attribute vector of each word generated by the word vector generation unit 5 for the target text.

  In step Sb8, the text class determination unit 9 applies the attribute vector corresponding to the new text generated by the text vector generation unit 6 to the discrimination model stored in the discrimination model storage unit 8, thereby Determine the text class corresponding to the text.

  By implementing the above steps Sb1 to Sb7 for all the texts newly stored in the text storage unit 1, the text classes of all the texts can be determined.

  The embodiment of the present invention has been described above. This embodiment is merely an example, and the present invention is not limited to the above-described embodiment. For example, in the extraction of feature words, an example is shown in which the overall evaluation value is calculated based on the product of the tf-idf value and the word evaluation value in the calculation of the overall evaluation value reflecting the structure information. It may be calculated, may be calculated based on the evaluation value other than the tf-idf value and the word evaluation value, or may be calculated based on information other than the depth in the syntax tree.

  In addition, in the generation of word vectors, an example is shown in which only the part of speech and negation are used as semantic attributes, and only the word depth and the number of branches are used as structural attributes. Other structural attributes such as the number of words attached to a word in the syntax tree and the value of the attribute assigned to the upper branch of the word may be used as the attribute and the structural attribute, or used for each word. The word vector may be generated by changing the semantic attribute and the structural attribute.

  Furthermore, in the generation of text vectors, in addition to concatenating word vectors, an attribute vector is generated in which each word is given a weight based on a comprehensive evaluation value, and a discriminant model is learned in consideration of the attribute weight. May be.

  In addition, the text discrimination device and the like can be configured with various modifications without departing from the spirit of the present invention.

The block diagram which shows the structure of one Embodiment of the text discrimination | determination apparatus based on this invention. The flowchart which shows the flow of discrimination | determination model learning in one Embodiment of the text discrimination | determination method concerning this invention. The flowchart which shows the flow of the text discrimination | determination procedure using a discrimination | determination model in one Embodiment of the text discrimination | determination method concerning this invention. The figure which shows the example of the syntax analysis result of the text example 1 analyzed by the text analysis part in one Embodiment of the text discrimination | determination apparatus based on this invention. The figure which shows the example of the syntax analysis result of the text example 2 analyzed by the text analysis part in one Embodiment of the text discrimination | determination apparatus based on this invention. The figure which shows the example of the attribute vector corresponding to the word "satisfaction" in the text example 1 produced | generated by the word vector production | generation part in one Embodiment of the text discrimination | determination apparatus based on this invention. The figure which shows the example of the attribute vector corresponding to the word "satisfaction" in the text example 2 produced | generated by the word vector production | generation part in one Embodiment of the text discrimination | determination apparatus based on this invention. The figure which shows the example of the attribute vector corresponding to the word "dissatisfaction" in the text example 1 produced | generated by the word vector production | generation part in one Embodiment of the text discrimination | determination apparatus based on this invention. The figure which shows the example of the attribute vector of a text corresponding to the text of the text example 1 and the example 2 produced | generated by the word vector production | generation part in one Embodiment of the text discrimination | determination apparatus based on this invention. The figure which shows the example of the learning example corresponding to the text of the text example 1 and the example 2 produced | generated by the discrimination | determination learning part in one Embodiment of the text discrimination | determination apparatus based on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Text storage part 2 ... Text class storage part 3 ... Text analysis part 4 ... Feature word extraction part 5 ... Word vector generation part 6 ... Text vector generation part 7 ... Discrimination model learning part 8 ... Discrimination model storage part 9 ... Text class Judgment part

Claims (3)

In a text discriminating apparatus for discriminating whether or not text described in a natural language is included in a specific text class,
A text storage for storing text written in natural language;
A text class storage for storing a text class corresponding to the text;
A text analysis unit that extracts structure information and semantic information of each text by analyzing a set of text stored in the text storage unit;
A feature word extraction unit that extracts a feature word for each word included in the set of texts based on structure information and frequency information in each text;
A word vector generation unit that generates a feature word vector corresponding to the feature word based on the semantic information and structure information of the feature word for each text;
A text vector generation unit that generates a text vector that characterizes the text based on a feature word vector corresponding to all feature words for the text;
A discriminant model learning unit that learns a discriminant model by inputting the text vector generated by the text vector generation unit and the text class stored in the text class storage unit as a set;
A discriminant model storage for storing the discriminated model learned, and
A text class determination unit that determines a text class corresponding to the text by applying the text vector generated by the text vector generation unit to the text to the determination model stored in the determination model storage unit;
A text discrimination device characterized by comprising: In a text determination method for determining whether text described in a natural language is included in a specific text class,
A text storage step for storing text written in a natural language in a text storage unit;
A text class storage step of storing a text class corresponding to the text in the text class storage unit;
A text analysis step for extracting structure information and semantic information of each text by analyzing a set of text stored in the text storage;
A feature word extraction step of extracting a feature word for each word included in the set of texts based on structure information and frequency information in each text;
A word vector generation step of generating a feature word vector corresponding to the feature word based on the semantic information and structure information of the feature word for each text;
A text vector generation step for generating a text vector characterizing the text from feature word vectors corresponding to all feature words for the text;
A discriminant model learning step of learning a discriminant model by inputting the text vector generated by the text vector generator and the text class stored in the text class storage unit as a set;
A discriminant model storing step for storing the learned discriminant model;
A text class determination step for determining a text class corresponding to the text by applying the text vector generated by the text vector generation step to the text to the discrimination model stored by the discrimination model storage step;
A text discrimination method characterized by comprising: In a text determination method for determining whether text described in a natural language is included in a specific text class,
A first text reading step for reading raw text from a plurality of texts stored in a text storage;
A first text analysis step of performing a syntax analysis on each text read in the first text reading step, and analyzing a dependency structure between words and a semantic attribute of the word to generate a syntax tree;
For each word extracted from the text stored in the text storage unit, a word evaluation value is calculated for each syntax tree based on the appearance position in the syntax tree generated in the first text analysis step. A word evaluation value calculation step;
Based on the word evaluation value for each syntax tree calculated in the word evaluation value calculation step, a total evaluation value of words for all texts is calculated, and a feature is selected from the words based on the total evaluation value. A feature word selection step for selecting a word;
A text extraction step of extracting one unprocessed one of the syntax trees corresponding to the text stored in the text storage unit;
A first word extracting step of extracting one unprocessed word from words extracted from the text stored in the text storage unit;
A first word existence determination step for determining whether the word extracted in the first word extraction step is included in the syntax tree generated in the first text analysis step;
Meanings associated with the searched word by searching for a position where the word appears in the syntax tree when it is determined in the first word existence determining step that the word is included in the syntax tree. A first feature word vector generation step for generating an attribute vector corresponding to the feature word based on the attribute and the structural attribute based on the appearance position of the searched word;
A first non-feature word vector that generates an attribute vector indicating that the word does not exist in the text when it is determined in the first word presence determination step that the word is not included in the syntax tree. Generation step;
A first text vector generation step for generating an attribute vector corresponding to the text based on the attribute vector of each word generated by the first feature word vector generation step and the first non-feature word vector generation step; ,
A discriminant model learning step of generating discriminant models corresponding to all texts by combining the attribute vector generated in the first text vector generating step and a text class corresponding to the text;
A second text reading step of reading unprocessed text from text newly stored in the text storage unit after the discriminant model learning step;
A second text analysis step of performing a syntax analysis on the text read in the second text reading step and analyzing a dependency structure between words and a semantic attribute of the word to generate a syntax tree;
A second word extracting step of extracting one unprocessed word from words extracted from the text stored in the text storage unit;
A second word presence determining step for determining whether the word extracted in the second word extracting step is included in the syntax tree generated in the second text analyzing step;
When it is determined in the second word presence determination step that the word is included in the syntax tree, the position where the word appears is searched, the semantic attribute attached to the searched word and the searched word A second feature word vector generation step for generating an attribute vector corresponding to the feature word based on the structure attribute based on the appearance position;
Second non-feature word vector generation for generating an attribute vector indicating that the word does not exist in the text when it is determined in the second word presence determination step that the word is not included in the syntax tree Steps,
A second text vector generation step for generating an attribute vector corresponding to the text based on the attribute vector of each word generated by the second feature word vector generation step and the second non-feature word vector generation step; ,
Discrimination for discriminating a text class corresponding to the text by applying the attribute vector corresponding to the new text generated in the second text vector generation step to the discrimination model generated in the discrimination model learning step Steps,
A text discrimination method characterized by comprising:

Images