JP7358132B2 - Computer systems and document classification methods - Google Patents

Description

The present invention relates to a computer system and method for classifying data.

For the purpose of supporting information search and browsing, tags and the like indicating document classification (category or class) are added. Known tags added to documents include F terms and international patent classifications for patent specifications, and MeSH terms for medical papers. Generally, tags are manually attached by a person with specialized knowledge, so there is a problem in that the task of attaching tags requires a great deal of effort.

In order to maintain the quality of information retrieval, it is a prerequisite that tags are assigned with a high degree of accuracy based on consistent standards, so rather than complete automation, there is a need for a method to assist users in making these classifications. It is being

For example, Non-Patent Document 1 discloses a method of assigning classification results to documents by constructing a graph from documents and words. However, Non-Patent Document 1 does not disclose a method for presenting the basis of classification together with the assignment of classification results. If the automatically assigned classification results are simply presented to the user, the user cannot grasp the basis for assigning the classification results. Therefore, the user cannot decide whether or not to accept the classification results.

On the other hand, the techniques disclosed in Non-Patent Document 2 and Non-Patent Document 3 are known. Non-Patent Document 2 discloses a method for improving the explainability of classification by presenting, along with the document classification results, locations (words) whose removal would significantly change the classification results. Non-Patent Document 3 discloses a method that uses a recurrent neural network and an attention mechanism to present a document classification result as well as a part of the document as a basis.

Liang Yao, Chengsheng Mao, and Yuan Luo. 2018. Graph Convolutional Networks for Text Classification. AAAI. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. KDD. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical Attention Networks for Document Classification. NACCL.

However, since the method described in Non-Patent Document 2 does not make any changes to the document classification method itself, the presented content is deviated from the basis for human thinking, and it cannot sufficiently support document classification. . In the evidence presentation based on the attention mechanism described in Non-Patent Document 3, the parts that contribute to classification and the parts of the evidence that people think of do not match, so there is a trade-off between the appropriateness of the evidence presentation and the classification accuracy. There is a problem with this.

The present invention realizes a system and method that presents a part of a document as a basis to a user along with a document classification result so that there is no trade-off between the appropriateness of presentation of the basis and classification accuracy.

A typical example of the invention disclosed in this application is as follows. That is, the computer system includes at least one computer, the at least one computer includes a processor, a memory connected to the processor, and an interface connected to the processor, and the computer system a graph construction unit that receives data input and generates a graph having the document and the elements of the document as vertices , and an index used for classifying the document into one of a plurality of classes for each of the plurality of vertices ; and a classification unit that classifies the document based on the index of at least one of the vertices , and identifies a base point on the document that is composed of at least one element of the document that contributed to the classification. , a document reconstruction unit that presents the results of the classification and the basis points on the document, and the classification unit uses the output of the model that receives the graph as input to determine the classification for each of the classes. A value representing the corresponding probability is calculated as the index, and the document reconstruction unit classifies the document based on the index of the vertex corresponding to the document, and classifies the document based on the index of the vertex corresponding to the element of the document. Based on the index of the class into which the document is classified, a basis location on the document is identified .

According to one embodiment of the present invention, a part of a document can be presented to a user as a basis along with the document classification result so that there is no trade-off between the appropriateness of presentation of the basis and classification accuracy. Problems, configurations, and effects other than those described above will be made clear by the description of the following examples.

1 is a diagram illustrating an example of the configuration of a computer system according to a first embodiment; FIG. 7 is a flowchart illustrating an example of processing executed by the graph construction unit of the first embodiment. 7 is a flowchart illustrating an example of processing executed by the graph construction unit of the first embodiment. 7 is a flowchart illustrating an example of processing executed by the graph construction unit of the first embodiment. FIG. 3 is a diagram illustrating data input/output in processing executed by the graph construction unit of the first embodiment. FIG. 3 is a diagram illustrating data input/output in processing executed by the graph construction unit of the first embodiment. FIG. 2 is a diagram illustrating a graph constructed by a graph construction unit of Example 1. FIG. 5 is a flowchart illustrating an example of processing executed by the document reconstruction unit of the first embodiment. FIG. 3 is a diagram illustrating the structure of output data in processing executed by the document reconstruction unit of the first embodiment. FIG. 2 is a diagram illustrating an example of a user interface presented by a display unit of Example 1. FIG. 5 is a flowchart illustrating an example of processing executed by the display unit of the first embodiment. 12 is a flowchart illustrating processing executed by a graph construction unit of Example 2. FIG. 12 is a flowchart illustrating processing executed by a graph construction unit of Example 2. FIG. FIG. 7 is a diagram illustrating data input/output in processing executed by the graph construction unit of the second embodiment. FIG. 7 is a diagram illustrating data input/output in processing executed by the graph construction unit of the second embodiment. FIG. 7 is a diagram illustrating a user interface in processing executed by the display unit of Example 3; FIG. 7 is a diagram showing an example of the configuration of a computer according to a fourth embodiment. FIG. 12 is a diagram illustrating a user interface in processing executed by a related element display unit according to a fourth embodiment. 12 is a flowchart illustrating an example of a process executed by the related element display unit according to the fourth embodiment. FIG. 12 is a diagram illustrating the data structure of data used in processing executed by the related element display unit of the fourth embodiment. FIG. 12 is a diagram illustrating data input/output in processing executed by the graph construction unit of Example 5; FIG. 12 is a diagram illustrating data input/output in processing executed by the graph construction unit of Example 5; 12 is a flowchart illustrating an example of step S211 executed by the graph construction unit of the fifth embodiment.

Embodiments of the present invention will be described below with reference to the drawings. However, the present invention should not be construed as being limited to the contents described in the embodiments shown below. Those skilled in the art will readily understand that the specific configuration can be changed without departing from the spirit or spirit of the present invention.

In the configuration of the invention described below, the same or similar configurations or functions are denoted by the same reference numerals, and redundant explanations will be omitted.

In this specification, etc., expressions such as "first," "second," and "third" are used to identify constituent elements, and do not necessarily limit the number or order.

The position, size, shape, range, etc. of each component shown in the drawings etc. may not represent the actual position, size, shape, range, etc. in order to facilitate understanding of the invention. Therefore, the present invention is not limited to the position, size, shape, range, etc. disclosed in the drawings and the like.

FIG. 1 is a diagram showing an example of the configuration of a computer system according to the first embodiment.

The computer system presents a part of the document as a proof point to the user along with the document classification result. Embodiment 1 assumes a computer system that classifies documents consisting only of character strings. Furthermore, it is assumed that document classification is a process of determining which of a plurality of classes a document falls into.

The computer system is composed of two computers 100-1 and 100-2. Computer 100-1 and computer 100-2 are connected to each other via network 120. The network 120 is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). Note that the present invention is not limited to the type of network 120. Further, the connection method of the network 120 may be either wired or wireless.

The hardware configurations of the computer 100-1 and the computer 100-2 in the first embodiment are the same. Note that the hardware configurations of the computer 100-1 and the computer 100-2 may be different. In the following description, if the computer 100-1 and the computer 100-2 are not distinguished, they will be referred to as the computer 100.

Computer 100 has a processor 101, memory 102, and network interface 103. Each piece of hardware is connected to each other via an internal bus. The computer 100 may include a storage device such as a hard disk drive (HDD) and a solid state drive (SSD), an input device such as a keyboard, a mouse, and a touch panel, and an output device such as a display.

Processor 101 executes a program stored in memory 102. By executing processing according to a program, the processor 101 operates as a functional unit (module) having a predetermined function. In the following description, when a process is described using a functional unit as a subject, it means that the processor 101 is executing a program that implements the functional unit.

The memory 102 stores programs executed by the processor 101 and information necessary for the programs. The memory 102 also includes a work area that is temporarily used by the program.

Network interface 103 connects to other devices via a network.

Here, the programs and information stored in the memory 102 of the computers 100-1 and 100-2 will be explained.

The memory 102 of the computer 100-1 stores a program that implements the graph construction unit 110, and also holds graph information 111.

The graph construction unit 110 receives input of document data, and constructs a graph 600 (see FIG. 6) having the document and the constituent elements of the document as vertices (nodes) from the data as information representing the structure of the document. Components of a document include, for example, words and paragraphs (sentences). Note that the present invention is not limited to a graph format, and any data that indicates the structure of a document may be used. Details of the graph 600 generated by the graph construction unit 110 will be explained using FIG. 6.

The graph information 111 stores data representing the graph 600 constructed by the graph construction unit 110.

The memory of the computer 100-2 stores programs that implement the classification model learning section 112, the classification section 114, the document reconstruction section 115, and the display section 116, and also holds classification model information 113.

The classification model learning unit 112 acquires the graph information 111 and executes a learning process for calculating various parameters that define the classification model. The classification model of this embodiment is a model for calculating a probability value corresponding to a certain class at the vertex of a graph. A probability value corresponding to a certain class is also described as a class probability value. As many class probability values as there are classes are calculated for one vertex.

The classification model information 113 stores information on the classification model learned by the classification model learning unit 112, that is, parameters.

The classification unit 114 calculates class probability values of vertices forming the graph based on the classification model stored in the classification model information 113. Further, the classification unit 114 classifies the document based on each class probability value of the vertices forming the document. More specifically, the classification unit 114 calculates, as a result of classification, a value for assigning a label to at least one vertex constituting the document. A label is a value that represents the class of a document.

The document reconstruction unit 115 determines the classification of the document based on the values calculated by the classification unit 114, reconstructs the document based on the graph 600, and further uses the values calculated by the classification unit 114 and Based on the reconstructed document, the basis points on the document that contributed to the classification of the document are identified. A proof point consists of at least one component. The document reconstruction unit 115 generates basis information for presenting the specified basis location on the document. Further, the document reconstruction unit 115 outputs the classification result and basis information along with the document to be classified.

The display unit 116 outputs data for presenting the classification results and the basis locations to the user.

Note that the computer system may include a terminal for inputting data to the computer 100 and acquiring data from the computer 100. The functions of computer 100-1 and computer 100-2 may be combined into one computer. Further, the computer 100-1 and the computer 100-2 may be realized using virtualization technology.

Here, the flow of processing of a computer system that classifies documents based on a predetermined classification system will be described.

(Process A1) First, document data 500 (see FIG. 5) of a document assigned any label in the classification system is input to the graph construction unit 110. The graph construction unit 110 constructs a document, a paragraph composed of two or more words, and a non-hierarchical graph 600 with words as vertices, and stores data representing the constructed graph 600 in the graph information 111.

(Processing A2) The classification model learning unit 112 executes a learning process and stores the learned parameters in the classification model information 113. The classification model generated by the learning process is a model for calculating the class probability value of each vertex of the graph 600.

(Process A3) Based on the classification model information 113, the classification unit 114 assigns a label to at least one vertex related to the document to be classified in the graph 600.

(Process A4) The document reconstruction unit 115 outputs the document to be classified, as well as the classification result and basis information, based on the graph 600 and the class probability value of each vertex. The display unit 116 presents the document, the classification results, and the evidence points to the user.

This computer system can be utilized to support the classification of patent specifications. For example, the computer system determines whether a certain international patent classification should be assigned to a patent specification related to an application, and presents the international patent classification to be assigned and the basis for assignment to the user.

This computer system can also be used to classify documents into exclusive categories. For example, in order to facilitate searching for medical articles, the computer system can search for ``internal medicine'', ``surgery'', ``orthopaedics'', ``neurosurgery'', ``obstetrics and gynecology'', ``dermatology'', ``ophthalmology'', etc. It is determined whether an abstract of a medical paper belongs to any category of "otolaryngology" and the basis of the determination is presented to the user along with the category to which it belongs.

Furthermore, this computer system is not limited to documents, and can also be used to classify data composed of elements with different granularity. For example, a computer system determines the presence or absence of toxicity from the molecular structure of a compound, and presents the basis for the determination to the user along with the determination result. In this case, molecules correspond to documents, atoms correspond to words, and groups correspond to paragraphs.

The processing executed by the graph construction unit 110 will be explained using FIGS. 2 to 6. 2 to 4 are flowcharts illustrating an example of processing executed by the graph construction unit 110 of the first embodiment. 5A and 5B are diagrams illustrating data input/output in processing executed by the graph construction unit 110 of the first embodiment. FIG. 6 is a diagram illustrating a graph constructed by the graph construction unit 110 of the first embodiment.

Here, we will consider as an example a use case that supports the determination of whether or not to assign an arbitrary international patent classification to a patent specification. For ease of explanation, the patent specification shall consist of two paragraphs. In this case, data representing the structure of the patent specification is expressed as a graph 600 as shown in FIG.

The graph 600 is managed on the computer as an adjacency matrix 520 shown in FIG. In the adjacency matrix 520, the element in the i-th row and the j-th column represents the existence and weight of an edge from the j-th vertex to the i-th vertex. Specifically, if the element value is zero, it means that there is no edge connecting the vertices, and if the element value is non-zero, in the direction from the j-th vertex to the i-th vertex, Indicates that there is an edge for which the element value is set as a weight.

The graph construction unit 110 generates an adjacency matrix 520 in order to treat the document as a graph 600. Note that since the adjacency matrix 520 is often a sparse matrix, it is desirable that it is also held as a sparse matrix in the memory 102 of the computer 100.

The graph construction unit 110 receives document data 500 including an index 501, a label 502, and a plurality of paragraph sentence data 503 (step S201).

Here, the index 501 is a number to uniquely identify the document data 500. Numbers shall be assigned sequentially starting from 0.

A label 502 is the document classification result. In the label 502 of this embodiment, a value indicating whether or not a predetermined international patent classification should be assigned to the document data 500 is stored. In this embodiment, when the international patent classification should be assigned to the document data 500, "1" is stored in the label 502, and when the international patent classification should not be assigned to the document data 500, the label 502 stores "1". 0'' shall be stored.

Paragraph text data 503 is data corresponding to paragraph sentences included in a document. For example, a sentence including one paragraph tag <p> is treated as paragraph sentence data 503. Note that the document division unit may be a unit consisting of one or more words; for example, the document may be divided based on formal paragraphs, sentences, and discourse structure.

In addition, in this embodiment, the number of document data 500 (learning data) with a known label 502 is N _A , and the number of document data 500 (evaluation data) to which a label 502 is assigned is N _Q , in total ( _NA + N _Q ). It is assumed that document data 500 exists. It is assumed that -1 is stored as a placeholder in the label 502 of the evaluation data. Returning to the explanation of FIG. 2.

Next, the graph construction unit 110 starts loop processing of the document data 500 (step S202).

Specifically, the graph construction unit 110 selects one target document data 500 from among the plurality of document data 500.

Next, the graph construction unit 110 performs preprocessing of the document data (step S203). Specifically, the following processing is executed.

The graph construction unit 110 starts loop processing of the paragraph sentence data 503 of the target document data 500 (step S301).

Specifically, the graph construction unit 110 selects one target paragraph sentence data 503 from a plurality of paragraph sentence data 503 included in the target document data 500.

Next, the graph construction unit 110 decomposes the target paragraph sentence data 503 into units of morphemes (referred to as words) (step S302). Note that the unit of decomposition may be an element below a morpheme, such as character and byte pair encoding, or a phrase composed of a plurality of words. At this time, processing may be performed to remove stop words such as punctuation marks and auxiliary verbs that frequently appear in sentences, and processing to return morphemes to their original forms.

Next, the graph construction unit 110 extracts the word Ngram from the words of the target paragraph sentence data 503, and stores it in the target document data 500 as paragraph Ngram data 504 (step S303). Here, the word Ngram indicates that a combination of n consecutive words is listed. For example, if n is 2 or more, the morpheme sequence "data", "no", and "classification" will be divided into five characters: "data", "no", "classification", "data no", and "no classification". One word Ngram is extracted. In this embodiment, n is 3 or less. However, n can be set arbitrarily.

In the following description, the word Ngram will be referred to as Ngram.

Next, the graph construction unit 110 determines whether the process has been executed for all paragraph text data 503 included in the target document data 500 (step S304).

If it is determined that there is at least one piece of paragraph text data 503 that has not been processed, the graph construction unit 110 returns to step S301 and executes the same process.

If it is determined that the processing has been performed on all paragraph text data 503 included in the target document data 500, the graph construction unit 110 ends the preprocessing of the document data and proceeds to step S204. The graph construction unit 110 determines whether document data preprocessing has been performed for all document data 500 (step S204).

If it is determined that there is at least one piece of document data 500 for which document data preprocessing has not been performed, the graph construction unit 110 returns to step S202 and executes the same process.

If it is determined that the document data preprocessing has been performed for all of the document data 500, the graph construction unit 110 generates the Ngram dictionary 510 (step S205).

Specifically, the graph construction unit 110 selects 20,000 pieces of document data 500 including the paragraph Ngram data 504 in which an Ngram satisfying the following conditions appears from among all the Ngrams extracted from the document data 500. , and generates an Ngram dictionary 510 from the selected Ngrams. Let _NW be the number of Ngrams registered in the Ngram dictionary 510.

(Condition 1) The number of document data 500 containing paragraph Ngram data 504 where Ngram appears is 5 or more (Condition 2) The ratio of document data 500 containing paragraph Ngram data 504 where Ngram appears is less than 100%

Note that the Ngram dictionary 510 includes an entry composed of an Ngram 511 that is a field that stores an Ngram, and an index 512 that is a field that stores identification information of the Ngram.

Next, the graph construction unit 110 calculates Pointwise Mutual Information (PMI) between Ngrams and generates an Ngram-Ngram matrix 521 (A ^ww ) based on the following equation (1) (step S206).

PMI between Ngrams is an index representing the co-occurrence frequency of two Ngrams, and is calculated as follows.

(Process B1) First, for Ngrams stored in the Ngram dictionary 510, the graph construction unit 110 calculates the frequency w _i of Ngrams whose index 512 is i and the co-occurrence frequency w i of Ngrams whose indexes 512 are i and j _{. , j} to 0.

(Processing B2) The graph construction unit 110 calculates a word Ngram sequence using a word sequence from the r-th word to a predetermined number of words (20 words in this embodiment) in the all-paragraph sentence data 503 of all documents. For each Ngram in the word Ngram sequence, if the Ngram is included in the Ngram dictionary 510, the graph construction unit 110 increments the frequency w _i of the Ngram based on the value i of the index 512 corresponding to the Ngram. For all pairs of Ngrams included in the word Ngram string, if both Ngrams included in the Ngram pair are included in the Ngram dictionary 510, the graph construction unit 110 calculates the value of the index 512 corresponding to each Ngram included in the Ngram pair. The co-occurrence frequency w _i,j of Ngram is incremented based on i and the value j.

(Process B3) The graph construction unit 110 increments r and repeats the above procedure.

(Processing B4) The graph construction unit 110 finally calculates the PMI of Ngram based on the following formula (2).

The element in the i-th row and j-column of the Ngram-Ngram matrix 521 represents the edge 601 from Ngram whose index 512 is j to Ngram whose index 512 is i. Note that the Ngram-Ngram matrix 521 may be generated using a different index other than the PMI, such as the cosine distance of the word distributed representation, the presence or absence of a connection in WordNet, or a combination thereof.

Next, the graph construction unit 110 starts loop processing of the document data 500 (step S207).

Specifically, the graph construction unit 110 selects one target document data 500 from among the plurality of document data 500. The graph construction unit 110 also initializes the paragraph number counter c to 0. Here, the index 501 of the target document data 500 is assumed to be d.

Next, the graph construction unit 110 executes document and paragraph matrix update processing (step S208). In the document and paragraph matrix update process, the process shown in FIG. 4 is executed.

The graph construction unit 110 updates the document-document matrix 522 (A ^dd ) (step S401).

Specifically, the graph construction unit 110 adds a vector with the same number of elements as the total number of documents to the d-th row of the document-document matrix 522. In the added vector, only the d-th element is 1, and all other elements are 0. This 1 corresponds to a self-loop from document d to document d.

Next, the graph construction unit 110 updates the document-paragraph matrix 523 (A ^dp ) (step S402).

Specifically, the graph construction unit 110 adds a vector with the same number of elements as the total number of paragraphs (N _P ) of each document to the d-th row of the document-paragraph matrix 523. In the added vector, the cth to (c+number of paragraphs in the target document - 1)th elements are 1, and all other elements are 0. This 1 represents a side 602 from a paragraph included in document d to document d.

Next, the graph construction unit 110 updates the document-Ngram matrix 524 (A ^dw ) (step S403).

Specifically, the graph construction unit 110 adds the tfidf vector v of the document data 500 to the d-th row of the document-Ngram matrix 524. The i-th element v _i of the tfidf vector v to be added is calculated based on the following equations (3) to (5).

However, ^d w _i represents the frequency of Ngrams whose index 512 is i in the d-th document data 500, and n _i represents the number of document data 500 in which Ngrams whose index 512 is i. A non-zero element of the tfidf vector v represents an edge 603 from Ngram included in the document d to the document d.

Next, the graph construction unit 110 updates the paragraph dictionary 530 (step S404).

Specifically, the graph construction unit 110 adds an entry in which the document index 531 is d and the paragraph index 532 is (N _A +N _Q +c) to the paragraph dictionary 530. The paragraph dictionary 530 is information indicating in which row of the adjacency matrix 520 the paragraph of the d-th document data 500 starts.

Next, the graph construction unit 110 starts loop processing of the paragraph Ngram data 504 of the target document data 500 (step S405).

Specifically, the graph construction unit 110 selects one target paragraph Ngram data 504 from a plurality of paragraph Ngram data 504 of the target document data 500.

Next, the graph construction unit 110 updates the paragraph-paragraph matrix 525 (A ^pp ) (step S406).

Specifically, the graph construction unit 110 adds a vector with the same number of elements as the total value of the paragraphs of each document to the c-th row of the paragraph-paragraph matrix 525. In the added vector, only the c-th element is 1, and all other elements are 0. This 1 corresponds to a self-loop from paragraph to paragraph.

Next, the graph construction unit 110 updates the paragraph-Ngram matrix 526 (A ^pw ) (step S407).

Specifically, the graph construction unit 110 adds the tfidf vector v of the target paragraph Ngram data to the c-th row of the paragraph-Ngram matrix 526. The i-th element v _i of the tfidf vector v to be added can be calculated based on equations (3) to (5) by considering a paragraph to be on the same level as a document. A non-zero element of the tfidf vector v represents an edge 604 from the paragraph belonging to the document to Ngram. After that, the graph construction unit 110 increments c.

Next, the graph construction unit 110 determines whether processing has been performed on all paragraph Ngram data 504 (step S408).

If it is determined that the process has not been executed for at least one paragraph Ngram data 504, the graph construction unit 110 returns to step S405 and executes the same process.

If it is determined that the process has been executed for all paragraph Ngram data 504, the graph construction unit 110 ends the document and paragraph matrix update process, and proceeds to step S209. The graph construction unit 110 determines whether the document and paragraph matrix update processing has been executed for all of the document data 500 (step S209).

If it is determined that there is at least one piece of document data 500 that has not undergone the document and paragraph matrix update process, the graph construction unit 110 returns to step S207 and executes the same process.

When it is determined that the document and paragraph matrix update processing has been executed for all document data 500, the graph construction unit 110 updates the Ngram-Ngram matrix 521 (A ^ww ), the document-document matrix 522 (A ^d-d ), document-paragraph matrix 523 (A ^d-p ), document-Ngram matrix 524 (A ^d-w ), paragraph-paragraph matrix 525 (A ^p-p ), and paragraph-Ngram matrix 526 (A ^p-w ) An adjacency matrix 520(A) is generated using (step S210).

Specifically, the graph construction unit 110 generates the adjacency matrix 520(A) based on equation (6).

Next, the graph construction unit 110 generates a correct label vector 540 with N elements (step S211). Note that N is the total value of _NA , _NQ , _NP , and _NW . The graph construction unit 110 stores an Ngram dictionary 510, an adjacency matrix 520, a paragraph dictionary 530, and a correct label vector 540 in the graph information 111. After that, the graph construction unit 110 ends the process.

The d-th element of the correct label vector 540 is the value of the label 502 of the document data 500 whose index 501 is d, and the other elements are 0. d is 0 or more and smaller than _NA .

In this embodiment, the graph 600 is generated by simultaneously inputting the document data 500 for learning and the document data 500 to be classified, but the present invention is not limited thereto. The graph construction unit 110 may generate a graph 600 from the learning document data 500, and update the graph 600 when the document data 500 to be classified is input.

The above is an explanation of the processing executed by the graph construction unit 110.

The classification model learning unit 112 executes a classification model learning process. The classification model learning unit 112 executes the following processing.

(Process C1) The classification model learning unit 112 applies a graph convolution network (GCN) to the adjacency matrix A and the N-dimensional unit matrix I stored in the graph information 111, as shown in equation (7). Calculate a vector z of length N. Note that equation (7) is calculated using equations (8) to (10). Here, ⁱ W is a learnable parameter, σ is a normalized linear function for each element of the tensor, and ζ represents dropout. However, instead of the two-layer GCN, z may be calculated using an arbitrary layer GCN or an algorithm such as Graph Attention Network (GAT) or GraphSAGE.

(Process C2) The classification model learning unit 112 calculates, for each document data 500, a probability value (class probability value) that the label 502 of the document data 500 is "1". Specifically, the classification model learning unit 112 calculates the probability value that the label 502 of the document data 500 whose index 501 is i is 1 by substituting z calculated from the formula (7) into the formula (11). Calculate ^y _i . i is less than or equal to _NA .

(Process C3) The classification model learning unit 112 calculates the parameters of the learning target so that the label 502 and the probability value ^y _i match. Specifically, the classification model learning unit 112 calculates the cross entropy L defined by equation (12), and calculates the parameters ⁰ W and ¹ W using the steepest descent method so that the cross entropy L is minimized. . y _i represents the i-th element of the correct label vector 540.

(Process C4) The classification model learning unit 112 stores the calculated parameters in the classification model information 113. In this embodiment, the parameters are calculated using the steepest descent method, but the method is not limited thereto. As a parameter estimation method, a quasi-Newton method, evolutionary calculation, Markov chain Monte Carlo method, etc. may be used.

In this embodiment, the classification model learning unit 112 learns the classification model using the graph 600 generated from the document data 500 for learning and the document data 500 to be classified, but the invention is not limited thereto. The classification model learning unit 112 may learn a classification model using the graph 600 generated only from the document data 500 for learning.

The above is an explanation of the processing of the classification model learning unit 112.

The classification unit 114 calculates the class probability value of each vertex of the graph 600. The classification unit 114 executes the following processing.

(Process D1) The classification unit 114 acquires parameters from the classification model information 113. Note that the timing is not limited to the acquisition timing of the parameters. For example, when a parameter is stored in the classification model information 113, the classification unit 114 may acquire the parameter.

(Processing D2) The classification unit 114 calculates a vector z of length N by applying GCN to the adjacency matrix 520 and the N-dimensional unit matrix I stored in the graph information 111, as shown in equation (7). do.

(Process D3) The classification unit 114 calculates a class probability value for each vertex of the graph 600. In this embodiment, the classification unit 114 calculates the probability value ^y _i that the label of the i-th vertex is 1 by substituting the vector z into equation (11).

The above is an explanation of the processing executed by the classification unit 114.

The document reconstruction unit 115 determines the classification of the document based on the adjacency matrix 520 and paragraph dictionary 530 stored in the graph information 111, and the class probability value of each vertex of the graph 600 calculated by the classification unit 114. , construct the document and identify the evidence points.

The processing executed by the document reconstruction unit 115 will be explained using FIGS. 7 and 8. FIG. 7 is a flowchart illustrating an example of processing executed by the document reconstruction unit 115 of the first embodiment. FIG. 8 is a diagram illustrating the structure of output data in the process executed by the document reconstruction unit 115 of the first embodiment.

The document reconstruction unit 115 obtains the adjacency matrix 520 and paragraph dictionary 530 stored in the graph information 111 (step S701).

Next, the document reconstruction unit 115 starts loop processing of the document data 500 (step S702).

Specifically, the document reconstruction unit 115 selects one target document data 500 from among the plurality of document data 500. Here, it is assumed that the index 501 of the target document data 500 is d, and the paragraph index 532 of the entry whose document index 531 of the paragraph dictionary 530 is _d is Pd.

Next, the document reconstruction unit 115 generates reconstructed document data 800 based on the adjacency matrix 520 and the paragraph dictionary 530 (step S703). At this point, the index 801 etc. are empty. At this time, the document reconstruction unit 115 stores d in the index 801 of the reconstructed document data 800.

Next, the document reconstruction unit 115 sets the score 802 of the reconstructed document data 800 (step S704).

Specifically, the document reconstruction unit 115 obtains the class probability value ^ _yd of the vertex corresponding to d, and stores it in the score 802. This corresponds to the class probability value of the target document data 500.

Next, the document reconstruction unit 115 sets the label 803 of the reconstructed document data 800 (step S705).

Specifically, if a value (label) is set in the label 502 of the target document data 500, the document reconstruction unit 115 copies the value to the label 803. If no value (label) is set for the label 502 of the target document data 500, the document reconstruction unit 115 determines the label based on the comparison result between the class probability value ^ _yd of the d-th vertex and the threshold value. , and sets the determined label to label 803. For example, the document reconstruction unit 115 sets the label "1" when the class probability value is greater than or equal to the threshold value, and sets the label "0" when the class probability value is smaller than the threshold value. The threshold value is, for example, 0.5. This corresponds to the predicted value of the classification of the target document data 500. Note that the threshold value is an example, and may be a fixed value other than 0.5, or may be a variable value.

The document reconstruction unit 115 sets the _Pd- th row vector of ⁰ W as the document feature amount 804 as the feature amount of the first layer GCN at _{the Pd-} th vertex (step S706). This corresponds to the feature amount of the target document data 500.

Next, the document reconstruction unit 115 initializes p to 0 (step S707).

Next, the document reconstruction unit 115 converts the paragraph data 805 composed of the paragraph index 806 of the p-th paragraph of the target document data 500, the paragraph score 807, the paragraph feature amount 808, and a plurality of word data 809 into the reconstructed document. It is added to the data 800 (step S708). Specifically, the following processing is executed.

(Processing E1) The document reconstruction unit 115 sets p to the paragraph index 806.

(Process E2) The document reconstruction unit 115 sets the class probability ^y ( _Pd +p) of the vertex corresponding to ( _{P d +p)} to the paragraph score 807. This corresponds to the class probability value of the pth paragraph of the target document data 500.

Here, the target for labeling is the document, and originally, the target for labeling is not the paragraph. However, in the first embodiment, documents and paragraphs are treated as equivalent vertices constituting the graph 600, and class probability values are calculated for paragraphs as well. From step S403 and step S407, since a paragraph is composed of a subset of the Ngram set that makes up the document, a paragraph with a class probability value close to 1 has a higher proportion of the Ngram sets that make up the document. Indicates that a large Ngram set is held. A subset of documents that makes a large contribution to classification can be considered as the basis for classification.

(Process E3) The document reconstruction unit 115 sets the vector of the (P _d +p)th row of ⁰ W to the paragraph feature amount 808 as the feature amount of the first layer GCN at the vertex corresponding to (P _d +p). This corresponds to the feature amount of the pth paragraph of the target document data 500.

(Process E4) The document reconstruction unit 115 inserts the paragraph data 805 into the reconstructed document data 800. Specifically, the following processing is executed.

(Process E4-1) The document reconstruction unit 115 starts loop processing. First, the document reconstruction unit 115 initializes the word number u to 0.

(Process E4-2) The document reconstruction unit 115 initializes the set S to an empty set. The document reconstruction unit 115 converts the u-th to (u+Δu)-th words of the p-th paragraph sentence data 503 of the target document data 500 into Ngrams 511 of the Ngram dictionary 510 while incrementing Δu from 0 to the maximum Ngram size. is matched using the greedy method. The document reconstruction unit 115 adds the index 512 corresponding to the Ngram 511 matched with each Δu to the set S.

(Process E4-3) The document reconstruction unit 115 calculates the value calculated based on equation (13) as a word score. However, |S| represents the number of elements included in the set S. This is equivalent to the average value of class probability values for Ngrams that include the word. Note that the formula for calculating the word score is just an example, and may be a weighted sum, a maximum value, or the like.

(Process E4-4) The document reconstruction unit 115 calculates the vector calculated based on equation (14) as a word feature amount. However, ⁰ W _k is the k-th row vector of the parameter ⁰ W of GCN. This is equivalent to a vector obtained by averaging the feature amounts of Ngrams containing the word for each element. When |S| is 0, the document reconstruction unit 115 calculates a special token NULL indicating that the word feature cannot be defined as the word feature. Note that the formula for calculating the word feature amount is just an example, and may be a weighted sum for each element, a maximum value for each element, or the like.

(Process E4-5) The document reconstruction unit 115 sets u in the word index 810, sets the u-th word of the p-th paragraph sentence data 503 of the target document data 500 in the word text 811, and sets the word score 812 to And the calculation results of equations (13) and (14) are inserted into the word feature amount 813 into the paragraph score 807.

(Process E4-6) The document reconstruction unit 115 increments u.

(Processing E4-7) If u is smaller than the number of words in the p-th paragraph sentence data 503 of the target document data 500, the document reconstruction unit 115 returns to (Processing E4-1). The above is the explanation of the process of step S708.

Next, the document reconstruction unit 115 increments p (step S709).

Next, the document reconstruction unit 115 determines whether the process of step S708 has been executed for all paragraphs (step S710).

Specifically, the document reconstruction unit 115 determines whether (P _d +p) is greater than or equal to P _{d +1} when the value of the paragraph index 532 of the line whose document index 531 is d in the paragraph dictionary 530 is P d. Determine whether If it is determined that (P _d +p) is smaller than P _d+1 , the document reconstruction unit 115 determines that there is at least one paragraph for which the process of step S708 has not been executed.

If it is determined that there is at least one paragraph for which the process of step S708 has not been performed, the document reconstruction unit 115 returns to step S708 and executes the same process.

If it is determined that all the paragraphs have been processed in step S708, the document reconstruction unit 115 determines whether or not the reconstructed document data 800 of all the document data 500 has been generated (step S711).

If there is at least one document data 500 for which the reconstructed document data 800 has not been created, the document reconstruction unit 115 returns to step S702 and executes the same process.

If it is determined that the reconstructed document data 800 of all the document data 500 has been generated, the document reconstruction unit 115 ends the process.

The above is an explanation of the processing executed by the document reconstruction unit 115.

The display unit 116 presents the classification result and the basis location to the user based on the reconstructed document data 800 generated by the document reconstruction unit 115.

Processing executed by the display unit 116 will be described using FIGS. 9 and 10. FIG. 9 is a diagram illustrating an example of a user interface 900 presented by the display unit 116 of the first embodiment. FIG. 10 is a flowchart illustrating an example of processing executed by the display unit 116 of the first embodiment.

The user interface 900 includes an input field 901 that accepts input of document data 500 selected by the user, a display field 902 that displays a list of classifications assigned to the selected document, and a display field 903 that displays the selected document. It consists of In the display field 903, a proof point 904 is highlighted and displayed.

The user interface 900 shown in FIG. 9 is an example, and may have different components, positional relationships of the components, emphasis method for the evidence point 904, display medium, and interface medium. It may also include a reduced display field 905 that displays the entire document in reduced size, and a window 906 that shows the displayed portion of the entire document. The location specified in the window 906 is enlarged and displayed in the display field 903.

The display unit 116 acquires the reconstructed document data 800 generated by the document reconstruction unit 115 (step S1001).

Next, the display unit 116 receives a user input specifying a document to be displayed (step S1002).

Specifically, the display unit 116 accepts input to the input field 901. An index 801 of the reconstructed document data 800 is input into the input field 901 . The value set in the input field 901 is stored in the variable d. Here, the score 802 of the selected reconstructed document data 800 is assumed to be ^D q _d .

Note that the publication number, applicant, search keyword, etc. may be entered in the input field 901. In this case, the display unit 116 searches the reconstructed document data 800 based on the input value and presents the search results to the user. The user selects documents based on the search results.

Next, the display unit 116 starts loop processing of the paragraph data 805 of the selected reconstructed document data 800 (step S1003).

Specifically, the display unit 116 selects one target paragraph data 805 from among the paragraph data 805 of the selected reconstructed document data 800. Here, the paragraph index 806 of the target paragraph data 805 is assumed to be p, and the paragraph score 807 of the target paragraph data 805 is assumed to be ^P q _d,p .

Next, the display unit 116 calculates the background color of the target paragraph data (step S1004).

Specifically, the display unit 116 sets the background color to a hue of 0 degrees, a brightness of l×100%, and a saturation of 100%, based on the value l calculated from equations (15) and (16).

Next, the display unit 116 displays the text of the target paragraph data 805 (step S1005).

Specifically, the display unit 116 draws the word text 811 of each word data 809 included in the target paragraph data 805 in ascending order with respect to the word index 810 in the background color described above.

Next, the display unit 116 determines whether processing has been performed on all paragraph data 805 of the selected reconstructed document data 800 (step S1006).

If it is determined that there is at least one piece of paragraph data 805 that has not been processed, the display unit 116 returns to step S1003 and executes the same process.

If it is determined that the process has been executed for all paragraph data 805, the display unit 116 ends the process.

According to the first embodiment, the computer system can assign a classification result to a document and present to the user the basis of the classification, such as a paragraph.

In the graph 600, documents and constituent elements (paragraphs and words) are treated as one vertex constituting the graph 600, so there is no trade-off between the appropriateness of evidence point presentation and classification accuracy, as in the prior art. It is possible to present the classification results and the basis without causing any problems.

Note that even if a paragraph is not explicitly provided as a vertex, it is possible to obtain the class probability value for each paragraph from the class probability value of Ngrams included in each paragraph as a post-processing.

In the second embodiment, a computer system will be described that classifies documents that include constituent elements (non-text elements) such as figures, tables, and formulas that are different from character strings. The second embodiment will be described below, focusing on the differences from the first embodiment.

The configuration of the computer system of the second embodiment is the same as that of the first embodiment. The hardware configuration and software configuration of the computer 100-2 in the second embodiment are the same as those in the first embodiment. Further, the hardware configuration and software configuration of the computer 100-1 of the second embodiment are the same as those of the first embodiment.

The second embodiment differs from the first embodiment in that the graph construction unit 110 generates a graph 600 whose vertices are paragraphs containing non-text elements. Specifically, the contents of the processes in step S203 and step S208 in FIG. 2 are different.

Processing executed by the graph construction unit 110 of the second embodiment will be described using FIGS. 11 to 13. 11 and 12 are flowcharts illustrating processing executed by the graph construction unit 110 of the second embodiment. 13A and 13B are diagrams illustrating data input/output in processing executed by the graph construction unit 110 of the second embodiment.

First, the process executed in step S203 of the second embodiment will be described using FIG. 11.

The graph construction unit 110 starts loop processing of the paragraph data 1301 of the target document data 500 (step S1101).

Specifically, the graph construction unit 110 selects one target paragraph data 1301 from among the plurality of paragraph data 1301 of the target document data 500.

If the target paragraph data 1301 is a drawing, the graph construction unit 110 applies optical character recognition (OCR) to the drawing to extract text (step S1102). If the target paragraph data 1301 is not a drawing, the process of step S1102 is omitted.

Next, the graph construction unit 110 decomposes the text of the target paragraph data 1301 into units of morphemes (step S1103). The processing in step S1103 is the same as the processing in step S302.

Next, the graph construction unit 110 extracts the word Ngram from the words of the target paragraph data 1301, and stores it in the target document data 500 as the paragraph Ngram data 504 (step S1104). The processing in step S1104 is the same as the processing in step S303.

Next, the graph construction unit 110 determines whether processing has been performed on all paragraph data 1301 included in the target document data 500 (step S1105).

If there is at least one piece of paragraph data 1301 that has not been processed, the graph construction unit 110 returns to step S1101 and executes the same process. If it is determined that all the paragraph data 1301 have been processed, the graph construction unit 110 ends the preprocessing of the document data and proceeds to step S204.

Next, the process executed in step S208 of the second embodiment will be described using FIG. 12.

The graph construction unit 110 updates the document-document matrix 522 (A ^dd ) (step S1201). The processing in step S1201 is the same as the processing in step S401.

Next, the graph construction unit 110 updates the document-paragraph matrix 523 (A ^dp ) (step S1202). The processing in step S1202 is the same as the processing in step S402.

Next, the graph construction unit 110 updates the document-Ngram matrix 524 (A ^dw ) (step S1203). The processing in step S1203 is the same as the processing in step S403.

Next, the graph construction unit 110 updates the paragraph dictionary 530 (step S1204). The processing in step S1204 is the same as the processing in step S404.

Next, the graph construction unit 110 starts loop processing of the paragraph Ngram data 504 of the target document data 500 (step S1205). The processing in step S1205 is the same as the processing in step S405.

Next, the graph construction unit 110 updates the paragraph-paragraph matrix 525 (A ^pp ) (step S1206). Specifically, the following processing is executed.

(Process F1) The graph construction unit 110 adds a zero vector with the same number of elements as the total value of the paragraphs of each document to the cth row of the paragraph-paragraph matrix 525 (A ^pp ).

(Processing F2) The graph construction unit 110 stores 1 in the element in the c row and c column of the paragraph-paragraph matrix 525 (A ^pp ). This 1 corresponds to a self-loop from paragraph to paragraph.

(Processing F3) When the target paragraph Ngram data 504 is derived from the paragraph data 1301 of the image, the graph construction unit 110 constructs the paragraph When the number of the paragraph corresponding to counter c is c', 1 is stored in the element at row c and column c' and the element at row c' and column c of the paragraph-paragraph matrix 525 (A ^pp ). For example, if the figure number is 1, each paragraph corresponding to the figure number of the paragraph data 1301 corresponding to the target paragraph Ngram data 504 can be obtained using the regular expression "Figure 1 [^0-9]". .

(Processing F4) If the target paragraph Ngram data 504 is derived from the image paragraph data 1301, and the paragraph data 1301 is of a type such as a diagram having geometric information, the graph construction unit 110 For each paragraph data 1301 that is similar to the paragraph data 1301 corresponding to the data 504, when the number corresponding to the paragraph counter c of the paragraph corresponding to the image is c', the paragraph-paragraph matrix 525 (A ^pp ), the similarity value is stored in the element in row c and column c' and the element in row c' and column c. The similarity value can be calculated using Bag-of-Visual-Words of the HOG feature amount of the image. Whether or not two images are similar can be determined based on the comparison result between the degree of similarity and the threshold value. The above is the explanation of the processing in step S1206.

Next, the graph construction unit 110 updates the paragraph-Ngram matrix 526 (A ^pw ) (step S1207). The processing in step S1207 is the same as the processing in step S407.

Next, the graph construction unit 110 determines whether processing has been performed on all paragraph Ngram data 504 (step S1208).

If it is determined that there is at least one paragraph Ngram data 504 that has not been processed, the graph construction unit 110 returns to step S1205 and executes the same process.

If it is determined that the process has been executed for all the paragraph Ngram data 504, the graph construction unit 110 ends the process of step S208 and proceeds to step S209.

The processing executed by the classification model learning section 112, the classification section 114, the document reconstruction section 115, and the display section 116 of the second embodiment is the same as that of the first embodiment.

According to the second embodiment, the computer system assigns a classification result to a document that includes non-text elements such as figures, tables, and formulas, and also provides the user with the basis for assigning the classification, such as a paragraph. can be presented.

In the third embodiment, the computer 100-2 presents the user with evidence points that have a different granularity than paragraphs. The third embodiment will be described below, focusing on the differences from the first embodiment.

The configuration of the computer system of the third embodiment is the same as that of the first embodiment. The hardware configuration and software configuration of the computer 100-1 in the third embodiment are the same as those in the first embodiment. The hardware configuration and software configuration of the computer 100-2 in the third embodiment are the same as those in the first embodiment. Further, the processing executed by the classification model learning section 112, classification model information 113, classification section 114, and document reconstruction section 115 of the third embodiment is the same as that of the first embodiment.

The processes executed by the graph construction unit 110, classification model learning unit 112, classification model information 113, classification unit 114, and document reconstruction unit 115 of the third embodiment are the same as those of the first embodiment.

The third embodiment differs in that the display unit 116 presents the evidence to the user in word granularity rather than in paragraphs. Specifically, the processing in step S1004 in FIG. 10 is different.

Processing executed by the display unit 116 of the third embodiment will be described using FIG. 14. FIG. 14 is a diagram illustrating a user interface 1400 in processing executed by the display unit 116 of the third embodiment.

The user interface 1400 in the third embodiment is composed of an input field 901, a display field 902, and a display field 903. In the third embodiment, the granularity of the evidence point 1401 in the display field 903 is different from that in the first embodiment.

The user interface 1400 shown in FIG. 14 is an example, and may have different components, positional relationships of the components, emphasis method for the evidence point 1401, display medium, and interface medium. It may also include a reduced display field 905 that displays the entire document in reduced size, and a window 906 that shows the displayed portion of the entire document. The location specified in the window 906 is enlarged and displayed in the display field 903.

In step S1004 in the third embodiment, the following processing is executed.

(Processing G1) The display unit 116 starts loop processing of the word data 809 of the target paragraph data 805. Specifically, the display unit 116 selects one target word data 809 from among the word data 809 of the target paragraph data 805. Here, the word index 810 of the target word data 809 is assumed to be t, and the word score 812 of the target word data 809 is assumed to be ^W q _{d, p, t} .

(Processing G2) The display unit 116 calculates the background color of the target word data 809. Specifically, based on the value l calculated from equations (17) and (18), the background color is set to a hue of 0 degrees, a brightness of l×100%, and a saturation of 100%.

(Process G3) The display unit 116 determines whether the background color has been calculated for all word data 809 of the target paragraph data 805. If the background color has not been calculated for at least one word data 809, the display unit 116 returns to process G1 and executes the same process.

According to the third embodiment, the computer system can present to the user, along with the classification results, evidence points such as words that have a different granularity than paragraphs.

In the fourth embodiment, the computer 100-2 presents the user with documents or document components that are related to the document or document components, in addition to the classification result and the basis location. The fourth embodiment will be described below, focusing on the differences from the first embodiment.

The configuration of the computer system of the fourth embodiment is the same as that of the first embodiment. The hardware configuration and software configuration of the computer 100-1 in the fourth embodiment are the same as those in the second embodiment. The hardware configuration of the computer 100-2 in the fourth embodiment is the same as that in the first embodiment.

In the fourth embodiment, the software configuration of the computer 100-2 is partially different. FIG. 15 is a diagram showing an example of the configuration of the computer 100-2 according to the fourth embodiment.

The memory 102 of the computer 100-2 of the fourth embodiment includes a related element display section 1501 in addition to programs for realizing the classification model learning section 112, classification model information, classification section 114, document reconstruction section 115, and display section 116. Stores the program that realizes this.

The classification model learning section 112, classification model information, classification section 114, document reconstruction section 115, and display section 116 of the fourth embodiment have the same functions as those of the first embodiment. Furthermore, the classification model information 113 of the fourth embodiment is the same as that of the first embodiment.

The related element display unit 1501 displays documents or document components that are related to the selected document or the selected document components to the user based on the feature amounts of the document components corresponding to the document and the basis location. to be presented.

Processing executed by the related element display unit 1501 of the fourth embodiment will be described using FIGS. 16 to 18. FIG. 16 is a diagram illustrating a user interface 1600 in the process executed by the related element display unit 1501 of the fourth embodiment. FIG. 17 is a flowchart illustrating an example of processing executed by the related element display unit 1501 of the fourth embodiment. FIG. 18 is a diagram illustrating a data structure 1800 of data used in processing executed by the related element display unit 1501 of the fourth embodiment.

The user interface 1600 of the fourth embodiment includes an input field 1601 that accepts input of document data 500 selected by the user, a display field 1602 that displays representative paragraphs of the selected document, and a display field that displays keywords of the selected document. 1603, a display field 1604 that displays representative drawings of the selected document, and a display field 1605 that displays related documents of the selected document.

The user interface 1600 of the fourth embodiment is an example, and may have different components, positional relationships of the components, display medium, and interface medium.

A data structure 1800 of data used in the process executed by the related element display unit 1501 of the fourth embodiment includes a priority queue 1801 of a predetermined length for storing related paragraphs, and a predetermined priority queue for storing related images. It consists of a priority queue 1802 of length, a priority queue 1803 of a predetermined length for storing related words, and a priority queue 1804 of a predetermined length for storing related documents.

The related element display unit 1501 acquires the reconstructed document data 800 generated by the document reconstruction unit 115 (step S1701).

Next, the related element display unit 1501 receives a user input specifying the target reconstructed document data 800 (step S1702).

Specifically, the related element display unit 1501 accepts input of the index 801 of the reconstructed document data 800 into the input field 1601. At this time, the related element display unit 1501 sets the received index 801 to the variable d. Here, it is assumed that the document feature amount 804 of the selected reconstructed document data 800 is ^D e _d . In the following description, the selected reconstructed document data 800 will also be referred to as selected document data 800.

Note that the publication number, applicant, search keyword, etc. may be entered in the input field 1601. In this case, the related element display unit 1501 searches the reconstructed document data 800 based on the input value and presents the search results to the user. The user selects documents based on the search results.

Next, the related element display unit 1501 starts loop processing of the paragraph data 805 of the selected document data 800 (step S1703).

Specifically, the related element display unit 1501 selects one target paragraph data 805 from among the paragraph data 805 of the selected document data 800. Here, the paragraph index 806 of the target paragraph data 805 is assumed to be p, and the paragraph feature amount 808 of the target paragraph data 805 is assumed to be ^P e _d,p .

Next, the related element display unit 1501 calculates the degree of similarity between the selected document data 800 and the target paragraph data 805 (step S1704). Specifically, the following processing is executed.

The related element display unit 1501 uses the document feature amount 804 of the selected document data 800 and the paragraph feature amount 808 of the target paragraph data 805 to calculate the cosine similarity sim( ^D e _d , ^P e _d,p ) is calculated as the degree of similarity.

If the target paragraph data 805 is not an image and the cosine similarity is greater than a predetermined threshold, the related element display unit 1501 stores the target paragraph in the priority queue 1801 that stores related paragraphs with the cosine similarity as the priority. A paragraph index 806 of data 805 is inserted. If the target paragraph data 805 is an image and the cosine similarity is larger than a predetermined threshold, the related element display unit 1501 stores the target paragraph in the priority queue 1802 that stores related images with the cosine similarity as the priority. A paragraph index 806 of data 805 is inserted. The above is the explanation of the process in step S1704.

Next, the related element display unit 1501 starts loop processing of the word data 809 of the target paragraph data 805 (step S1705).

Specifically, the related element display unit 1501 selects one target word data 809 from among the word data 809 of the target paragraph data 805. Here, the word index 810 of the target word data 809 is t, and the word feature amount 813 of the target word data 809 _{is Wet} ^.

Next, the related element display unit 1501 calculates the degree of similarity between the selected document data 800 and the target word data 809 (step S1706). Specifically, the following processing is executed.

The related element display unit 1501 uses the document feature amount 804 of the selected document data 800 and the word feature amount 813 of the target word data 809 to calculate the cosine similarity shown in equation (19) as the similarity.

If the cosine similarity is greater than a predetermined threshold, the related element display unit 1501 inserts the word index 810 of the target word data 809 into the priority queue 1803 that stores related words, with the cosine similarity as the priority. The above is the explanation of the process in step S1706.

Next, the related element display unit 1501 determines whether processing has been performed on all word data 809 of the target paragraph data 805 (step S1707).

If it is determined that there is at least one piece of word data 809 that has not been processed, the related element display unit 1501 returns to step S1705 and executes the same process.

If it is determined that all the word data 809 of the target paragraph data 805 have been processed, the related element display unit 1501 determines whether or not all the paragraph data 805 of the selected document data 800 have been processed ( Step S1708).

If it is determined that there is at least one piece of paragraph data 805 that has not been processed, the related element display unit 1501 returns to step S1703 and executes the same process.

If it is determined that the process has been executed for all paragraph data 805 of the selected document data 800, the related element display unit 1501 starts loop processing of the reconstructed document data 800 excluding the selected document data 800 (step S1709).

Specifically, the related element display unit 1501 selects one target reconstructed document data 800 from among the reconstructed document data 800 excluding the selected document data 800. Here, it is assumed that the index 801 of the target reconstructed document data 800 is d', and the document feature amount 804 of the target reconstructed document data 800 is ^D e _d' .

Next, the related element display unit 1501 calculates the degree of similarity between the selected document data 800 and the target reconstructed document data 800 (step S1710). Specifically, the following processing is executed.

The related element display unit 1501 uses the document feature amount 804 of the selected document data 800 and the document feature amount 804 of the target reconstructed document data 800 to calculate the cosine similarity shown in equation (19) as the similarity.

If the cosine similarity is greater than a predetermined threshold, the related element display unit 1501 inserts the index 801 of the target reconstructed document data 800 into the priority queue 1804 that stores related documents, with the cosine similarity as the priority. The above is the explanation of the process in step S1710.

Next, the related element display unit 1501 displays the user interface 1600 (step S1712).

Specifically, the related element display unit 1501 displays paragraph text data 503 corresponding to each paragraph index 806 stored in the priority queue 1801 in the display column 1602. The related element display unit 1501 displays paragraph data 1301 corresponding to each paragraph index 806 stored in the priority queue 1802 in a display column 1604. The related element display unit 1501 displays word text 811 corresponding to each word index 810 stored in the priority queue 1803 in a display column 1603. Further, the related element display unit 1501 displays each index 801 stored in the priority queue 1804 in a display column 1605.

In the fourth embodiment, a method of presenting documents or document components related to documents or document components to the user based on loop processing was described, but instead of loop processing, an approximate K-neighbor search such as FLANN is used. You may also use the law.

According to the fourth embodiment, the computer system can present to the user documents or document components that are related to the selected document or the components of the selected document.

In the fifth embodiment, the classification model learning unit 112 learns a classification model based on one or more correct labels given to each of the constituent elements of a document and a sentence. Example 5 will be described below, focusing on the differences from Example 1.

The configuration of the computer system of the fifth embodiment is the same as that of the first embodiment. The hardware and software configurations of computers 100-1 and 100-2 in the fifth embodiment are the same as those in the first embodiment.

The processing executed by the classification model learning section 112, the classification section 114, the document reconstruction section 115, and the display section 116 of the fifth embodiment is the same as that of the first embodiment.

The graph construction unit 110 of the fifth embodiment differs in that it accepts not only the correct label given to the document but also one or more correct labels given to the conceptual elements. Specifically, the processes in step S203 and step S211 in FIG. 2 are different.

Here, the correct label given to the component corresponds to a label indicating whether it is a basis part or not. In the following description, a correct label given to a document will be referred to as a document label, and a correct label given to a paragraph, which is one of the constituent elements, will be referred to as a paragraph label.

Processing executed by the graph construction unit 110 of the fifth embodiment will be described using FIG. 3, FIG. 19A, FIG. 19B, and FIG. 20. 19A and 19B are diagrams illustrating data input/output in processing executed by the graph construction unit 110 of the fifth embodiment. FIG. 20 is a flowchart illustrating an example of step S211 executed by the graph construction unit 110 of the fifth embodiment.

In step S203 in the fifth embodiment, the following processing is executed.

The graph construction unit 110 starts loop processing of the paragraph data 1901 of the target document data 500 (step S301).

Specifically, the graph construction unit 110 selects one target paragraph data 1901 from a plurality of paragraph data 1901 included in the target document data 500.

Next, the graph construction unit 110 decomposes the text data 1902 of the target paragraph data 1901 into units of morphemes (step S302).

Note that the unit of division may be a unit smaller than a morpheme, such as character and byte pair encoding, or a phrase composed of a plurality of words. At this time, processing may be performed to remove stop words such as punctuation marks and auxiliary verbs that frequently appear in sentences, and processing to return morphemes to their original forms.

Next, the graph construction unit 110 extracts the word Ngram from the words of the target paragraph data 1901, and stores it in the target document data 500 as the paragraph Ngram data 504 (step S303).

Next, the graph construction unit 110 determines whether processing has been performed on all paragraph data 1901 included in the target document data 500 (step S304).

If it is determined that there is at least one piece of paragraph data 1901 that has not been processed, the graph construction unit 110 returns to step S301 and executes the same process.

If it is determined that all the paragraph data 1901 included in the target document data 500 have been processed, the graph construction unit 110 ends the preprocessing of the document data and proceeds to step S204.

In step S211 in the fifth embodiment, the following processing is executed.

The graph construction unit 110 initializes the correct label vector 540 (step S2001).

Specifically, the graph construction unit 110 sets a vector in which the number of elements is N and all elements are 0 as the correct label vector 540. The graph construction unit 110 also initializes the paragraph number counter c to 0.

Next, the graph construction unit 110 starts loop processing of the document data 500 (step S2002).

Specifically, the graph construction unit 110 selects one target document data 500 from among the plurality of document data 500. Here, the index 501 of the target document data 500 is assumed to be d.

Next, the graph construction unit 110 stores the value of the label 502 of the target document data 500 in the d-th element of the correct label vector 540 (step S2003).

Next, the graph construction unit 110 starts loop processing of the paragraph data 1901 of the target document data 500 (step S2004).

Specifically, the graph construction unit 110 selects one target paragraph data 1901 from among the plurality of paragraph data 1901 of the target document data 500.

Next, the graph construction unit 110 stores the value of the label 1903 of the target paragraph data 1901 in the (N _A +N _Q +p)th element of the correct label vector 540 (Step S2005).

Next, the graph construction unit 110 increments c and determines whether the process has been executed for all paragraph data 1901 of the target document data 500 (step S2006).

If it is determined that there is at least one piece of paragraph data 1901 that has not been processed, the graph construction unit 110 returns to step S2004 and executes the same process.

If it is determined that the process has been executed for all the paragraph data 1901, the graph construction unit 110 determines whether the document labels of all the document data 500 have been stored (step S2007).

If it is determined that there is at least one piece of document data 500 that does not store a document label, the graph construction unit 110 returns to step S2002 and executes the same process.

If it is determined that the document labels of all document data 500 have been stored, the graph construction unit 110 ends the process of step S211.

According to the fifth embodiment, the computer system can learn a classification model based on the correct label given to the document and the correct label given to the component serving as the basis. This makes it possible to improve the accuracy of classification and the accuracy of presentation of evidence points.

Similar to the first embodiment, in the graph 600 generated by the graph construction unit 110, documents and constituent elements (paragraphs and paragraphs) are treated as one vertex that constitutes the graph 600. The classification results and rationale can be presented without creating a trade-off relationship between the appropriateness of the data and classification accuracy.

Further, each of the above-mentioned configurations, functions, processing units, processing means, etc. may be partially or entirely realized in hardware by designing, for example, an integrated circuit. Further, the present invention can also be realized by software program codes that realize the functions of the embodiments. In this case, a storage medium on which a program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored on the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the embodiments described above, and the program code itself and the storage medium storing it constitute the present invention. Examples of storage media for supplying such program codes include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, A non-volatile memory card, ROM, etc. are used.

Further, the program code for realizing the functions described in this embodiment can be implemented in a wide range of program or script languages such as assembler, C/C++, Perl, Shell, PHP, Python, and Java (registered trademark).

Furthermore, by distributing the software program code that realizes the functions of the embodiment via a network, it can be stored in a storage means such as a computer's hard disk or memory, or a storage medium such as a CD-RW or CD-R. Alternatively, a processor included in the computer may read and execute the program code stored in the storage means or the storage medium.

In the above-described embodiments, the control lines and information lines are those considered necessary for explanation, and not all control lines and information lines are necessarily shown in the product. All configurations may be interconnected.

100 Computer 101 Processor 102 Memory 103 Network interface 110 Graph construction unit 111 Graph information 112 Classification model learning unit 113 Classification model information 114 Classification unit 115 Document reconstruction unit 116 Display unit 120 Network 500 Document data 510 Ngram dictionary 520 Adjacency matrix 530 Paragraph dictionary 540 Correct label vector 600 Graph 800 Reconstructed document data 900, 1400, 1600 User interface 1501 Related element display section

Claims (4)

A computer system comprising at least one computer,
The at least one computer has a processor, a memory connected to the processor, and an interface connected to the processor,
The computer system is
a graph construction unit that receives input of document data and generates a graph having the document and the elements of the document as vertices;
a classification unit that calculates, for each of the plurality of vertices, an index used to classify the document into one of the plurality of classes;
classifying the document based on the index of at least one of the vertices, identifying evidence points on the document consisting of at least one element of the document that contributed to the classification; A document reconstruction unit that presents a basis on the document,
The element of the document is at least one of a word, a diagram, a table, a formula, and a sentence composed of multiple words,
The classification unit includes a graph convolution network that receives the graph as input and outputs a vector representing the characteristics of the graph, and a function that uses the vector as a variable and outputs a value representing the probability of each of the plurality of classes. Calculate a value representing the probability of each class as the index using
The document reconstruction unit includes:
determining the class to which the document belongs based on the size of the index of the vertex corresponding to the document;
A computer system that identifies an element of the document that is a basis on the document based on the magnitude of the index of the class to which the document belongs , of the vertex corresponding to the element of the document. The computer system according to claim 1,
The computer system is characterized in that the computer system includes a classification model learning unit that generates the graph convolution network by machine learning using the graph representing a learning document to which a class has been assigned. A document classification method executed by a computer system including at least one computer, the method comprising:
The at least one computer has a processor, a memory connected to the processor, and an interface connected to the processor,
The method of classifying the documents is as follows:
a first step in which the processor receives input of document data, generates data representing the document and a graph having the elements of the document as vertices, and stores the data in the memory;
a second step in which the processor calculates, for each of the plurality of vertices, an index used to classify the document into one of a plurality of classes, and stores the index in the memory;
a third step of the processor classifying the document based on the index of at least one of the vertices and storing the results of the classification in the memory;
a fourth step in which the processor identifies evidence points on the document that are comprised of at least one element of the document that contributed to the classification, and stores the identified results in the memory;
a fifth step in which the processor generates display information for presenting the classification results and the evidence points on the document;
The element of the document is at least one of a word, a diagram, a table, a formula, and a sentence composed of multiple words,
In the second step, the processor inputs the graph and outputs a graph convolution network that outputs a vector representing the characteristics of the graph, and uses the vector as a variable and outputs a value representing the probability of falling into the class. the step of calculating, for each of the classes, a value representing the probability of falling into the class as the index using a function;
The third step includes the processor classifying the document based on the index of the vertex corresponding to the document,
The fourth step includes the step of the processor identifying a base location on the document based on the magnitude of the index of the class into which the document is classified, of the vertex corresponding to the element of the document. A document classification method characterized by: A method for classifying documents according to claim 3, comprising:
The document analysis method includes the step of the processor generating the graph convolution network by machine learning using the graph representing a learning document to which a class has been assigned. Method.

Images