WO2022130579A1

WO2022130579A1 - Similarity determination program, similarity determination device, and similarity determination method

Info

Publication number: WO2022130579A1
Application number: PCT/JP2020/047219
Authority: WO
Inventors: 伸之片江
Original assignee: 富士通株式会社
Priority date: 2020-12-17
Filing date: 2020-12-17
Publication date: 2022-06-23
Also published as: JPWO2022130579A1

Abstract

This similarity determination program is for causing a computer to execute, with respect to a plurality of first partial-document groups (PX1-PXN) obtained by dividing a first document (11a) to obtain a plurality of first partial-documents (PX) and categorizing the obtained first partial-documents on the basis of a plurality of first groups (CX1-CXN) that are obtained by categorizing a plurality of first unique expressions included in the first partial-documents (PX), a process for: calculating a plurality of first vectors corresponding to the first partial-document groups (PX1-PXN), on the basis of words included in the respective first partial-document groups (PX1-PXN); acquiring a plurality of second vectors respectively corresponding to a plurality of second partial-document groups (PY1-PYM) obtained from a second document (12a); and determining the similarity (Sim) between the first document (11a) and the second document (12a) on the basis of a comparison between the plurality of first and second vectors.

Description

Similarity determination program, similarity determination device, and similarity determination method

The present invention relates to a similarity determination program, a similarity determination device, and a similarity determination method.

By dividing multiple documents into words, calculating the vector expressing the meaning of the word, and calculating the weight of each word, and calculating the document vector of each document based on the vector and weight, the similarity between the documents can be calculated. A method for determining is known.

Japanese Unexamined Patent Publication No. 2006-331245

Since various information is described in the document, if the entire document is vectorized and the similarity between the documents is judged, it is possible to judge that the similarity is low even between partially similar documents. There is sex.

In one aspect, one of the objects of the present invention is to improve the accuracy of determining the degree of similarity between partially similar documents.

In one aspect, the similarity determination program may cause the computer to perform the following processing. The first processing is obtained by classifying the first plurality of partial documents obtained by dividing the first document into the first plurality of named entity included in the first plurality of partial documents. With respect to the first plurality of subdocument groups obtained by classifying based on the plurality of groups of the above, the first plurality of parts are based on the words contained in each of the first plurality of subdocument groups. It may include a process of calculating a first plurality of vectors corresponding to each of the document groups. In addition, the process corresponds to each of the second plurality of sub-document groups obtained by classifying the second plurality of sub-documents obtained by dividing the second document. It may include the process of acquiring the vector of. Further, the process includes a process of determining the degree of similarity between the first document and the second document based on the comparison between the first plurality of vectors and the second plurality of vectors. good.

In one aspect, the present invention can improve the accuracy of determining the degree of similarity between partially similar documents.

It is a figure for demonstrating the similarity determination system which concerns on a comparative example. It is a figure explaining the determination example of the similarity degree by the similarity degree determination system shown in FIG. It is a figure for demonstrating the similarity determination system which concerns on 1st Embodiment. It is a figure for demonstrating an example of the process of a similarity determination system. It is a figure for demonstrating an example of the process of a similarity determination system. It is a block diagram which shows the functional structure example of the server in the similarity determination system which concerns on 1st Embodiment. It is a figure which shows the screen output example by a server. It is a flowchart explaining the operation example of a server. It is a block diagram which shows the hardware (HW) configuration example of the computer which realizes the function of a server. It is a figure for demonstrating the similarity determination system which concerns on 2nd Embodiment. It is a figure for demonstrating an example of the process of a similarity determination system. It is a block diagram which shows the functional structure example of the server in the similarity determination system which concerns on 2nd Embodiment. It is a figure which shows the screen output example by a server. It is a flowchart explaining the operation example of the document DB (Database) construction process of a server. It is a flowchart explaining the operation example of the document search process of a server. It is a figure for demonstrating the similarity determination system which concerns on 3rd Embodiment. It is a figure for demonstrating an example of the process of a similarity determination system. It is a figure for demonstrating an example of the process of a similarity determination system. It is a block diagram which shows the functional structure example of the server in the similarity determination system which concerns on 3rd Embodiment. It is a flowchart explaining the operation example of the document DB construction process of a server. It is a flowchart explaining the operation example of the document search process of a server. It is a block diagram which shows the functional structure example of the server in the similarity determination system which concerns on the 1st modification of 3rd Embodiment. It is a figure which shows the screen output example by a server. It is a flowchart explaining the operation example of the document search process of a server. It is a block diagram which shows the functional structure example of the server in the similarity determination system which concerns on the 2nd modification of 3rd Embodiment. It is a flowchart explaining the operation example of the document DB construction process of a server. It is a flowchart explaining the operation example of the document search process of a server.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and there is no intention of excluding the application of various modifications or techniques not specified below. For example, the present embodiment can be variously modified and implemented without departing from the spirit of the present embodiment. In the drawings used in the following description, the parts with the same reference numerals represent the same or similar parts unless otherwise specified.

[1] First Embodiment [1-1] Comparative Example As described above, when the entire document is vectorized and the similarity between the documents is determined, the similarity is low even between partially similar documents. May be determined.

FIG. 1 is a diagram for explaining the similarity determination system 100 according to a comparative example. As shown in FIG. 1, the similarity determination system 100 has a meaning of a word based on a query 101 requesting determination of similarity of a query document (input document) and a document set 102 including one or more comparison target documents. Calculate the similarity based on the vector.

For example, the similarity determination system 100 extracts words from each of a plurality of documents, that is, the query document included in the query 101 and the comparison target document included in the document set 102, for example, by morphological analysis (process P110).

The similarity determination system 100 statistically calculates the word weights for each of the plurality of documents based on the words obtained in the process P110 (process P120). For example, the similarity determination system 100 may evaluate the importance of a word in a document as a weight by using an evaluation method such as tf-idf (Term Frequency-Inverse Document Frequency).

Further, the similarity determination system 100 executes the process P130 in parallel with or before and after the process P120 and at least a part of the processes. For example, the similarity determination system 100 calculates a word vector for each of a plurality of documents based on the words obtained in the process P110 (process P130). The word vector may be referred to as a word embedding vector or a meaning vector.

For example, the similarity determination system 100 may search a vector database in which a vector expressing the meaning of a word is stored and acquire a word vector.

The similarity determination system 100 calculates a document vector by adding the result of multiplying the word vector acquired in the process P130 and the weight of the word acquired in the process P120 over all the words in the document for each document. do. Then, the similarity determination system 100 calculates the similarity between the document vector of the query document and each document vector of the comparison target document, thereby determining the text similarity between the query document and the comparison target document. Calculate (process P140).

The similarity determination system 100 performs ranking processing based on the calculated text similarity (processing P150), and stores the comparison target document having a high similarity with the query document as the ranking result 103 together with the similarity.

FIG. 2 is a diagram illustrating an example of determination of similarity by the similarity determination system 100 shown in FIG. The example of FIG. 2 shows a case where the similarity is determined for the query document 101a and the comparison target document 102a relating to the lithium ion battery.

By the way, the "document" includes a document including a description of a plurality of elements, and, for example, a document such as a patent document or a paper describing a device, a system, a manufacturing method, etc. having a plurality of components. For example, in the document relating to the lithium ion battery shown in FIG. 2, each of the components of the lithium ion battery such as "positive electrode active material", "negative electrode active material", "binder", "electrolyte", and "electrolyte solution solvent" is provided. Compound names related to the classification (group) of may be mixed and described.

Therefore, even if it is desired to determine the similarity with the comparison target document 102a by focusing on a predetermined element described in the query document 101a, the comparison target is for other elements, in other words, the elements that are not the investigation target. Differences from documents may affect the judgment result of similarity between documents.

For example, as shown on the left side of the paper in FIG. 2, the paragraphs shown in underline and bold in both the query document 101a and the comparison target document 102a are the descriptions relating to the “positive electrode active material”, and the contents of both paragraphs are similar. Suppose.

On the right side of the page of FIG. 2, a two-dimensional semantic vector space that maps the range 101b of the description content of the query document 101a and the range 102b of the description content of the comparison target document 102a is shown.

When the element under investigation is "positive electrode active material", comparing

ranges

101b and 102b, the paragraph description for "positive electrode active material" is similar between documents, while the description for other elements is different between documents. .. In this case, the similarity between the document vector 101c of the query document 101a and the document vector 102c of the comparison target document 102a is calculated as a low value.

In this way, when the entire document is represented by one vector, even if the components described in a part of the document are similar to each other, the document as a whole may be calculated as a value having a low degree of similarity. The accuracy of determining the degree of similarity between partially similar documents may decrease.

Although the semantic vector space is shown in two dimensions in FIG. 2 for convenience, it can actually be a vector of several hundred dimensions. As the number of dimensions of the semantic vector space increases, even if the elements to be investigated are similar between documents, it is possible that the similarity is judged to be low when the descriptions about other elements are different between the documents. Will be higher.

[1-2] Description of the First Embodiment Therefore, the similarity determination system 1 according to the first embodiment acquires a plurality of vectors corresponding to each of the partial documents obtained by dividing the document, and obtains the document. Document similarity is determined based on the comparison of multiple vectors between them.

For example, the similarity determination system 1 may acquire a plurality of sub-document groups by classifying a plurality of sub-documents based on a plurality of groups. Further, the similarity determination system 1 may use the similarity of the subdocument clusters having the highest similarity as the document similarity by comparing the similarity between the subdocument clusters of both documents to be determined.

FIG. 3 is a diagram for explaining the similarity determination system 1 according to the first embodiment, and FIGS. 4 and 5 are diagrams for explaining an example of processing of the similarity determination system 1.

As shown in FIG. 3, the similarity determination system 1 according to the first embodiment includes a query 11 requesting determination of the similarity of a query document (input document) and one or more comparison target documents to be determined. Based on the document set (document group) 12, the similarity based on the meaning vector of the word is calculated.

In the example shown in FIGS. 4 and 5, the similarity determination system 1 determines the similarity between the query document 11a specified by the query 11 and the comparison target document 12a in the document set 12. The query document 11a is an example of the first document, and the comparison target document 12a is an example of the second document.

For example, the similarity determination system 1 extracts words from each of a plurality of documents by, for example, morphological analysis (process P1), as in the comparative example.

The similarity determination system 1 statistically calculates the word weights for each of the plurality of documents based on the words obtained in the process P1 (process P2). For example, the similarity determination system 1 may evaluate the importance of a word in a document as a weight by using an evaluation method such as tf-idf.

Further, the similarity determination system 1 executes the process P3 in parallel with or before and after the process P2 and at least a part of the processes. For example, the similarity determination system 1 calculates a word vector for each of a plurality of documents based on the words obtained in the process P1 (process P3). The word vector may be referred to as a word embedding vector or a meaning vector.

For example, the similarity determination system 1 may search a vector database in which a vector expressing the meaning of a word is stored and acquire a word vector. As an example, the similarity determination system 1 may acquire a word vector corresponding to each of the words obtained in the process P1 based on the trained model.

The similarity determination system 1 divides each of a plurality of documents into a plurality of sub-documents (for example, paragraphs), clusters the plurality of sub-documents based on the named entity included in each sub-document (process P4), and sub-document cluster. To generate. Further, the similarity determination system 1 calculates the partial document vector of each partial document cluster.

The similarity determination system 1 calculates the text similarity between the partial document clusters based on the plurality of partial document vectors of the query document 11a and each of the plurality of partial document vectors of the comparison target document 12a (process P5).

Then, the similarity determination system 1 performs a ranking process of ranking each of the plurality of comparison target documents 12a according to the similarity with the query document 11a based on the text similarity (process P6), and outputs the result 13. do. The result 13 may include a ranking result.

Hereinafter, an example of each of the partial document clustering process (process P4), the text similarity calculation process (process P5), and the ranking process (process P6) will be described.

(Partial document clustering process)
The similarity determination system 1 acquires a plurality of partial documents (partial texts) by dividing the document for each document in the process P4. Sub-documents, in other words, document division units include, for example, sentences, paragraphs, chapters, sections, and the like. Hereinafter, it is assumed that the partial document is a paragraph.

In the examples of FIGS. 4 and 5, the similarity determination system 1 divides the query document 11a (document X), acquires a plurality of paragraphs _PX which are examples of the first plurality of partial documents, and compares them. The document 12a (document Y) is divided to obtain a plurality of paragraphs P _Y which are examples of the second plurality of partial documents. Hereinafter, when paragraphs _PX and P _Y are not distinguished from each other, they are simply referred to as "paragraph P".

The similarity determination system 1 has a plurality of parts by clustering a plurality of paragraphs P based on a named entity list indicating the named entity cluster, for example, compound lists C _X1 to C _XN and _CY1 to _CYM shown in FIG. Acquire the document clusters _PX1 to _PXN and _PY1 to _PYM . In the first embodiment, the named entity is the compound name and the document is a document in the field of chemistry that includes the compound name. As the clustering method, various existing methods such as the shortest distance method may be used.

The compound list C _X1 to C _XN is a list of first multiple named entity contained in the document X, for example, a list of named entity clusters obtained by classifying a plurality of compound names, and is an example of the first plurality of groups. be. For example, the compound lists C _X1 to C _XN classify, in other words, the names of the first plurality of compounds based on the respective positions of the first plurality of compounds and the respective similarity of the first plurality of compounds. It may be acquired by clustering and may be referred to as a first cluster group. N is an integer of 1 or more, and indicates the number of groups included in the document X, in other words, the number of clusters.

The compound lists _CY1 to _CYM are a list of second named entities contained in the document Y, for example, a list of named named entity clusters obtained by classifying a plurality of compound names, and are examples of the second plurality of groups. be. For example, the compound lists _CY1 to _CYM classify, in other words, the names of the second plurality of compounds based on the respective positions of the second plurality of compounds and the respective similarity of the second plurality of compounds. It may be acquired by clustering and may be referred to as a second cluster group. M is an integer of 1 or more, and indicates the number of groups included in the document Y, in other words, the number of clusters.

For example, the similarity determination system 1 may cluster the paragraph P by a clustering process using the degree of agreement between the named entity included in the named entity cluster and the named entity included in the plurality of paragraphs P.

In the example of FIG. 4, the similarity determination system 1 determines the degree of coincidence between each of the compound lists C _X1 to C _XN for each cluster and each of the plurality of paragraphs PX for the document _X according to the following formula (1). Based on this, partial document clusters _PX1 to _PXN are generated. Further, the similarity determination system 1 has a degree of coincidence cos (CPX,) between each of the compound lists _CY1 to _CYN for each cluster and each of the plurality of paragraphs PY for the document _Y according to the following formula (2) _. Sub-document clusters P _Y1 to P _YN are generated based on C _Xa ).

In the above formulas (1) and (2), C _PX is a compound list included in paragraph _PX , a is an integer of 1 to N, and C _X a is a compound list C _X 1 to C for each cluster. _XN . _CP Y is a compound list included in paragraph P _Y , b is an integer of 1 to M, and _CY b is a compound list _CY1 to _CYM for each cluster. cos is a function that calculates the cosine similarity between two elements in parentheses. argmax is a function that extracts the condition (here, cluster) when the element in parentheses is the maximum. According to the above formulas (1) and (2), the cosine similarity between each of the compound names included in paragraph P and each of the compound names in the compound list is maximum, for example, the number of occurrences is the largest. Paragraph P can be assigned to the element (cluster of compounds).

In the example shown in FIG. 5, it is assumed that the compound lists C _X1 to C _X4 and _CY1 to _CY4 for the following four clusters are used with N = M = 4. It is assumed that the compound lists C _X1 to C _XN and _CY1 to _CYM are stored in the similarity determination system 1 in advance.
-Compound list C _X1 and _CY1 :
A cluster having elements (characteristics) of "negative electrode active material".
-Compound list C _X2 and _CY2 :
A cluster having an element (characteristic) of "positive electrode active material".
-Compound list C _X3 and _CY3 :
A cluster with a "binder" element (characteristic).
-Compound list C _X4 and _CY4 :
A cluster having an element (characteristic) of "electrolyte solvent".

In the example of FIG. 5, the similarity determination system 1 classifies paragraphs _PX and P _Y into four clusters (N = M = 4), respectively, and divides the partial document clusters _PX1 to _PX4 and _PY1 to _PY4 . Generate. As a result, paragraphs _PX and P _Y can be classified into sub-document clusters of the following four elements (characteristics) by such clustering.
_-Partial document clusters PX1 and _PY1 :
A paragraph describing "negative electrode active material".
_{-Partial document clusters PX2} and _PY2 :
A paragraph describing "positive electrode active material".
_-Partial document clusters PX3 and _PY3 :
A paragraph describing "binder".
_-Partial document clusters PX4 and _PY4 :
A paragraph describing "electrolyte solvent".

In the examples of FIGS. 4 and 5, the numbers N and M of the partial document clusters of the documents X and Y are assumed to match the numbers N and M of the compound list, but the number N and M are not limited to this. It may be acceptable if they do not match and do not match. For example, the number of partial document clusters may be smaller than N and M.

Then, the similarity determination system 1 calculates a plurality of subdocument vectors corresponding to each of the plurality of subdocument clusters based on the words included in each of the subdocument clusters. For example, the similarity determination system 1 adds the result of multiplying the word vector acquired in the process P3 and the weight of the word acquired in the process P2 over all the words in the subdocument cluster for each subdocument cluster. By doing so, the partial document vector may be calculated.

(Text similarity calculation process)
In the process P5, the similarity determination system 1 is based on the similarity between the partial document vector of the query document 11a and each partial document vector of the comparison target document 12a, in other words, the partial document based on the meaning vector of the word. Calculate the text similarity between clusters. The partial document vector of the query document 11a is an example of the first plurality of vectors, and the partial document vector of the comparison target document 12a is an example of the second plurality of vectors.

For example, the similarity determination system 1 calculates the text similarity, for example, the cosine similarity between the partial document cluster of the query document 11a and the partial document cluster of the comparison target document 12a by the calculation of the following equation (3). good.

In the above equation (3), WP _Xa is a dispersion vector of words included in paragraph _PXa , and WP _Yb is a dispersion vector of words included in paragraph P _Yb .

In the example shown in FIG. 4, the similarity determination system 1 has partial document clusters _PX1 , _PX2 , _PX3 , ... _PXN , and partial document clusters _XY1 , _PHY2 , _PHY3 , ... _PHYM . The text similarity may be calculated according to the above equation (3) for all pairs of and.

(Ranking process)
The similarity determination system 1 performs a ranking process in the process P6 to rank each of the plurality of comparison target documents 12a according to the similarity with the query document 11a based on the text similarity, and outputs the result 13.

For example, the similarity determination system 1 outputs rankings of a plurality of comparison target documents 12a according to the similarity with the query document 11a based on the text similarity in the ranking process.

The similarity determination system 1 may calculate the document similarity Sim (X, Y) between the document X and one comparison target document Y, for example, according to the following equation (4).

In the above equation (4), ft is the text similarity according to the above equation (3), and max is a function that adopts the maximum value among all the combinations in parentheses.

The similarity determination system 1 is a pair of partial document clusters (a = 1 to N, b = 1) having the maximum value among the text similarity calculated by the above equation (3) according to the above equation (4). Any combination of ~ M) may be adopted as the document similarity Sim (X, Y) between the documents X and Y.

In the example of FIG. 5, the similarity determination system 1 determines that the pair of compound lists C _X2 and _CY2 , in other words, the text similarity between the partial document clusters of the “positive electrode active material” is the maximum, and the relevant The text similarity is determined as the document similarity Sim (X, Y) between the documents X and Y.

The above equation (4) shows an example of calculating the document similarity between the document X (query document 11a) and one document Y (comparison target document 12a). The similarity determination system 1 performs the above processing for each of a plurality of comparison target documents 12a, for example, documents Y ₁ to Y _L (L is an integer of 2 or more and the number of documents of comparison target document 12a), and the number of documents Y. Document similarity Sim (X, Y ₁ ) to Sim (X, Y _L ) according to the above may be acquired.

Then, the similarity determination system 1 sorts all the documents Y ₁ to Y _L to be searched in descending order from the documents Y having the highest document similarity Sim (X, Y ₁ ) to Sim (X, Y _L ), for example. The sort result may be output as the result 13. The result 13 may include the identification information of the document Y together with the rank (rank), and may include the document similarity Sim (X, Y) of each document Y. The identification information of the document Y includes at least one of an identifier such as a document number or a document code, bibliographic information such as a document name, and at least a part of the contents of the document Y such as a summary and a predetermined part. But it may be.

As a result 13, the similarity determination system 1 identifies information of the document Y having the highest document similarity Sim (X, Y) with the document Y determined to have a specific order, for example, the query document 11a. May be output.

As described above, according to the similarity determination system 1 according to the first embodiment, the similarity between documents is partially determined based on the text similarity for each partial document cluster classified by the clustering process. It is possible to improve the accuracy of determining the degree of similarity between similar documents.

For example, the similarity determination system 1 has a high degree of similarity between the documents X and Y because the semantic vectors for the "positive electrode active material" are similar by comparing the partial document vectors between the documents X and Y. , Can be judged. In FIG. 5, for convenience, the semantic vector space is shown in two dimensions, but it can actually be a vector of several hundred dimensions. According to the first embodiment, the accuracy of determining the degree of similarity between partially similar documents can be improved by comparing the partial document clusters.

Here, in the first embodiment, the similarity determination system 1 has been described as calculating a plurality of partial document vectors for both the query document 11a and the comparison target document 12a, but the present invention is limited to this. It's not something.

For example, for any one of the documents, for example, a plurality of comparison target documents 12a, when the similarity determination system 1 stores the document set 12 in advance, a plurality of portions of each of the plurality of comparison target documents 12a. The document vector may be calculated in advance and accumulated.

In this case, the similarity determination system 1 calculates a plurality of partial document vectors for the other document, for example, the query document 11a, and acquires a plurality of partial document vectors to be accumulated for the comparison target document 12a. You can do it. Then, the similarity determination system 1 performs the above-mentioned text similarity calculation process and ranking process based on the calculated plurality of partial document vectors of the query document 11a and the plurality of partial document vectors of the acquired comparison target document 12a. You can do it.

The document in which a plurality of partial document vectors are calculated and accumulated in advance is not limited to the comparison target document 12a, and may be a query document 11a in place of or in addition to the comparison target document 12a.

[1-3] Functional Configuration Example FIG. 6 is a block diagram showing a functional configuration example of the server 2 in the similarity determination system 1 according to the first embodiment, and FIG. 7 is a diagram showing a screen output example by the server 2. Is.

The server 2 is an example of a similarity determination device, an information processing device, or a computer. For example, in the similarity determination system 1, the server 2 performs various communications such as reception of the query document 11a and the comparison target document 12a and transmission of the result 13 with a terminal device (not shown), another server, or the like. good.

The server 2 may provide, for example, a function for enabling access to the terminal device. Examples of the function include generation and display control of a screen such as a web page used for access by a terminal device. For example, the terminal device sends an access request to the server 2 using an application such as a browser, and accesses the server 2 via a web page displayed on the application based on the screen information received from the server 2. good. For example, as shown in FIG. 7, the server 2 may output the screen information of the query specification screen 210 for designating the query and the determination result output screen 240 for outputting the determination result.

The above-mentioned similarity determination process by the similarity determination system 1 may be realized by the server 2. As shown in FIG. 6, the server 2 may optionally include a memory unit 21, a document input unit 22, a similarity calculation unit 23, and a similarity output unit 24. The memory unit 21, the document input unit 22, the similarity calculation unit 23, and the similarity output unit 24 are examples of control units.

The memory unit 21 has a storage area for storing various data related to the similarity determination process. The memory unit 21 may store information such as the query document 11a shown in FIG. 3, a plurality of comparison target documents 12a, the result 13, and a compound list for each cluster preclassified for each document. Further, the memory unit 21 may store information such as paragraph P, partial document cluster, text similarity, document similarity Sim, etc. for each document shown in FIGS. 4 and 5 as intermediate data in the similarity determination process. ..

The document input unit 22 may receive input of the query document 11a and the comparison target document 12a from a computer such as a terminal device (not shown) or another server, and store the query document 11a and the comparison target document 12a in the memory unit 21, for example, as a DB (Database). In this way, the document input unit 22 may be able to construct and refer to the DB of the document.

Further, the document input unit 22 may receive the input of the query document 11a related to the similarity determination request from a computer such as a terminal device (not shown) or another server and store it in the memory unit 21. The query document 11a may be included in the query 11, for example.

The document input unit 22 may accept, for example, as the query 11, not the query document 11a itself, but the identification information of the query document 11a, for example, information such as a document number and a document code. In this case, the document input unit 22 may specify the query document 11a related to the similarity determination request from, for example, the DB of the memory unit 21 based on the identification information.

For example, as shown in FIG. 7, the document input unit 22 may accept the document number set in the input field 211 when the determination button 212 of the query specification screen 210 is pressed.

The similarity calculation unit 23 calculates the similarity between the query document 11a and the comparison target document 12a. As illustrated in FIG. 6, the similarity calculation unit 23 may include a document division unit 231, a partial document clustering unit 232, and a document similarity calculation unit 233.

The document division unit 231 divides each of the query document 11a and the comparison target document 12a stored in the memory unit 21 to generate partial documents, for example, paragraphs _PX and _PY .

The partial document clustering unit 232 clusters each of the plurality of paragraphs _PX and the plurality of paragraphs P _Y based on the compound lists C _X1 to C _XN and _CY1 to _CYM stored in the memory unit 21, and the partial document cluster P. Acquire _X1 to P _XN and P _Y1 to P _YM . Further, the partial document clustering unit 232 is based on the results of morphological analysis, word weight calculation, and word vector calculation for each of the documents X and Y for each of the partial document clusters _PX1 to _PXN and _PY1 to _PYM . , Calculate the partial document vector.

The processing of the document division unit 231 and the partial document clustering unit 232 is an example of the processes P1 to P4 in FIG.

The document similarity calculation unit 233 calculates the text similarity for each partial document based on the partial document vector for each partial document cluster, and determines the text similarity of the cluster having the highest similarity in the document as the similarity of the document. Calculated as Sim (X, Y). When a plurality of comparison target documents 12a (for example, L) exist, the document similarity calculation unit 233 calculates the similarity Sim (X, Y ₁ ) to Sim (X, Y _L ) for each comparison target document 12a. You can do it. The document similarity calculation unit 233 may store the calculated similarity Sim (X, Y) in the memory unit 21.

The similarity output unit 24 outputs the similarity Sim (X, Y) calculated by the similarity calculation unit 23. When there are a plurality (for example, L) of the documents to be compared in the similarity output unit 24, the documents to be compared are compared in descending order of the calculated similarity Sim (X, Y ₁ ) to Sim (X, Y _L ). Information on 12a and the similarity Sim (X, Y) may be output.

The processing of the document similarity calculation unit 233 and the similarity output unit 24 is an example of the processes P5 and P6 of FIG. The output by the similarity output unit 24 may include, for example, transmission to a computer such as a terminal device (not shown), storage in a storage area of a server 2 such as a memory unit 21, and the like.

For example, as shown in FIG. 7, the similarity output unit 24 may output the determination result output screen 240. The determination result output screen 240 may include a display area 241 of the query document 11a and display areas 245a to 245c of at least one (three in FIG. 7) of the comparison target document 12a. The display area 241 may include a display area 242 such as bibliographic information and a summary, and a full-text reference button 243 for transitioning to a screen for displaying the full text of the query document 11a.

The display areas 245a to 245c may include display areas 246a to 246c for bibliographic information and summaries, and full text reference buttons 247a to 247c. In the display areas 245a to 245c, one or more paragraphs PY or compound list corresponding to the partial document cluster determined to be similar, or / and the similarity Sim (X, _Y ) may be displayed. ..

In this way, the similarity output unit 24 can present to the user information about the document determined to have the highest similarity as a result of the similarity calculation between the query document 11a and the comparison target document 12a.

[1-4] Operation Example FIG. 8 is a flowchart illustrating an operation example of the server 2. As shown in FIG. 8, the server 2 may execute the processing for the query document 11a and the processing for the comparison target document 12a at different timings.

As illustrated in FIG. 8, the document input unit 22 accepts the input of the query document 11a (step S1). The document division unit 231 divides the query document 11a into a plurality of subdocuments, for example, a plurality of paragraphs _PX (step S2).

The partial document clustering unit 232 clusters a plurality of paragraphs _PX based on the compound lists C _X1 to C _XN , and acquires the partial document clusters _PX1 to _PXN (step S3). Further, the partial document clustering unit 232 calculates each partial document vector of the partial document clusters PX1 to _PXN based on the weight of each word included in the document _X and the meaning vector of each word (step S4).

Further, the document input unit 22 accepts the input of the comparison target document 12a (step S5). The document division unit 231 selects an unselected comparison target document 12a (step S6), and divides the selected comparison target document 12a into a plurality of partial documents, for example, a plurality of paragraphs _PY (step S7).

The partial document clustering unit 232 clusters a plurality of paragraphs P _Y based on the compound lists _CY1 to _{CYN, and acquires the partial document clusters P Y1} _to P _YM (step S8). Further, the sub-document clustering unit 232 calculates each sub-document vector of the sub-document clusters P _Y1 to P _YM based on the weight of each word included in the document Y and the meaning vector of each word (step S9).

When the compound list and the partial document vector of the comparison target document 12a are acquired in advance and stored in the similarity determination system 1 (for example, the memory unit 21), the processes of steps S7 to S9 may be omitted.

The document similarity calculation unit 233 compares the partial document vectors of the query document 11a and the comparison target document 12a, calculates the similarity Sim between the documents (step S10), and stores it in the memory unit 21 (step S11).

The document similarity calculation unit 233 determines whether or not there is an unselected comparison target document 12a (step S12), and if it determines that there is (YES in step S12), the process proceeds to step S6.

When the document similarity calculation unit 233 determines that there is no unselected comparison target document 12a (NO in step S12), the similarity output unit 24 determines the comparison target document 12a in descending order of similarity Sim (X, Y). And its similarity Sim (X, Y) are output (step S13). Then, the process ends.

[1-5] Hardware Configuration Example The server 2 may be a virtual server (VM; Virtual Machine) or a physical server. Further, the function of the server 2 may be realized by one computer or may be realized by two or more computers. Further, at least a part of the functions of the server 2 may be realized by using the HW (Hardware) resource and the NW (Network) resource provided by the cloud environment.

FIG. 9 is a block diagram showing a hardware (HW) configuration example of the computer 10 that realizes the function of the server 2. When a plurality of computers are used as the HW resource that realizes the function of the server 2, each computer may have the HW configuration illustrated in FIG.

As shown in FIG. 9, the computer 10 has an HW configuration, for example, a processor 10a, a memory 10b, a storage unit 10c, an IF (Interface) unit 10d, an I / O (Input / Output) unit 10e, and a reading unit. It may be provided with 10f.

The processor 10a is an example of an arithmetic processing unit that performs various controls and operations. The processor 10a may be connected to each block in the computer 10 so as to be communicable with each other by the bus 10i. The processor 10a may be a multi-processor including a plurality of processors, a multi-core processor having a plurality of processor cores, or a configuration having a plurality of multi-core processors.

Examples of the processor 10a include integrated circuits (ICs) such as CPUs, MPUs, GPUs, APUs, DSPs, ASICs, and FPGAs. As the processor 10a, two or more combinations of these integrated circuits may be used. CPU is an abbreviation for Central Processing Unit, and MPU is an abbreviation for Micro Processing Unit. GPU is an abbreviation for Graphics Processing Unit, and APU is an abbreviation for Accelerated Processing Unit. DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

The memory 10b is an example of HW that stores information such as various data and programs. Examples of the memory 10b include one or both of a volatile memory such as DRAM (Dynamic Random Access Memory) and a non-volatile memory such as PM (Persistent Memory).

The storage unit 10c is an example of HW that stores information such as various data and programs. Examples of the storage unit 10c include a magnetic disk device such as an HDD (Hard Disk Drive), a semiconductor drive device such as an SSD (Solid State Drive), and various storage devices such as a non-volatile memory. Examples of the non-volatile memory include flash memory, SCM (Storage Class Memory), ROM (Read Only Memory) and the like.

Further, the storage unit 10c may store a program 10g (similarity determination program) that realizes all or a part of various functions of the computer 10. For example, the processor 10a of the server 2 can realize the function as the server 2 illustrated in FIG. 6 by expanding and executing the program 10g stored in the storage unit 10c in the memory 10b.

The memory unit 21 shown in FIG. 6 may be realized by a storage area of one or both of the memory unit 10b and the storage unit 10c.

The IF unit 10d is an example of a communication IF that controls connection and communication with a network. For example, the IF unit 10d may include an adapter compliant with LAN (Local Area Network) such as Ethernet (registered trademark) or optical communication such as FC (Fibre Channel). The adapter may support one or both wireless and wired communication methods. For example, the server 2 may be connected to the terminal device and each of the other servers so as to be able to communicate with each other via the IF unit 10d. Further, for example, the program 10g may be downloaded from the network to the computer 10 via the communication IF and stored in the storage unit 10c.

The I / O unit 10e may include one or both of an input device and an output device. Examples of the input device include a keyboard, a mouse, a touch panel, and the like. Examples of the output device include a monitor, a projector, a printer and the like.

The reading unit 10f is an example of a reader that reads data and program information recorded on the recording medium 10h. The reading unit 10f may include a connection terminal or device to which the recording medium 10h can be connected or inserted. Examples of the reading unit 10f include an adapter compliant with USB (Universal Serial Bus), a drive device for accessing a recording disk, a card reader for accessing a flash memory such as an SD card, and the like. The program 10g may be stored in the recording medium 10h, or the reading unit 10f may read the program 10g from the recording medium 10h and store it in the storage unit 10c.

Examples of the recording medium 10h include non-temporary computer-readable recording media such as magnetic / optical disks and flash memories. Examples of the magnetic / optical disk include flexible discs, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray discs, HVDs (Holographic Versatile Discs), and the like. Examples of the flash memory include semiconductor memories such as USB memory and SD card.

The above-mentioned HW configuration of the computer 10 is an example. Therefore, the increase / decrease of HW (for example, addition or deletion of arbitrary blocks), division, integration in any combination, addition or deletion of buses, etc. may be appropriately performed in the computer 10. For example, in the server 2, at least one of the I / O unit 10e and the reading unit 10f may be omitted.

[2] Second Embodiment [2-1] Description of the Second Embodiment Next, the second embodiment will be described. In the first embodiment, it has been described that the similarity determination system 1 stores the compound lists C _X1 to C _XN and _CY1 to _CYM for each cluster in advance. In the second embodiment, the method in which the similarity determination system 1A calculates the compound lists C _X1 to C _XN and _CY1 to _CYM will be described.

In the following description of the second embodiment, the configuration, processing or function not particularly mentioned shall be the same as the configuration, processing or function according to the first embodiment described above.

FIG. 10 is a diagram for explaining the similarity determination system 1A according to the second embodiment, and FIG. 11 is a diagram for explaining an example of processing of the similarity determination system 1A. As shown in FIG. 10, in the similarity determination system 1A according to the second embodiment, the processes P1 to P6 based on the query 11 and the document set 12 are the same as those in the first embodiment.

As illustrated in FIG. 10, in the similarity determination system 1A, the processes P7 and P8 may be executed in parallel with or before and after the processes P1 to P3 and at least a part of the processes. Hereinafter, the processes P7 and P8 will be described.

For example, the similarity determination system 1A extracts a compound name as an example of a unique expression from each of a plurality of documents, for example, a query document 11a and a plurality of comparison target documents 12a (process P7), and the unique expression is used for each document. Generate a list, eg a compound list.

In the example of FIG. 11, the similarity determination system 1A extracts the compound name from the query document 11a (denoted as “document X”) included in the query 11 and generates the compound list _CX . Further, the similarity determination system 1A extracts a compound name from the comparison target document 12a (denoted as “document Y”) included in the document set 12 to generate a compound list _CY .

In the second embodiment, as in the first embodiment, the query document 11a and the comparison target document 12a are documents relating to the lithium ion battery. Hereinafter, when the compound lists C _X and _CY generated for the set of documents to be determined are not distinguished from each other, they are simply referred to as “compound list C”.

The similarity determination system 1A according to the second embodiment executes clustering for classifying and grouping named entity based on the named entity list (process P8). As the clustering method, various existing methods such as the shortest distance method may be used.

For example, the similarity determination system 1A may calculate the similarity score S between named entities included in the named entity list for each pair (set) of named entity based on the named entity list. For example, the similarity determination system 1A calculates a similarity score S for a pair of named entity based on each position of the named entity and the similarity between the named entity.

As an example, when the pair of named entity is expressed as compounds x ₁ and x ₂ , the similarity determination system 1A may calculate the similarity score S (x ₁ , x ₂ ) using the following formula (5). ..

In the above equation (5), TC (x ₁ , x ₂ ) is the Tanimoto coefficient of MACCS Key. MACCS Key is one of the expression methods (compound descriptors) of the characteristics of compounds. The Tanimoto coefficient is one of the indexes showing the structural similarity between compounds using MACCS Key, and is an example of the similarity between named entities when the named entity is a compound name. Further, Distance (x ₁ , x ₂ ) is, for example, a numerical value obtained by quantifying the proximity of each appearance position of the named entity in a document, and is, for example, a value corresponding to the following conditions.
_-When compounds x1 and x2 appear in the same sentence and _are in a parallel relationship: "1.0"
_-When compounds x1 and _x2 appear in the same sentence: "0.8"
_-When compounds x1 and _x2 appear in the same paragraph: "0.5"
・ Other than the above: “0.1”

The similarity determination system 1A applies the above formula (5) to each combination of compound name pairs (x ₁ , x ₂ ) for a plurality of compound names included in the compound list C, and applies each pair (x ₁ , x 2). The similarity score S (x ₁ , x ₂ ) of x ₂ ) may be calculated.

The similarity determination system 1A classifies a plurality of compound names included in the compound list C by applying a method such as the shortest distance method to a plurality of calculated similarity scores S (x ₁ , x ₂ ). By grouping them together, the compound names may be clustered.

In the example of FIG. 11, the similarity determination system 1A divides the compound names in the compound list C _X into N clusters (groups) by clustering to the compound list C _X , and the compound lists C _X 1 to C for each cluster. Generate _XN . Further, the similarity determination system 1A divides the compound names in the compound list _CY into M clusters (groups) by clustering to the compound list _CY , and generates the compound lists _CY1 to _CYM for each cluster. ..

As an example, focus on the compound list used in the process P4 of FIG. In this case, the similarity determination system 1A classifies the compound lists C _X and _CY into four clusters (N = M = 4), respectively, as shown in FIG. 5, and the compound lists C _X1 to C _X4 and _CY1 are used. ~ _CY4 is generated. As a result, the compound lists C _X and _CY can be classified into clusters of the following four elements (characteristics) by such clustering.
-Compound list C _X1 and _CY1 :
A cluster having elements (characteristics) of "negative electrode active material".
-Compound list C _X2 and _CY2 :
A cluster having an element (characteristic) of "positive electrode active material".
-Compound list C _X3 and _CY3 :
A cluster with a "binder" element (characteristic).
-Compound list C _X4 and _CY4 :
A cluster having an element (characteristic) of "electrolyte solvent".

As described above, the similarity determination system 1A according to the second embodiment generates compound lists C _X1 to C _XN and _CY1 to _CYM for each cluster used in partial document clustering (processing P4). Can be done.

Up to this point, the explanation has been made assuming that the Tanimoto coefficient of MACCS Key is used as the structural similarity, but the description is not limited to this. For example, the method for expressing the characteristics of a compound is not limited to MACCS Key, in other words, MACCS fingerprint, and various compound descriptors such as Morgan fingerprint may be adopted. Further, the index indicating the structural similarity between the compounds is not limited to the Tanimoto coefficient, and various coefficients such as the Dice coefficient may be used.

Further, in the above equation (5), the similarity determination system 1A uses the similarity score S (x ₁ , x ₂ ) as a numerical value of the proximity of the appearance position in the document of the named entity and the named entity. The product with the similarity is calculated, but the product is not limited to this.

As an example, the similarity determination system 1A may calculate the similarity score S (x ₁ , x ₂ ) using the following equation (6).

In the above equation (6), W is a weight. As W, for example, a value such as "0.5" may be appropriately defined and set by the user or the like so that each position of the named entity and the similarity between the named entity are considered evenly. Alternatively, W may be set based on a model trained so that the correct answer example is searched higher by machine learning based on the search query and the training data including the correct answer example (correct answer data). ..

For example, compounds that are not similar in chemical structure but are used in the same way in one component (which is likely to be written together in the same sentence) may be underestimated in similarity when the above formula (6) is used. There is sex. On the other hand, as in the above equation (6), the similarity score is calculated based on the weighted sum of the value obtained by quantifying the proximity of the appearance position of the named entity in the document and the similarity of the named entity. The similarity of the compounds can be justified.

As described above, the similarity determination system 1A is based on the respective positions of the first plurality of compound names included in the query document 11a and the respective similarity of the first plurality of compound names. By classifying the compound names, the first cluster group is generated. Further, the similarity determination system 1A is based on the position of each of the second plurality of compound names included in the comparison target document 12a and the similarity of each of the second plurality of compound names. By classifying the names, a second cluster group is generated. The first cluster group is an example of the first plurality of groups, and the second cluster group is an example of the second plurality of groups.

As described above, according to the similarity determination system 1A according to the second embodiment, the same effect as that of the first embodiment can be obtained. Further, since the compound list for each cluster can be generated for each document, it is possible to avoid the user from manually generating the compound list, which is convenient. Further, even when the similarity determination system 1A does not store one or both documents of the query document 11a and the comparison target document 12a, the similarity of the documents can be determined. Such cases include, for example, the case where the document is included in the query 11, or the case where the location of the document (a storage location other than the similarity determination system 1A) is specified by the query 11.

[2-2] Functional Configuration Example FIG. 12 is a block diagram showing a functional configuration example of the server 3 in the similarity determination system 1A according to the second embodiment, and FIG. 13 is a diagram showing a screen output example by the server 3. Is.

The server 3 is an example of a similarity determination device, an information processing device, or a computer. For example, in the similarity determination system 1A, the server 3 performs various communications such as reception of the query document 11a and the comparison target document 12a and transmission of the result 14 with a terminal device (not shown), another server, or the like. good.

Similar to the server 2, the server 3 may provide, for example, a function for enabling access to the terminal device. For example, as shown in FIG. 13, the server 3 may output screen information of a search query specification screen 330 for designating a search query and a search result output screen 340 for outputting search results.

The above-mentioned similarity determination process by the similarity determination system 1A may be realized by the server 3. As shown in FIG. 12, the server 3 may optionally include a document DB unit 31 and a document retrieval unit 32. The document DB unit 31 and the document search unit 32 are examples of control units. The server 3 may include the document input unit 22 shown in FIG.

The document DB unit 31 stores the query document 11a and the comparison target document 12a, and performs a document DB construction process for constructing the document DB.

The document search unit 32 performs a document search process for searching a comparison target document 12a similar to the query document 11a specified in the query 11 based on the information stored in the document DB unit 31 in response to the acceptance of the query 11. The document search process is a process including a similarity determination process, and is an example of use (application example) of the similarity determination process.

(Explanation of document DB section 31)
As shown in FIG. 12, the document DB unit 31 includes, for example, a document storage unit 311, a compound name extraction unit 312, a clustering unit 313, a document cluster vector calculation unit 314, and a document cluster vector storage unit 315. good.

The document storage unit 311 is an example of the memory unit 21 (see FIG. 6) according to the first embodiment, and stores a plurality of documents. The document is a document that can be used as either the query document 11a or the comparison target document 12a. Therefore, it can be said that the document storage unit 311 stores the query document 11a and the document set (document group) 12 including the plurality of comparison target documents 12a that are the targets of the query 11. The document storage unit 311 may store a plurality of documents in advance before receiving the query 11. The document storage unit 311 may store a plurality of documents received by the document input unit 22 according to the first embodiment.

The compound name extraction unit 312 extracts a compound name as an example of a named entity from each of a plurality of documents accumulated by the document storage unit 311 and generates compound lists C _X and _CY for each document. The treatment of the compound name extraction unit 312 is an example of the treatment P7 in FIG.

The clustering unit 313 calculates the similarity score S for each of the compound names included in the compound lists C _X and _CY . Further, the clustering unit 313 classifies the compound names into a plurality of clusters based on the similarity score S, and the compound lists C _X1 , C _X2 , C _X3 , ... C _XN , and the compound lists _CY1 and _CY2 . Generate _CY3 , ... _CYM . The process of the clustering unit 313 is an example of the process P8 of FIG.

The document cluster vector calculation unit 314 is based on the information of the compound cluster from the clustering unit 313 and the weights and word vectors calculated based on the words extracted from each of the plurality of documents accumulated by the document storage unit 311. The document vector for each may be calculated. The process of the document cluster vector calculation unit 314 is an example of at least a part of the processes P1 to P4 and the process P5 in FIG.

The document cluster vector storage unit 315 is an example of the memory unit 21 shown in FIG. 6, and stores the document vector for each partial document cluster calculated by the document cluster vector calculation unit 314.

(Explanation of the document search unit 32)
As shown in FIG. 12, the document search unit 32 may optionally include a search query designation unit 321, a document similarity calculation unit 322, a search result generation unit 323, and a search result output unit 324.

The search query designation unit 321 is an example of the document input unit 22 shown in FIG. 6, and is a query 11 requesting a document search from a computer such as a terminal device (not shown) or another server (hereinafter referred to as “search query 11”). (May be) Accept the input.

For example, as shown in FIG. 13, the search query specification unit 321 may accept the document number of the query document 11a set in the input field 331 when the search button 332 of the search query specification screen 330 is pressed. ..

The document similarity calculation unit 322 is an example of the document similarity calculation unit 233 shown in FIG. The document similarity calculation unit 322 uses the document similarity Sim (X, Y) between the query document 11a specified by the search query 11 and the comparison target document 12a based on the document vector stored in the document cluster vector storage unit 315. ) Is calculated.

For example, the document similarity calculation unit 322 compares a plurality of partial document vectors corresponding to the query document 11a and the comparison target document 12a among the partial document vectors stored in the document cluster vector storage unit 315, and texts. The similarity may be calculated.

Then, the document similarity calculation unit 322 calculates the document similarity Sim (X, Y) based on the text similarity, and sorts the comparison target documents 12a in descending order of the document similarity Sim (X, Y). , The ranking result 14 may be generated. The content and output method of the result 14 are the same as those of the result 13 according to the first embodiment.

The process of the document similarity calculation unit 322 is an example of at least a part of the process P5 in FIG. 10 and the process P6.

The search result generation unit 323 generates a search result for output based on the result 14. For example, the search result generation unit 323 may generate the search result output screen 340 shown in FIG. The search result output screen 340 may replace the determination result 244 in the determination result output screen 240 shown in FIG. 7 with the search result 344.

As shown in FIG. 13, the search result output screen 340 includes a display area 341 of the query document 11a and display areas 345a to 345c of at least one (three in FIG. 13) of the comparison target document 12a. good. The display area 341 may include a display area 342 such as bibliographic information and a summary, and a full-text reference button 343 of the query document 11a.

The display areas 345a to 345c may include display areas 346a to 346c for bibliographic information and summaries, and full text reference buttons 347a to 347c. In the display areas 346a to _346c , one or more paragraphs PY or compound list corresponding to the partial document cluster determined to be similar, and / or the similarity Sim (X, Y) may be displayed. ..

The search result output unit 324 outputs the search result output screen 340 to a computer such as a terminal device or another server (not shown).

[2-3] Operation Example FIG. 14 is a flowchart illustrating an operation example of the document DB construction process of the server 3, and FIG. 15 is a flowchart illustrating an operation example of the document retrieval process of the server 3.

(Operation example of document DB construction process)
As illustrated in FIG. 14, the document storage unit 311 selects an unselected document (step S21) and registers the document in the document DB (step S22).

The compound name extraction unit 312 extracts the compound name from the text of the document (step S23). The clustering unit 313 clusters the extracted compound names (step S24).

The document cluster vector calculation unit 314 divides the document into a plurality of sub-documents (step S25), and clusters a plurality of sub-documents based on the compound cluster generated by the clustering unit 313 (step S26).

The document cluster vector calculation unit 314 calculates the document vector of each partial document cluster (step S27). The document cluster vector storage unit 315 associates the calculated document vector with the document and registers (stores) it in, for example, a document DB or a document cluster vector DB (step S28).

The document storage unit 311 determines whether or not there is an unselected document (step S29), and if it determines that there is an unselected document (YES in step S29), the process proceeds to step S21. When the document storage unit 311 determines that there is no unselected document (NO in step S29), the process ends.

(Operation example of document search processing)
As illustrated in FIG. 15, the search query designation unit 321 accepts the designation of the query document 11a from the search query designation screen 330 (step S31).

The document similarity calculation unit 322 acquires the document vector of the query document 11a from the document cluster vector storage unit 315 (step S32).

The document similarity calculation unit 322 selects an unselected document (step S33), and acquires the document vector of the partial document cluster of the selected document from the document cluster vector storage unit 315 (step S34).

The document similarity calculation unit 322 compares the document vectors of a plurality of partial document clusters between the query document 11a and the selected document, and calculates the document similarity Sim (X, Y) (step S35).

The document similarity calculation unit 322 determines whether or not there is an unselected document (step S36), and if so (YES in step S36), the process proceeds to step S33. When the document similarity calculation unit 322 determines that there is no unselected document (NO in step S36), the document similarity calculation unit 322 extracts a predetermined number of documents in descending order of document similarity (step S37). ..

The search result generation unit 323 generates a search result based on the extracted data, the search result output unit 324 outputs a search result, for example, a search result output screen 340 (step S38), and the process ends.

[3] Third Embodiment [3-1] Description of the Third Embodiment Next, the third embodiment will be described. In the third embodiment, a method of utilizing the named entity similarity based on the compound cluster will be described in the document similarity calculation process according to the second embodiment.

In the following description of the third embodiment, the configurations, processes or functions not particularly mentioned are the same as the configurations, processes or functions according to the first embodiment and the second embodiment described above.

FIG. 16 is a diagram for explaining the similarity determination system 1B according to the third embodiment, and FIGS. 17 and 18 are diagrams for explaining an example of processing of the similarity determination system 1B.

As shown in FIG. 16, the similarity determination system 1B according to the third embodiment replaces the process P6 of the similarity determination system 1A shown in FIG. 10 with the process P10, and adds the process P9 using the result of the process P8. The process P10 is executed using the results of both the processes P5 and P9.

Process P9 is a process of calculating named entity similarity for each cluster, for example, compound similarity for each pair of clusters between documents. The process P10 is a process of ranking each of the plurality of comparison target documents 12a according to the similarity with the query document 11a based on the text similarity and the named entity similarity. Hereinafter, the processes P9 and P10 will be described.

(Example of named entity similarity calculation processing)
In the process P9, the similarity determination system 1B is, for example, a list of a plurality of compounds of the first plurality of clusters generated from the query document 11a and a list of a plurality of compounds of the second plurality of clusters generated from the comparison target document 12a. And may be compared with each other. Then, the similarity determination system 1B performs compound similarity, for example, cosine similarity by the calculation of the following formula (7) for all the cluster pairs between the first plurality of clusters and the second plurality of clusters. The degree may be calculated.

In the above formula (7), i is an index for specifying all the compound names contained in the compound lists C _Xa and _CYb , and C _Xai and _CYbi are i in the compound lists C _Xa and _CYb . The number of appearances of the second compound name is shown. In the above formula (7), the denominator is the square root of the sum of squares of the number of occurrences of the compound of C _Xa and the square root of the sum of squares of the number of appearances of the compound of _CYb , and the molecule is C _Xa . It is the sum of the products of the number of appearances of the common compound with _CYb .

In the example of the compound lists C _X and _CY shown in FIG. 17, the similarity determination system 1 has the compound lists C _X1 , C _X2 , C _X3 , ... C _XN and the compound lists _CY1 , _CY2 , _CY3 . , ... For all pairs (combinations) with _CYM , the compound similarity may be calculated according to the above formula (7).

(Example of ranking processing)
The similarity determination system 1B performs a ranking process of ranking each of the plurality of comparison target documents 12a according to the similarity with the query document 11a based on the text similarity and the named entity similarity (process P10), and the result. 14 is output.

For example, the similarity determination system 1B calculates the similarity in which the text similarity and the named entity similarity are integrated in the ranking process, and based on the similarity, a plurality of comparison targets according to the similarity with the query document 11a. The ranking of the document 12a is output.

The similarity determination system 1B may calculate the document similarity Sim (X, Y) between the document X and one comparison target document Y, for example, according to the following equation (8).

In the above equation (8), fc is a cosine similarity according to the above equation (7), in other words, a named entity similarity.

The above formula (8) shows an example of calculating the document similarity between the document X (query document 11a) and one document Y (comparison target document 12a). Similar to the second embodiment, the similarity determination system 1B may acquire document similarity Sims (X, Y ₁ ) to Sim (X, Y _L ) according to the number of documents Y.

Then, in the similarity determination system 1B, for example, as in the second embodiment, all the documents Y to be searched in descending order from the documents Y having the highest document similarity Sim (X, Y ₁ ) to Sim (X, Y _L ). Ranking processing is performed by sorting ₁ to Y _L. Further, the similarity determination system 1B may output the sort result as the result 14.

The similarity determination system 1B sets the document similarity Sim (X, Y) between the document X and one comparison target document Y as the named entity similarity and the text similarity according to the following equation (9). It may be calculated as a weighted sum.

In the above equation (9), w is a weight. As w, for example, a value such as “0.5” may be appropriately defined and set by the user or the like so that the named entity similarity and the text similarity are considered equally. Alternatively, w may be set based on a model trained so that the correct answer example is searched higher by machine learning based on the search query and the training data including the correct answer example (correct answer data). ..

As described above, according to the similarity determination system 1B according to the third embodiment, the same effects as those of the first and second embodiments can be obtained.

Further, the similarity determination system 1B uses the named entity similarity calculated by the above formula (7) for the calculation of the document similarity Sim (X, Y) according to the above formula (8) or the above formula (9). Can be done. For example, the similarity determination system 1B sets the pair of clusters (a combination of a = 1 to N and b = 1 to M) having the maximum value among the named entity similarity to the document similarity Sim (a combination of any of them). It may be adopted as the value of fc used for the calculation of X, Y).

Here, in the example shown in FIG. 18, it is assumed that the compound lists C _X and _CY are compared with each other. When the element to be investigated is "positive electrode active material", compound names related to "positive electrode active material" such as "LiCoO2" appear in common between documents, while compound names related to other elements differ between documents. Therefore, it may be calculated as a value with a low degree of compound similarity between documents. In this way, when comparing the compound list C for each document, the compound similarity may be calculated as a value having a low similarity even if the elements to be investigated are similar between the documents.

On the other hand, in the similarity determination system 1B, as illustrated in FIG. 18, the compound similarity between the pairs of the compound lists C _X2 and _CY2 , in other words, the clusters of the “positive electrode active material” is the maximum. It can be determined. Then, the similarity determination system 1B can adopt the compound similarity as a value of fc used for calculating the document similarity Sim (X, Y).

As described above, according to the similarity determination system 1B according to the third embodiment, the similarity between documents is partially determined based on the named entity similarity for each cluster classified by the clustering process. It is possible to further improve the determination accuracy of the similarity between the documents.

[3-2] Functional Configuration Example FIG. 19 is a block diagram showing a functional configuration example of the server 4 in the similarity determination system 1B according to the third embodiment. Unless otherwise specified, the server 4 may be the same as the server 3 shown in FIG.

The above-mentioned similarity determination process by the similarity determination system 1B may be realized by the server 4. As shown in FIG. 19, the server 4 may optionally include a document DB unit 41 and a document retrieval unit 42. The document DB unit 41 and the document search unit 42 are examples of control units.

The document DB unit 41 may include a compound cluster storage unit 416 in addition to the configuration of the document DB unit 31 shown in FIG. The document retrieval unit 42 may include a document similarity calculation unit 422 instead of the document similarity calculation unit 322 shown in FIG. 12.

(Explanation of document DB unit 41)
For example, the compound cluster storage unit 416 is an example of the memory unit 21 shown in FIG. 6, and the information of the compound cluster calculated by the clustering unit 313, for example, the compound list C may be stored in association with the document.

(Explanation of the document search unit 42)
The document similarity calculation unit 422 compares the partial document vectors of the query document 11a and the comparison target document 12a stored in the document cluster vector storage unit 315, and calculates the text similarity. Further, the document similarity calculation unit 422 compares the compound lists of the query document 11a and the comparison target document 12a stored in the compound cluster storage unit 416, and calculates the compound similarity.

Then, the document similarity calculation unit 422 calculates the document similarity Sim (X, Y) based on the text similarity and the compound similarity, and generates the result 14 from the document similarity Sim (X, Y). The process of the document similarity calculation unit 422 is an example of the processes P5, P9, and P10 of FIG.

Note that the document retrieval unit 42 according to the third embodiment may output the screen illustrated in FIG.

[3-3] Operation Example FIG. 20 is a flowchart illustrating an operation example of the document DB construction process of the server 4, and FIG. 21 is a flowchart illustrating an operation example of the document retrieval process of the server 4.

(Operation example of document DB construction process)
FIG. 20 shows that step S41 is added between steps S24 and S25 shown in FIG. As illustrated in FIG. 20, the compound cluster storage unit 416 stores the calculated compound cluster information for each document in step S41.

(Operation example of document search processing)
In FIG. 21, step S51 is added between steps S32 and S33 shown in FIG. 15, and step S35 is replaced with steps S52 and S53.

In step S51, the document similarity calculation unit 422 acquires the compound cluster of the query document 11a, for example, the compound list from the compound cluster storage unit 416.

In step S52, the document similarity calculation unit 422 acquires a compound cluster of a document selected from the compound cluster storage unit 416, for example, a compound list.

In step S53, the document similarity calculation unit 422 calculates the document similarity Sim (X, Y) based on the document vector acquired in steps S32 and S34, respectively, and the compound cluster acquired in steps S51 and S52, respectively. ..

[3-4] First Modification Example Next, a first modification of the third embodiment will be described.

(Functional configuration example)
FIG. 22 is a block diagram showing a functional configuration example of the server 5 in the similarity determination system 1C according to the first modification of the third embodiment, and FIG. 23 is a diagram showing a screen output example by the server 5.

The similarity determination system 1C according to the first modification determines the text similarity by comparing the partial document cluster containing a predetermined keyword in the query document 11a with each partial document cluster of the plurality of comparison target documents 12a. calculate.

As shown in FIG. 22, the server 5 may optionally include a document DB unit 41 and a document search unit 52. The document DB unit 41 and the document search unit 52 are examples of control units. The document DB unit 41 is the same as the document DB unit 41 shown in FIG.

The document search unit 52 may include a document similarity calculation unit 522, a keyword input unit 525, and a document cluster identification unit 526 in place of the document similarity calculation unit 422 of the document search unit 42 shown in FIG.

The keyword input unit 525 accepts input of one or more keywords from the user. For example, as shown in FIG. 23, the keyword input unit 525 has a document number of the query document 11a and one or more set in the input fields 531 and 532 when the search button 533 of the search query specification screen 530 is pressed. Notify the document cluster identification unit 526 of the keyword.

The document cluster identification unit 526 refers to the document cluster vector storage unit 315, and is a portion including one or more notified keywords (for example, including a predetermined number of times or more) from a plurality of partial document clusters of the notified query document 11a. Identify the document cluster.

The document similarity calculation unit 522 limits the partial document vector of the query document 11a to be compared with the plurality of partial document vectors of the comparison target document 12a to the document vector of the partial document cluster specified by the document cluster identification unit 526. In other words, the document similarity calculation unit 522 sets the importance (priority) of the specified subdocument cluster to be higher than that of other subdocument clusters. Then, the document similarity calculation unit 522 calculates the text similarity for the specified partial document cluster, and calculates the inter-document similarity based on the text similarity and the compound similarity.

As described above, according to the server 5 according to the first modification, the same effect as that of the first and second embodiments can be obtained. Further, among the plurality of partial document clusters in the query document 11a, the comparison target document 12a can be searched by an appropriate partial document cluster including the keyword intended by the user, and the determination accuracy of the similarity between the documents can be further improved. Can be improved. Further, since the number of partial document clusters used for determining the similarity can be limited, the processing time of the document retrieval process can be shortened. In addition, the user can flexibly specify a cluster including a predetermined keyword, which is highly convenient.

(Operation example of document search processing)
FIG. 24 is a flowchart illustrating an operation example of the document retrieval process of the server 5. In FIG. 24, steps S61 and S62 are added between steps S51 and S33 shown in FIG. 21, and step S53 is replaced with step S63.

As illustrated in FIG. 24, the keyword input unit 525 accepts the designation of the keyword in step S61.

In step S62, the document cluster specifying unit 526 specifies a partial document cluster of the query document 11a that includes the keywords accepted by the keyword input unit 525 by the first threshold value (predetermined number of times) or more.

In step S63, the document similarity calculation unit 522 calculates the text similarity between the specified partial document cluster of the query document 11a and all the partial document clusters of the selected document. Then, the document similarity calculation unit 522 calculates the document similarity Sim (X, Y) based on the calculated text similarity and compound similarity.

[3-5] Second Modified Example Next, a second modified example of the third embodiment will be described.

(Functional configuration example)
FIG. 25 is a block diagram showing a functional configuration example of the server 6 in the similarity determination system 1D according to the second modification of the third embodiment.

The similarity determination system 1D according to the second modification calculates the text similarity by comparing the partial document clusters whose similarity with the text of a predetermined part of the document is equal to or higher than the second threshold value. Further, the similarity determination system 1D calculates the compound similarity by comparing the compound clusters whose degree of agreement with the text of the predetermined portion included in the partial document cluster is equal to or more than the third threshold value.

For a specific type of document for which a predetermined format (document format) is defined, the accuracy of determining the similarity of partially similar documents can be further improved by determining the document similarity based on the description content of the predetermined portion. In some cases.

As an example, when a specific type of document is a patent document, a partial document cluster whose similarity with the text of "(patent) claims" and "abstract" is equal to or more than a threshold value is "(patent) claims". And may be important text (eg, paragraphs) related to the description of the "summary".

Therefore, the similarity determination system 1D specifies a predetermined part of the text such as "(patent) claims" from the document according to the type of the input document. Further, the similarity determination system 1D accumulates only the partial document cluster and the compound cluster related to the text among the clusters calculated from the document. Then, in the similarity determination process, the similarity determination system 1D determines the similarity based on the partial document cluster and the compound cluster related to the text of the predetermined portion according to the type of the designated query document 11a.

As shown in FIG. 25, the server 6 may optionally include a document DB unit 61 and a document search unit 42. The document DB unit 61 and the document search unit 42 are examples of control units. The document search unit 42 is the same as the document search unit 42 shown in FIG.

The document DB unit 61 may include a predetermined document cluster vector storage unit 615 and a predetermined compound cluster storage unit 616 instead of the document cluster vector storage unit 315 and the compound cluster storage unit 416 of the document DB unit 41 shown in FIG. Further, the document DB unit 61 may include a predetermined document structure analysis unit 617.

The predetermined document cluster vector storage unit 615 stores the information of the partial document cluster specified by the predetermined document configuration analysis unit 617, which will be described later, among the partial document clusters calculated by the document cluster vector calculation unit 314, in association with the document.

The predetermined compound cluster storage unit 616 stores the information of the compound cluster specified by the predetermined document structure analysis unit 617, which will be described later, among the compound clusters calculated by the clustering unit 313, in association with the document.

The predetermined document structure analysis unit 617 specifies the text of the predetermined part from the document according to the type of the input document. The "predetermined portion" may be preset according to the type of document, for example, a predetermined document type in which a document structure is defined, such as a patent document, a paper, and various materials.

Further, the predetermined document composition analysis unit 617 identifies the sub-document cluster whose similarity with the specified text is equal to or higher than the second threshold among the sub-document clusters calculated from the input document, and sets the document vector as the predetermined document. It is stored in the cluster vector storage unit 615. For example, the predetermined document structure analysis unit 617 treats the specified text as a partial document (partial document cluster), and texts of the specified text subdocument (partial document cluster) and each of the other subdocument clusters in the document. The similarity may be compared with the second threshold.

Further, the predetermined document composition analysis unit 617 identifies, among the compound clusters calculated from the input document, the compound cluster whose degree of agreement with the compound name included in the specified partial document cluster is equal to or higher than the third threshold value, and determines. It accumulates in the compound cluster storage unit 616. For example, the predetermined document composition analysis unit 617 treats the compound names included in the specified partial document cluster as a compound list for each cluster, and the compound list is similar to each of the compound list for each other cluster in the document. The degree may be compared with the third threshold.

When a document of a type in which a predetermined portion is not set is input, the predetermined document composition analysis unit 617 uses the calculated document vector and compound cluster of the partial document cluster as the predetermined document cluster vector storage unit 615 and the predetermined compound cluster storage unit. It may be accumulated in the unit 616.

The document similarity calculation unit 422 may perform the same operation as the similarity determination system 1B shown in FIG. 19, but when the type of the document related to the query 11 is a predetermined document type in which a "predetermined portion" is set. , The information of each cluster used to calculate the document similarity is limited. That is, the document similarity calculation unit 422 determines the document similarity based on the partial document cluster and the compound cluster related to the text of the predetermined portion according to the type of the input query document 11a.

As described above, according to the server 6 according to the second modification, the same effect as that of the first and second embodiments can be obtained. Further, by setting a "predetermined part" for each document type in advance, important (high priority) partial document clusters and compound clusters according to the document type can be easily identified. Therefore, the accuracy of determining the similarity between documents can be further improved by determining the similarity based on the important partial document cluster and the compound cluster. Further, since the number of partial document clusters and compound clusters used for determining the similarity can be limited, the processing time of the document retrieval process can be shortened.

(Operation example of document DB construction process)
FIG. 26 is a flowchart illustrating an operation example of the document DB construction process of the server 6. FIG. 26 shows that step S28 shown in FIG. 14 is replaced with steps S71 to S75.

As illustrated in FIG. 26, the predetermined document structure analysis unit 617 specifies the text of the predetermined portion in the document in step S71.

In step S72, the predetermined document structure analysis unit 617 identifies a partial document cluster whose similarity with the text of the predetermined portion is equal to or higher than the threshold value in the partial document cluster.

The predetermined document cluster vector storage unit 615 registers the document vector of the specified partial document cluster in step S73.

In step S74, the predetermined document structure analysis unit 617 identifies a compound cluster whose degree of agreement with the compound name included in the specified partial document cluster is equal to or greater than the threshold value.

The predetermined compound cluster storage unit 616 registers the specified compound cluster in step S75.

(Operation example of document search processing)
FIG. 27 is a flowchart illustrating an operation example of the document retrieval process of the server 6. In FIG. 27, steps S32 and S51 shown in FIG. 21 are replaced with S81 and S82, and S34, S52 and S53 are replaced with S83, S84 and S85.

In step S81, the document similarity calculation unit 422 acquires the document vector of the partial document cluster of the query document 11a from the predetermined document cluster vector storage unit 615. When the query document 11a is a predetermined document type, the document similarity calculation unit 422 acquires the document vector of the predetermined partial document cluster.

In step S82, the document similarity calculation unit 422 acquires the compound cluster of the query document 11a, that is, the predetermined compound cluster when the query document 11a is the predetermined document type, from the predetermined compound cluster storage unit 616.

In step S83, the document similarity calculation unit 422 acquires the document vector of the partial document cluster of the selected document from the predetermined document cluster vector storage unit 615. When the selected document has a predetermined document type, the document similarity calculation unit 422 acquires the document vector of the predetermined partial document cluster.

In step S84, the document similarity calculation unit 422 acquires a predetermined compound cluster from the predetermined compound cluster storage unit 616, that is, when the selected document has a predetermined document type.

The document similarity calculation unit 422 calculates the document similarity based on the acquired document vector of the predetermined partial document cluster and the predetermined compound cluster in step S85.

[4] Other Techniques according to the first to third embodiments described above and the first and second modifications of the third embodiment can be modified or modified as follows.

For example, in the above-mentioned first to third embodiments and the first and second modifications of the third embodiment, the case where the compound name is used as a named entity has been described as an example, but the present invention is limited to this. It is not something that is done. As the named entity, various terms that can be the target of the named entity extraction process in natural language processing, such as a gene sequence (genome), may be used.

Further, for example, the functional configurations included in each of the servers 2 to 6 shown in FIGS. 6, 12, 19, 22, and 25 may be merged or divided in any combination. Further, the first to third embodiments and the first and second modifications of the third embodiment may be combined as appropriate. Further, each of the servers 2 to 6 may generate screen information of any of the screens of FIGS. 7, 13, and 23, and may have a functional configuration according to the screen.

Further, for example, the functions of the servers 5 and 6 according to the first and second modifications of the third embodiment shown in FIGS. 22 and 25 may be implemented in combination with each other. Further, the function may be applied to the document similarity determination process based on the text similarity in the

server

2 or 3 according to the first or second embodiment shown in FIG. 6 or FIG.

Further, each of the servers 2 to 6 shown in FIGS. 6, 12, 19, 22, and 25 has a configuration in which a plurality of devices cooperate with each other via a network to realize each processing function. May be good. As an example, the memory unit 21 is a DB server, the

document DB units

31, 41 and 61 are a combination of an application server and a DB server, a document input unit 22, a similarity calculation unit 23, a similarity output unit 24, and a document search unit 32, 42. And 52 may be a combination of an application server and a Web server, and the like. In these cases, the computer, the application server, and the DB server may cooperate with each other via the network to realize each processing function as the servers 2 to 6.

Further, each of the servers 3 to 6 may be provided with the HW configuration of the computer 10 illustrated in FIG.

1, 1A ~ 1D similarity judgment system 10 computer 11 query 11a query document 12 document set 12a

comparison target document

13, 14 result 2 ~ 6 server 21 memory part 22 document input part 23 similarity calculation part 24 similarity output part 231 312 Compound

name extraction unit

232, 313

Clustering unit

233, 322, 422, 522 Document

similarity calculation unit

31, 41, 61 Document DB unit 311 Document storage unit 314 Document cluster vector calculation unit 315 Document cluster

vector storage unit

32, 42, 52 Document search unit 321 Search query specification unit 323 Search result generation unit 324 Search result output unit 416 Compound cluster storage unit 525 Keyword input unit 526 Document cluster identification unit 615 Predetermined document cluster vector storage unit 616 Predetermined compound cluster storage unit 617 Predetermined document configuration Analysis department

Claims

A first plurality of groups obtained by classifying the first plurality of subdocuments obtained by dividing the first document into the first plurality of named entities contained in the first plurality of subdocuments. With respect to the first plurality of sub-document groups obtained by classifying based on the above, each of the first plurality of sub-document groups is based on the words contained in each of the first plurality of sub-document groups. Calculate the first plurality of vectors corresponding to
Obtaining the second plurality of vectors corresponding to each of the second plurality of sub-document groups obtained by classifying the second plurality of sub-documents obtained by dividing the second document.
Based on the comparison between the first plurality of vectors and the second plurality of vectors, the degree of similarity between the first document and the second document is determined.
A similarity determination program that causes a computer to execute processing.
The process of acquiring the second plurality of vectors is a second plurality obtained by classifying the second plurality of subdocuments into the second plurality of proper representations included in the second plurality of subdocuments. With respect to the second plurality of subdocument groups obtained by classifying based on the group of, the second plurality of subdocuments based on the words contained in each of the second plurality of subdocument groups. The process of calculating the second plurality of vectors corresponding to each of the groups is included.
The similarity determination program according to claim 1.
The process of calculating the first plurality of vectors is the process of calculating the first plurality of named vectors by the first named entity included in the first plurality of partial documents and the plurality of named entity included in the first plurality of groups. Including the process of generating the first plurality of partial document groups by the clustering process using the degree of agreement between the two.
The process of calculating the second plurality of vectors is a process of calculating the first plurality of named entities included in the second plurality of partial documents and the plurality of named entity included in the second plurality of groups. A process of generating the second plurality of subdocument groups by a clustering process using the degree of agreement between the two is included.
The similarity determination program according to claim 2.
The process of calculating the first plurality of vectors is based on the respective positions of the first plurality of named entity and the similarity of each of the first named entity. Including the process of generating the first plurality of groups by classifying the expressions.
The process of calculating the second plurality of vectors is based on the respective positions of the second plurality of named entity and the similarity of each of the second plurality of named entity. Including the process of generating the second plurality of groups by classifying the expressions.
The similarity determination program according to claim 2 or 3.
In the process of generating the first plurality of groups, the value obtained by quantifying the proximity of the appearance position in the first document of each of the first plurality of named entity and the first plurality of named entity. Includes clustering with each similarity of representation
In the process of generating the second plurality of groups, the value obtained by quantifying the proximity of the appearance position in the second document of each of the second plurality of named entity and the second plurality of named entity. Includes a clustering process with each similarity of representation.
The similarity determination program according to claim 4.
The process of determining the similarity is the similarity of the vectors in the combination of each of the first plurality of vectors and each of the second plurality of vectors in which the similarity of the vectors is the maximum. Is included in the process of determining the degree of similarity between the first document and the second document.
The similarity determination program according to any one of claims 1 to 5.
The process of determining the similarity is
Comparison of each of the first plurality of vectors with each of the second plurality of vectors,
The first plurality of groups and the second plurality of groups obtained by classifying the second plurality of subdocuments into the second plurality of named entity included in the second plurality of subdocuments. Comparison and
A process of determining the degree of similarity between the first document and the second document based on the above.
The similarity determination program according to any one of claims 1 to 6.
The process of comparing the first plurality of groups with the second plurality of groups is performed in the combination of each of the first plurality of groups and each of the second plurality of groups. The process of acquiring the similarity of the group of the combination having the maximum similarity is included.
The similarity determination program according to claim 7.
The process of determining the similarity is based on a comparison between the sub-document group including the specified keyword in the first plurality of sub-document groups and the second plurality of sub-document groups. The process of determining the similarity between the document 1 and the document 2 is included.
The similarity determination program according to any one of claims 1 to 8.
In the process of determining the similarity, when the first document has a predetermined document type, the partial document group specified based on the named entity included in the predetermined portion of the first document and the second document group. A process of determining the similarity between the first document and the second document based on a comparison with a sub-document group specified based on the named entity contained in the predetermined portion of the document.
The similarity determination program according to any one of claims 1 to 9.
The first document is the document specified in the search query.
The second document is one of a plurality of the second documents included in the document group to be searched by the search query.
The information of the second document determined according to the plurality of similarities between the first document and each of the plurality of second documents is output as the search result of the search query.
Let the computer perform the process,
The similarity determination program according to any one of claims 1 to 10.
The named entity is a compound name.
Each of the similarity of each of the first named entity and the similarity of each of the second named entity is the structural similarity of the compound.
The similarity determination program according to any one of claims 2 to 5.
A first plurality of groups obtained by classifying the first plurality of subdocuments obtained by dividing the first document into the first plurality of named entities contained in the first plurality of subdocuments. With respect to the first plurality of sub-document groups obtained by classifying based on the above, each of the first plurality of sub-document groups is based on the words contained in each of the first plurality of sub-document groups. Calculate the first plurality of vectors corresponding to
Obtaining the second plurality of vectors corresponding to each of the second plurality of sub-document groups obtained by classifying the second plurality of sub-documents obtained by dividing the second document.
A control unit for determining the degree of similarity between the first document and the second document based on the comparison between the first plurality of vectors and the second plurality of vectors is provided.
Similarity determination device.
In the process of acquiring the second plurality of vectors, the control unit obtains the second plurality of subdocuments by classifying the second plurality of named entities included in the second plurality of subdocuments. The second plurality of sub-document groups obtained by classifying based on the second plurality of sub-document groups, based on the words contained in each of the second plurality of sub-document groups. Performs the process of calculating the second plurality of vectors corresponding to each of the plurality of partial document groups of.
The similarity determination device according to claim 13.
The control unit
In the process of calculating the first plurality of vectors, the first plurality of named entity included in the first plurality of partial documents and the plurality of named entity included in the first plurality of groups are used. A process of generating the first plurality of partial document groups is performed by a clustering process using the degree of agreement between the two.
In the process of calculating the second plurality of vectors, the first plurality of named entity included in the second plurality of partial documents and the plurality of named entity included in the second plurality of groups are used. A process of generating the second plurality of partial document groups is performed by a clustering process using the degree of agreement between the two.
The similarity determination device according to claim 14.
The control unit
In the process of calculating the first plurality of vectors, the first plurality of named entities are based on the respective positions of the first named entity and the similarity of each of the first named entity. The process of generating the first plurality of groups by classifying the expressions is performed.
In the process of calculating the second plurality of vectors, the second plurality of named entities are based on the respective positions of the second named entity and the similarity of each of the second named entity. The process of generating the second plurality of groups by classifying the expressions is performed.
The similarity determination device according to claim 14 or 15.
The control unit
In the process of generating the first plurality of groups, the value obtained by quantifying the proximity of the appearance position of each of the first plurality of named entity in the first document and the first plurality of named entity. Perform clustering processing using the similarity of each expression.
In the process of generating the second plurality of groups, the value obtained by quantifying the proximity of the appearance position of each of the second plurality of named entity in the second document and the second plurality of named entity. Perform clustering processing using the similarity of each expression.
The similarity determination device according to claim 16.
In the process of determining the similarity, the control unit determines the similarity of the vector having the maximum similarity among the combinations of each of the first plurality of vectors and each of the second plurality of vectors. A process of determining the similarity between the vectors as the similarity between the first document and the second document is performed.
The similarity determination device according to any one of claims 13 to 17.
The control unit is in the process of determining the similarity.
Comparison of each of the first plurality of vectors with each of the second plurality of vectors,
The first plurality of groups and the second plurality of groups obtained by classifying the second plurality of subdocuments into the second plurality of named entity included in the second plurality of subdocuments. Comparison and
Based on the above, a process of determining the degree of similarity between the first document and the second document is performed.
The similarity determination device according to any one of claims 13 to 18.
A first plurality of groups obtained by classifying the first plurality of subdocuments obtained by dividing the first document into the first plurality of named entities contained in the first plurality of subdocuments. With respect to the first plurality of sub-document groups obtained by classifying based on the above, each of the first plurality of sub-document groups is based on the words contained in each of the first plurality of sub-document groups. Calculate the first plurality of vectors corresponding to
Obtaining the second plurality of vectors corresponding to each of the second plurality of sub-document groups obtained by classifying the second plurality of sub-documents obtained by dividing the second document.
Based on the comparison between the first plurality of vectors and the second plurality of vectors, the degree of similarity between the first document and the second document is determined.
A similarity determination method in which a computer executes processing.