JP2008123111A - Document similarity-deriving device and answer-supporting system using the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a new method for finding similarity between documents. <P>SOLUTION: A similarity between documents is found by using at least either a co-occurrence vector or a sentence type vector in addition to a TF-IDF vector. Accordingly, the similarity, which more reflects the meaning and contents of the documents, can be found. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a document similarity deriving device for obtaining document similarity.

  In recent years, with the spread of the Internet and personal computers, the need for personal computer technical support is increasing as part of after-sales service. In many PC technical support centers, in addition to a conventional call center that mainly provides technical support by telephone, there are many mail call centers that accept inquiries by E-mail via the Internet.

All technical support provided at the mail call center is free support. Many question mails are sent from evening to midnight. It is required to complete within a predetermined time (for example, within 24 hours) from the reception of the inquiry mail to the transmission of the answer. Due to such restrictions, for companies to provide accurate and quick support, costs such as labor costs are becoming enormous. Therefore, there is a strong demand for automation of mail call centers.
In view of this, an answer support apparatus that supports the creation of an answer suitable for an inquiry is disclosed in Japanese Patent Laid-Open No. 2001-273308.

The response support apparatus disclosed in Japanese Patent Application Laid-Open No. 2001-273308 is preliminarily assumed to be support information storage means in which the contents of an inquiry that is assumed in advance and support information that supports the preparation of an answer to the inquiry are stored in association with each other. The response creator information storage means in which the content of the inquiry received and the answer creator information of the answer creator for this inquiry are stored in association with each other, and the support information storage means is retrieved by the content of the input inquiry. Information sending means for sending the provided support information and the contents of the inquiry to the answer creator corresponding to the answer creator information obtained by searching the answer creator information storage means based on the contents. .
JP 2001-273308 A

  According to the answer support device of the background art, an inquiry is forwarded to an appropriate answer creator in response to an inquiry from a consumer, and support information related to the inquiry can be obtained from the answer creator quickly. You can answer.

  However, the answer support device of this background art is for the consumer to specify the answer creator and support information corresponding to the inquiry of the selection format such as the list box and the check box. Has a problem that it cannot respond to the inquiry sentence created by itself. The support information specifically refers to manuals and specifications related to inquiries to consumers.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an answer support system that identifies the type of an inquiry by text and supports an appropriate answer. Another object of the present invention is to provide a new method for obtaining similarity between documents used in this answer support system.

  In response to questions from consumers and other questioners, create a support system in which the system presents a small number of answer candidates, and the respondent selects the best answer from them and finalizes the answers after final review. It is realistic to use. In such a system, processes other than the final review are automatically processed, and a significant cost reduction can be realized. The inventor created the system ACCESS (Automated Call-Centre Service System) with diligent efforts.

  A “question answering database” was constructed using approximately 10,000 of the “question answering” data collected from more than 30,000 final peer-reviewed data collected from actual mail call centers for about three years.

  There are many actual questions that have the same contents or similar contents. Therefore, it is useless to create an answer each time a question from the user comes. For this reason, if a “question response database” is constructed from already answered question response data and can be reused, the cost can be greatly reduced.

  The “questions from the user” include those that do not seem to be “questions”. Some of these “questions” need to be dealt with from a different perspective, rather than being reviewed by the final reviewer.

  Since the question from the user is a free description, there are many mistypes, kana-kanji conversion errors, grammatical imperfections, etc. in the question sentence. It must be processed on the premise of such a phenomenon. However, since they are corrected and corrected in the question-response data that has been finally reviewed, there is a great advantage in searching the question-answer database and making good use of it.

  The question answering database has a structure categorized into two layers. Each category has a tag that characterizes it. Actual question answer data is stored in the lowest category. The reason for categorizing into two layers is similar to the idea of using a superordinate concept in a so-called thesaurus when there is no good match in the lowest layer. Although two layers are shown here, three or more layers may be used.

  The question answering database is reused by estimating which question category the question belongs to from an inquiry mail sent from a personal computer user by statistical processing or the like. By presenting the answer candidates created using the category answer sentence for each estimated question category to the final reviewer, the answer creation is made more efficient.

  In order to verify the accuracy of question category estimation, a system was created and an evaluation experiment was conducted. When an experiment was performed using 145 question categories, the correct question category was estimated within the top three at a rate of 86%. I tried to build a “question answering database” using about 10,000 of the 30,000 “question answering” data stored, so the remaining 20,000 “question answering” data By adding to the “question answering database”, the accuracy of finding the correct question category can be greatly improved.

(1) Similarity between documents (see FIGS. 1 and 2)
The document similarity deriving device according to the present invention includes a means for performing morphological analysis on a sentence of a document consisting of sentences, and a TF / ID having a weight based on TF / IDF of an index word appearing in the document from the morphologically analyzed document as an element. A means for obtaining an IDF vector, a means for obtaining a co-occurrence vector from a document subjected to morphological analysis, and a means for obtaining a co-occurrence vector having the frequency of a prescriptive co-occurrence in a sentence in which the utterance appears in the morphological analysis. And determining a sentence type of a sentence in which an index word appearing in the document appears, and obtaining a sentence type vector having the frequency of each sentence type as an element, and the TF / IDF of the first document First document obtained by obtaining document vector, co-occurrence vector and sentence type vector, obtaining TF / IDF document vector, co-occurrence vector and sentence type vector of the second document The similarity between the first document and the second document is obtained from the TF / IDF document vector, the co-occurrence vector, the sentence type vector, and the TF / IDF document vector, the co-occurrence vector, and the sentence type vector of the second document. .

  As described above, according to the present invention, since similarity between documents is obtained using at least one of a co-occurrence vector and a sentence type vector in addition to a TF-IDF vector, similarity more reflecting the meaning content in the document. There is an effect that can be obtained.

  A document consists of one or more sentences. Therefore, when the first document is composed of one sentence and the second document is composed of a plurality of sentences, the first document is composed of a plurality of sentences, and the second document is composed of one sentence, the first document When both the first document and the second document are composed of a plurality of sentences, the first document and the second document may be composed of one sentence. That is, the document similarity deriving device of the present invention can determine the similarity between sentences, between documents composed of a plurality of sentences, and between sentences composed of sentences-sentences.

  As shown in FIG. 2 (upper part), the co-occurrence vector calculates the frequency of the predicates that co-occur in the sentence in which the word appears in the word that appears in the document, and the co-occurrence vector is calculated using the calculated frequency as an element. Ask. In FIG. 2 (upper part), as an example, it is calculated that the word “aa” found by morphological analysis appears in the document three times for the word “AAA” found by morphological analysis.

  Similarly, FIG. 2 (lower part) also illustrates a sentence type vector. The sentence type of a sentence in which an index word appearing in the document appears is determined, and the sentence type vector is obtained using the frequency of each sentence type as an element. Yes. In FIG. 2 (lower part), as an example, it is calculated that the sentence type “QUESTION” appears in the document three times through the sentence type determination process for the nomenclature “AAA” found by morphological analysis.

(2) Document group and document similarity (see FIG. 3)
A document-document group similarity deriving device according to the present invention includes each means of the document similarity deriving device, and a document vector that is a TF-IDF vector, a co-occurrence vector, and / or a sentence type vector for the first document. The obtained document vector of each document of the second document group consisting of a plurality of documents is obtained, the average document vector is obtained from the document vector of each document of the obtained second document group, and the obtained average document of the second document group The similarity between the first document and the second document group is obtained from the vector and the document vector of the first document.
As described above, the present invention has an effect that not only the similarity between documents but also the similarity between a document group and a document can be obtained.

(3) Identification of document having high similarity The high similarity document identification device according to the present invention includes each means of the document similarity derivation device, and includes a TF-IDF vector, a co-occurrence vector, and / or a sentence type vector. Is obtained for the comparison target document to be compared, the document vector of the index TF-IDF of the nth document, the co-occurrence vector, and the sentence type vector are obtained, and the document vector of the comparison target document and the nth document The similarity between the comparison target document and the nth document is obtained from the document vector, and n is 1 to N, and the nth document having a high similarity is identified from the similarities between each nth document and the comparison target document. To do.
As described above, in the present invention, the similarity between the plurality of documents and the comparison target document is obtained, and the document having high similarity is specified. Therefore, it is possible to obtain a document that is very similar to the content of the comparison target document. Have.

(4) Identification of document group having high similarity The high similarity document group identification device according to the present invention includes each means of the document similarity deriving device, and includes a TF-IDF vector, a co-occurrence vector, and / or a sentence. A document vector that is a type vector is obtained for a comparison target document, a document vector of each document in the nth document group composed of a plurality of documents is obtained, and an average document vector is obtained from the document vectors of each document in the obtained nth document group. The similarity between the first document and the nth document group is obtained from the obtained average document vector of the nth document group and the document vector of the first document, and n is 1 to N, and each nth document The nth document group having a high similarity is identified from the similarities between the group and the comparison target document.

(5) Response support system (see Fig. 4)
The answer support system according to the present invention includes the high similarity document group specifying device, wherein each nth document group includes similar question sentences, the comparison target document is also a question sentence, and each nth document group Answer sentences corresponding to the contents of the questions are recorded in advance, and the answer sentence associated with the nth document group determined to be highly similar by the high similarity document group specifying device is output.

  In the embodiment described later, this answer support system is shown as a specific example. In particular, in the embodiment, each n-th document group is databased in a tree structure having a plurality of layers. In addition, the high similarity document group specifying device, in this way, the nth document group having the highest similarity to which the comparison target document should belong so that the documents in each nth document group have high similarity to each other. Can also be used to specify a question response database construction support system which will be described later.

  FIG. 4 is a diagram showing the principle of the invention of the answer support system. There are a first document group to an Nth document group, and an average document vector is obtained and recorded in advance for each document group, and a common answer sentence is obtained for each document group. Then, the document vector of the target document is obtained, the similarity between the target document and each document is obtained from the document vector of the target document and the average document vector of each recorded document group, and the nth document having the highest similarity The group is specified, and the answer sentence of the nth document group is output to the user as the optimum answer sentence. Note that the document that is the current target document is also distributed to the nth document group, and the average document vector of the nth document group is obtained again including the newly distributed document. Each time a new document is allocated, or when a predetermined number of documents are accumulated, an average document vector may be obtained every predetermined period. Similarly, when a predetermined number of documents are accumulated, new distribution may be executed every predetermined period.

The present invention has been grasped as an apparatus or a system so far, but can be grasped as a program or a method as will be apparent to those skilled in the art.
These outlines of the invention do not enumerate the features essential to the present invention, and a sub-combination of these features can also be an invention.

[1. System overview]
When the system receives a question mail from the user, the system automatically starts processing. As a result of the processing, that is, the answer candidate is formed into a response mail to the questioner and waits for a review by the final reviewer.

For the analysis of the question sentence, only the morphological analysis is performed, and the result is compared with the language database and the knowledge database to prepare for the question answering database search, and the search is automatically performed. A language database is like a dictionary with linguistic features, and a knowledge database is like a dictionary about personal computers.
The reason why only the morphological analysis is used as the analysis of the question sentence is due to the reliability such as the dependency structure analysis.

  The final reviewer's peer-reviewed question answer data is used to update the question answering database. That is, the question answering database is updated with the question answering data triggered by a reply of the peer-ready answer sentence.

  An example of the system screen is shown in FIG. Several operational functions are provided to ensure that the final reviewer can conduct efficient review. When the final reviewer determines that any of several answer candidates is valid and presses a predetermined button, the selected answer candidate is automatically returned to the questioner by e-mail. When fine processing is necessary, it is processed on this screen, and it is automatically returned as an answer mail to the questioner. In other words, the final reviewer can appropriately modify the sentence of the selected answer candidate and send an e-mail.

  The configuration of the system screen of FIG. 5 is as illustrated in FIG. When the question text is input to the question text input form and the user presses the category determination button, the similarity between the question text and the question category is calculated. The question categories are displayed in the category list box in the order of the calculated similarity. When the user selects a question category or a question sentence belonging to the question category, the answer sentence is displayed in the text box.

  Here, the input of the question text to the question text input form is performed by pasting text data from the mailer body display, for example. However, it is not limited to this example. The mailer may be provided with the function of the answer support system, or conversely, the answer support system may be provided with the function of the mailer. Furthermore, it is also possible to apply a method for acquiring a question sentence from a questioner by another communication method without using a mail system. For example, HTTP and FTP can be used.

[1.1 Morphological analysis]
Morphological analysis that distinguishes parts of speech by dividing a sentence into morphemes is one of the basic techniques of natural language processing and can be applied as appropriate by those skilled in the art, and will not be described in detail here. Examples of the morphological analysis engine include MeCab, ChaSen, and KAKASI.

[1.2 System configuration]
An answer support system is constructed on the answerer computers 100 and 200 used by the answerer. An answer support program is installed in the respondent computers 100 and 200, and an answer support system is constructed. In this embodiment, the answer support system is constructed by one computer as described above, but it can also be constructed by a client-server type. For example, the client has a configuration in which the client receives a question sentence from the user and transmits it to the server, which is processed by the server and returns a plurality of answer candidates to the client.

  FIG. 7 shows an example of a network configuration to which a computer that has constructed the answer support system of this embodiment belongs. The respondent computer 100, the respondent computer 200, the server 300, the printer (server) 400, and the network device 500 are connected on the LAN so that they can communicate with each other. The network device 500 is also connected to an external network so that other computers can communicate with computers on the LAN. Here, it is assumed that a mail is transmitted from a user who is a questioner, and a mail transmitted from the mail server is transmitted to the respondent computer via the network device 500. The mail distribution processing when there are a plurality of respondent computers is implemented as one function of the system built in the call center and is a well-known and commonly used technique, so it will not be described in detail here.

The respondent computer 100 in which the answer support system is constructed includes, for example, a CPU (Central Processing Unit) 101, a RAM 102, a ROM 103, an external storage device HD (hard disk) 104, and a CD-ROM drive for reading data from a CD-ROM. 105, a mouse 111 and a keyboard 112 as input devices, a display 121 and a speaker 122 as output devices, and a LAN interface 131 for connecting to a network.
Although an example of the configuration of the respondent computer 100 has been shown, the respondent computer 200, the server 300, and the user computer 600 have the same configuration.

[2. Question answering database]
The question answering database is also constructed in each of the respondent computers 100 and 200. Here, it can also be constructed as a separate database server so that a plurality of respondent computers can use it in common.

[2.1 Question answering database construction method]
The question response mail data is a pair of an inquiry mail sent from a personal computer user and a peer-reviewed answer sentence.
There are many identical or similar data in the question answering mail data. Therefore, when the contents or meanings of the inquiry mails are the same or similar, and the sentence expressions of those answer sentences are the same or similar, they are regarded as “similar data”.

  In constructing the question answering database for the mail call center, 10135 question answering mail data were actually used. For these, a “question category” was created and “question category classification” of similar data was performed. The question category has a two-layer structure as described above, and the upper layer is called a branch category and the lower layer is called a leaf category. Similar data is classified into the same leaf category, and leaf categories having further correlation are classified into the same branch category.

[2.2 Configuration of question answering database]
The built-in mail call center question answering database has a two-level tree structure. The leaf category may belong to the branch category or may directly belong to the root category. The root category is an upper category of the branch category, but is not regarded as a “layer” because it is a root node of a tree structure. In other words, the whole is not called a three-layer structure.

The branch category has no data, and the leaf category has the same data or similar data. The correlation between the route and each category is as follows:
Root category-> branch category-> leaf category-Root category-> leaf category This outline is shown in FIG.

  In the built-in mail call center question answering database, 83 branch categories and 634 leaf categories are set for a total of 6537, excluding 3598 non-usable data out of 10135 question answering mail data. It was. Data that is not to be used is data that is not so-called simple form / content, and that cannot be reused. The numbers that came out during the presentation were obtained in a verification experiment.

[2.3 Question answering database construction support system]
Construction of a question answering database requires a lot of man-hours. Therefore, at first, a question answering database was manually constructed for 1705 question answering data.
In order to improve the efficiency of question answering database construction, a question answering database construction support system was created and utilized. The above 6537 question response data are constructed using this question response database construction support system. The screen shot is shown in FIG. The question answering database may be constructed manually without using the question answering database construction support system.

[2.3.1 System configuration]
The features of the mail call center question answering database construction support system are as follows.
・ Calculate the similarity between the question mail to be classified and the existing question category ・ Provide an easy-to-operate GUI interface −Display the contents of the existing question mail and the question category in the question mail and the question category − On the other hand, presenting question categories in order of similarity − Selecting / creating / deleting question categories that belong to question emails to be classified − Creating answer template for each question category (see below)

Here, a system configuration of a question answering database construction support system of a mail call center, a system database, a question category determination system, and a GUI interface will be described.
The configuration of the mail call center question answering database construction support system is shown in FIG.
In the question answering database construction support system of the mail call center, the question emails to be classified are classified into question categories through the following four steps (explained on the creation of answer sentences).

  1) The degree of similarity between the question mail to be classified and the existing question category is calculated through the question category determination system.

  2) Present all existing question categories in the order of similarity calculated in 1) with the GUI interface.

  3) If it is determined that the question mail to be classified belongs to the existing leaf category presented, it is classified into that leaf category. Although processing can be performed automatically, in this embodiment, classification is performed through approval from the user via the GUI. Specifically, it accepts designation from the user of the question category displayed in the list format presented in 2), and further accepts approval of classification.

  4) When it is determined that the question mail to be classified does not belong to the presented existing leaf category, a new leaf category or branch category is created for the question mail to be classified according to the question category creation criteria. Classify the question emails into the newly created leaf category. Although processing can be performed automatically, in this embodiment, classification is performed through approval from the user via the GUI. Specifically, when the question category is not displayed in 2), even when the question category is displayed, when the question category is not an appropriate question category, an instruction to create a new leaf category or branch category is received from the user.

[2.3.2 System database]
When classifying question mails in the question answering database construction support system of the mail call center, the question mail data to be classified is calculated through the question category judgment system and the similarity to the existing question category is calculated. It is necessary to prepare a question category for a call center question answering database construction support system.
Here, the question category created using 1705 question mails is used as the database for the question answering database construction support system of the mail call center (refer to [2.1 Question Answering Database Construction Method]).

[2.3.3 Question category determination system]
In the question answering database construction support system of the mail call center, the question category judgment system is used to judge the leaf category to which the question mail belongs.
As a determination method, the question mail and the question category are represented by points on the vector space, and the similarity between the vectors is defined.

  Regarding the document vector of the question mail, the vector elements are determined so that the similarity between the question mail and the question category to which the question mail belongs is increased. The determination method uses a document vector that more accurately reflects the contents of the question mail by extending the document vector based on the weighting of TF-IDF and taking into account the co-occurrence of the body and the predicate and the feature of the sentence.

There are three types of question email document vectors:
-Document vector based on TF-IDF-Document vector considering co-occurrence of body and prescription-Document vector considering sentence type Also, the document vector of the question category obtained by averaging the document vectors of the question mail belonging to the question category The degree of similarity of both is obtained by a weighted cosine scale with the document vector of the question mail to be determined. The question category to which the question mail belongs is determined based on the similarity calculation result.
For details, see [3. Feature vector]. In other words, the feature vector is used not only to identify an appropriate answer sentence for the question sentence in the answer support system but also in the question answer database construction support system.

[2.3.4 GUI]
A GUI interface for the system has been created so that a user can easily use the question answering database construction support system of the mail call center as a tool.
By pressing the button displayed in FIG. 9, it is possible to realize the interface function described at the beginning of [2.3.1 System Configuration].

At the initial startup, a list of mails, leaf categories, and branch categories to be classified is displayed on the left side of the window in FIG.
By selecting and clicking one question mail to be classified, the contents of the selected question mail are displayed in the center of the window of FIG.

  By pressing the “similarity calculation” button for the question mail selected in FIG. 9, the leaf category and branch category on the left side of the window in FIG. 9 are redisplayed in order of similarity. The order of similarity is displayed with color, and the darker the color, the higher the similarity. In other words, the degree of similarity is manifested by color shading.

  Clicking on a leaf category displays a list of question emails classified in the leaf category. When a leaf category is selected here, a list of question emails belonging to that leaf category is displayed. Clicking on a question email in the leaf category displays the content of the question email on the right side of the window in FIG. Is possible. Here, it is determined to which leaf category the question mail to be classified belongs. When there is a leaf category to which the user belongs, the user pushes a “sort” button to classify the leaf category to which the user belongs. If the leaf category does not exist, the “(New) Category Creation” button is pressed to create the leaf category to which it belongs.

[3. Category answer sentence creation]
Since the answer sentences of the same data or similar data classified in the leaf category are the same or similar, a leaf category answer sentence that is a common answer sentence is created for the question mail belonging to the leaf category. When creating an answer text of an unknown inquiry mail, the answer text is created using the category answer text of the leaf category to which the question mail belongs.
There are two types of leaf category answer sentences: a standard answer sentence and a template answer sentence. The fixed answer text is for cases where model-related information is not included, and is output as it is when the answer text is created. On the other hand, the template response text has slots for various types of model-related information such as hardware and software. The model-related information is extracted from the model-related information database, put into the slot, and a response text is created. Output.

In this chapter, we will describe the application of the built-in mail call center question answering database to the creation of answer sentences.
In the mail call center question answering database, the leaf category is a collection of similar question mails. The similar question mail refers to a question mail in which the content or meaning of the inquiry mail is similar, and the answer sentence which is the final review data transmitted to the inquiry mail is similar or the same.

  Therefore, it can be considered that all similar question mails belonging to the leaf category have a common answer sentence. The common answer sentence is defined as a leaf category answer sentence. Each leaf category has one type of leaf category answer text. Further, it is possible to create a leaf category answer sentence by referring to similar question mails belonging to the leaf category.

As previously mentioned, there are two types of leaf category answer sentences as follows. The leaf category answer text includes a standard answer text that does not include a part that is changed by information such as the OS and model, and a leaf category answer text that includes a part that is changed by information such as the OS and the model.
We will explain each leaf category answer sentence from the next section.

[3.1 Response text creation]
A method for creating a reply message for an inquiry mail using a leaf category reply sentence is described.
FIG. 11 shows a flow of creating an answer message for the inquiry mail.
The inquiry mail determines the leaf category to which it belongs through the question category determination system. If the leaf category to which it belongs is determined, an answer sentence is created using the leaf category answer sentence.

[3.1.1 Use of standard answer sentences]
When the leaf category answer sentence of the leaf category to which the inquiry mail belongs is a standard answer sentence, the leaf category answer sentence is output as an inquiry mail answer sentence.

[3.1.2 Use of template response]
When the leaf category answer sentence of the leaf category to which the inquiry mail belongs is a template answer sentence, the answer sentence of the inquiry mail is created in the following four steps.

  1) Extract the model name in the inquiry email.

  2) Use the model name extracted in 1) to match the model information that enters the slot in the template response sentence in the model DB.

  3) Replace the model information matched in 2) with a slot.

  4) Output an answer sentence.

When creating a reply to an inquiry mail using a template reply, a model DB is created in advance. In this study, we created a model DB by collecting model information corresponding to the slot items used when creating a template response for all models supported by the mail call center.
Here, the device name and the device information have been explained, but other information may naturally be used.

[4. Feature vector]
Hereafter, a method for determining to which category a question sentence newly given as input belongs will be described using a classified question answering database.
In the present embodiment, the question sentence and the category are represented by points on the vector space. Also, the similarity between vectors is defined. The vector element is determined so that the similarity between the question sentence and the category to which the question sentence belongs is increased, and the similarity is defined to estimate the category of the question sentence based on the similarity.

In this method, a document vector that reflects the contents of a question sentence more accurately is used by extending a document vector based on TF-IDF weighting, and taking into account the co-occurrence of body / phrase and the characteristics of the sentence.
That is, the following feature vectors are used in combination.
-Feature vector based on TF (Term Frequency) / IDF (Inverse Document Frequency)-Feature vector considering co-occurrence of body and prescription-Feature vector considering sentence type

  Further, the average of the document vectors of the question sentences belonging to the category is used as the category document vector, and the similarity is obtained by a weighted cosine scale. The cosine scale considers the similarity of two vectors by the cosine of the angle formed by the vectors. The same vector has an angle of 0 and the cosine is 1, and vectors having completely different elements are orthogonal to each other. The cosine is zero. The cosine of the angle θ formed by the vectors can be expressed by the following equation.

ベクトルが正規化済みであるならば、これはベクトル同士の内積に等しい。以降、基本的にベクトルはすべて正規化済みであると仮定する。つまり、余弦尺度は内積によって求められる。

If the vector is normalized, this is equal to the inner product of the vectors. Hereinafter, it is basically assumed that all vectors have been normalized. In other words, the cosine scale is obtained from the inner product.

  In the present embodiment, as a word characterizing a document, a combination of a prototype of independent words such as nouns (including unknown words), verbs, and adjectives and part of speech information is used. The analysis of words was performed up to morphological analysis and paragraph breakage using Japanese dependency analyzer CaboCha and morphological analyzer MeCab. A pre-processed sentence was applied to the morpheme analyzer so as not to distinguish between half-width and full-width characters, and uppercase and lowercase letters. However, it is not considered that continuous nouns are compound nouns because learning data becomes insufficient. For the continuation of the number and the classifier, we replaced the number with a number class instead of the actual sequence of numbers. It should be noted that in a sentence in parentheses, a sentence of two or more clauses is handled as a separate sentence while retaining the information that it is a sentence in parentheses.

[4.1 Feature vector based on TF / IDF]
In the system, a question sentence is expressed by extending a document vector whose elements are weights of TF / IDF of words appearing in an inquiry mail.
TF-IDF weighting is a technique for calculating the weight of an index word in automatic text indexing. TF (Term Frequency) is an occurrence frequency of the index word t in a certain document d, and is expressed as tf (d, t). Further, IDF (Inverse Document Frequency) is defined as follows by the number N of documents and the number dfreq (t) of documents in which the index word t occurs one or more times.

索引語ｔの文書ｄにおける重みｗ（ｔ，ｄ）として、ＴＦとＩＤＦの積をもちいるのがＴＦ−ＩＤＦ重み付けである。重み付けにＴＦを用いるのは、文書中で繰り返し生起する語はその文書において重要な概念であると考えるためである。しかし、多くの文書に生起する語は、文書を特定する性質を持たず、索引語として適していない。そこで、語がどのくらい特定性を持つかをＩＤＦによって重み付けに反映させている。

TF-IDF weighting uses the product of TF and IDF as weight w (t, d) in document d of index word t. The reason for using TF for weighting is that words that occur repeatedly in a document are considered to be important concepts in the document. However, words that occur in many documents do not have the property of specifying the document and are not suitable as index words. Therefore, how specific the word is is reflected in the weighting by the IDF.

[4.2 Feature vector considering co-occurrence of body and prescriptive terms]
Weighting by TF / IDF usually expresses a measure by which a word characterizes a specific document and does not reflect the structure of the sentence. Therefore,
·"power off."
·"Turn on the power."
For the two sentences, the word “power” is given the same weight. In practice, however, the two categories should be recognized as having different characteristics in determining the target question category. This is a feature that is difficult to grasp if only the appearance frequency of words is considered. Therefore, in addition to the weighting by TF / IDF, it is considered to use the degree of co-occurrence in one sentence of the prescriptive word as a weight. For each body word, a feature vector whose element is the frequency of co-occurrence in one sentence is used. It is stored together with the TF-IDF weight as an element of the document vector. Thus, when two words are compared, it is considered to use the similarity based on the cosine scale of the co-occurrence vector.
Normally, the similarity sim (V, V ′) between the document vectors V and V ′ based only on TF-IDF is obtained by a cosine scale, that is, an inner product. When the number of words in the entire document, that is, the dimension of the document vector is n, it is expressed as follows.

ここで、共起ベクトルの類似度を重みに加える。要素にTF-IDFによる重みと体言・用言の共起ベクトルを持つ文書ベクトルＶ_c、Ｖ´_cの類似度ｓｉｍ（Ｖ、Ｖ´）を、以下のように定義する。

Here, the similarity of the co-occurrence vector is added to the weight. Document vector V _c which elements having the co-occurrence vector of weights and substantive-verb by TF-IDF, the similarity sim of _V'c (V, V'), defined as follows.

上式は、ある語ｉについて、ＴＦ−ＩＤＦの重みが大きいほど、また、語ｉに同じような共起の傾向があるほど、文書ベクトルの類似度が高くなる。上記の「電源」の例の場合、それぞれに共起している用言は「切る」、「入れる」であるので、共起ベクトルの類似度は０である。したがって文書ベクトル全体の類似度も０となり、ふたつの文は似ていないと判断される。

In the above formula, the similarity of the document vector increases as the weight of TF-IDF increases for a certain word i, and as the word i has a similar co-occurrence tendency. In the case of the above “power supply”, the co-occurrence remarks are “turn off” and “turn on”, respectively, and the similarity of the co-occurrence vectors is 0. Therefore, the similarity of the entire document vector is also 0, and it is determined that the two sentences are not similar.

[4.3 Feature vector considering sentence type]
In order to create a feature vector that more accurately reflects the contents of an inquiry mail sent from a personal computer user, consider the tendency of the meaning of a word in a sentence. For this purpose, the inquiry mail is first analyzed to check the characteristics of each sentence type, and a rule for identifying the sentence type is created based on the analysis result.

[4.3.1 Analysis of question text]
Here, by analyzing the question mail, a few sentence types were set as follows.
・ Question: Sentences with questions such as "Can you do it?"
・ Problem: A sentence that states the problem, such as "I can't do it" or "I don't know how to do it".
Intention: A sentence that states the intent / intention of the questioner, such as “I want to do” or “I want to do”.
・ Situation: A sentence describing the procedure / situation of the problem.
・ Think: A sentence in which the questioner's thoughts are stated, such as "I think it is."
・ Other case: A sentence that states that there is no problem in other situations, such as “There is no problem with playback on the HDD”.
・ About: A sentence that expresses the content of the question, such as “about”. It may be stated in the first line of questions and answers.
-Message: A sentence that describes the contents of a character string displayed on the screen, such as an error message or dialog.
・ Etc: Other information

[4.3.2 Result of question sentence analysis]
The result of totaling the above sentence types is shown below.
・ Question 324
・ Problem 648
・ Intention 87
・ Situation 398
・ Think 37
・ Other case 80
・ About 368
・ Message 96
・ Etc 34

  The above nine types of sentences, such as sentences describing questions and sentences describing personal computer malfunctions / problems, are set, analyzing about one week of inquiry mail, 323 cases, 2072 sentences, A rule for identifying a sentence type is created from the function word, and a feature vector having the frequency of each word in each sentence type as an element is used. The numbers on the right side of the table above are for reference only, and are the number of times each sentence type appeared during the analysis.

Most questions included either QUESTION or PROBLEM, and if neither appeared, only 3 out of 323. All three of them included INTENTION. Furthermore, the analysis for each type is shown below.
・ Question: In most cases, the sentence ends with the symbol “?” Or the particle “ka” or “Please tell me” “Please tell me” or “Please do it”. Other types rarely match.
・ Problem: Two-thirds of the cases consist of basic forms of verbs and adjectives that end with sentences, and cases that end with “can't do”, “do not”, “do”. Moreover, many appear just before the QUESTION statement.
・ Intention: In most cases, the sentence ends with "I want to do", "I want to do" or "I want to do".
・ Situation: It often ends with a past tense like “I did”, but there are many cases where it does not. Many appear just before the PROBLEM statement.
・ Think: Includes "~ kato" or ends with "I think" or "I feel".
・ Other case: Sentences that include "~ ha", "in", "~ mo", "toto", "to dato", etc., end of sentence "can be done""abnormal""noproblem""normal""occurrence" It ends with "No".
・ About: In the first line of questions and answers, the sentence ends with a noun.
-Message: All or part of the sentence is often enclosed in "" or "". Immediately after that, words such as “to”, “to”, and “te” are added, followed by sentences such as “displayed”, “out”, and “message out”.
・ Etc: There is information such as “I am a beginner” and “My name is ~”. For these, it is difficult to prepare correspondence rules in advance and the frequency of occurrence is low, so this time we will not respond.

[4.3.3 Sentence type identification rule]
Based on the above analysis results, a sentence type identification rule was created. Rules are applied in three stages. First, the rules to be applied first will be described. For each of the following statements, assign a type to allow duplication.
Based on the above analysis results, a sentence type identification rule was created. Rules are applied in three stages. First, the rules to be applied first will be described. For each of the following statements, assign a type to allow duplication.

・ Question: The sentence ends with a question mark other than “ga?”. Alternatively, the end of the sentence ends with the particle “ka”. Alternatively, "Teach me", "Teach", "Professor", "Notice", "Why", "Wish" are included within the last five sentences.
・ Problem: The sentence ends with the basic form of an independent verb / adjective. Alternatively, in the sentence including the case particle “ga” instead of “was” and “ga” at the end of the sentence, no other particle is included between the case particle “ga” and the end of the sentence. Or, the three phrases at the end of the sentence do not include "I'm sorry", "I'm sorry", "I'm not doing", "I'm not doing", but "I'm missing", "I'm gone", "I'm gone" ”“ Now ”“ I did it ”“ I will do it ”“ I don't ”“ Not ”“ It will ”“ It will ”“ It will be ”is included.
Intention: The five clauses at the end of the sentence include “(verb) + tai”, “want” and “(verb) + (yo) uto”, followed by the verb “think” and “think”.
・ Think: The sentence includes the word “kato” in the sentence, or includes “I think”, “I thought”, “I think”, “I think”, “I feel”, “I feel” in the last three sentences.
・ Other case: Sentences that include particles or particle sequences “ha” “in” “mo” “to” “dato”, and the end of the sentence is “can” “move” “activate” “activate” ”" Startup "to use, it ends with" basic form "" ta "" mass "" is "or" abnormal "" problem "" inconvenient "after" no "" not "" no " Followed by. Alternatively, after the particles “ha” and “to”, sentences including “normally”, “correctly”, “normally”, “normally”, “properly”, “successfully”, and “properly” come.
・ About: In the first line of questions and answers, the sentence ends with a noun.
Message: A sentence containing the particle “to”, followed by “message”, “box”, “pop-up”, “display”, “error”, and the verb “out”. If there is a part enclosed by ``, (), ', ``, just before the particle `` to'', even if it spans multiple sentences, all the parts enclosed in parentheses etc. are `` MESSAGE Judge that.

Next, SITUATION or PROBLEM is allocated to a statement for which no type has been allocated. Consider the following three cases.
・ When neither QUESTION nor PROBLEM appears in the question text. When ABOUT appears, the sentence including the word in ABOUT is set to PROBLEM. If there is no sentence containing the word in ABOUT, or if ABOUT does not appear, the first sentence with no type assigned is set to PROBLEM. The rest is SITUATION.
・ When QUESTION appears in the question text. If no type is assigned to the statement immediately before QUESTION, it is set to PROBLEM. The rest is SITUATION.
・ Other than that. All statements that are not assigned types are called SITUATION.

Next, a rule based on the priority order between the types is applied to the duplicate sentence type to determine the type.
A sentence type is determined in the following order of priority for a sentence having a duplicate sentence type.
"MESSAGE">"ABOUT">"QUESTION">"PROBLEM">"OTHERCASE">"INTENTION">"THINK"

Apply the above rules to determine the sentence type. A word that appears in the question sentence has information as to how many times it appears in what sentence type as a vector.
Using the obtained sentence type, Equation (4) is expanded as follows. Weight by TF-IDF elements, co-occurrence vector of nominal-verb, document vector V _t with statement type vector, the similarity sim (V _{t, V't)} of _V't a is defined as follows .

式（５）は、語ｉが同じような文タイプに出現する傾向がある場合、文書ベクトルの類似度が大きくなることを表している。質問文が、どのようなことについて述べているのか、という傾向が似ているものを、類似度が高い、と評価する。

Formula (5) represents that the similarity of the document vector increases when the word i tends to appear in a similar sentence type. If the question sentence has a similar tendency of what is being described, it is evaluated that the degree of similarity is high.

[4.4 Similarity Calculation Using Category Average Vector]
In order to calculate which question category an unknown inquiry mail belongs to, an average of the feature vectors of question sentences in each question category is used as a question category vector for convenience. The conceptual diagram is shown in FIG.
For these, the similarity to the unknown inquiry mail vector is calculated. This is because by averaging the document vectors of the question sentences in the category, it is possible to remove a small number of noises and leave only information of words that are truly characteristic in the category.

そして、質問ベクトルと各質問カテゴリとの距離を計算し、最も近い質問カテゴリをもつカテゴリに質問が属すると判断する。
この平均ベクトルを正規化したものと、未知の質問文の文書ベクトルとの類似度の計算結果を利用して、回答作成支援システムを作成する。

Then, the distance between the question vector and each question category is calculated, and it is determined that the question belongs to the category having the closest question category.
An answer creation support system is created using the calculation result of the similarity between the normalized average vector and the document vector of an unknown question sentence.

[4.5 Specific example]
[4.5.1 Document vector]
W: Word space W∋i: Corresponds to a certain expression Example: i = PC, i ′ = power source. . .
Here, a value obtained by weighting a word i in a document vector V in TF-IDF is represented as TF-IDF (i).

At this time, the sentence vectors in the sentences 1 and 2 given below as an example are w: TF-IDF (i) with the body language i in the sentence as an axis.
c: {v: TF-IDF (v), v ': TF-IDF (v'),. . } V, v ': body i
Co-occurrence in the sentence, c: vector with the axis of v: {sentence type 1: number of occurrences of i in sentence type 1,. . } T: Vector with sentence type as axis Set and have the above three values, and normalize element w of the vector.

● Sentence 1
The PC I just bought doesn't turn on.
What should I do?
[Text vector V]
{I1: [w = TF-IDF (PC), c = {Buy: TF-IDF (Buy), Enter: TF-IDF (Enter)}, t = {Negation: 1}]
i2: [w = TF-IDF (power source), c = {buy: TF-IDF (buy), enter: TF-IDF (enter)}, t = {deny: 1}]
i3: [w = TF-IDF (what), c = {do: TF-IDF (do)}, t = {question: 1}]} * i1 = PC, i2 = power supply, i3 = what

● Sentence 2
My PC doesn't start, what should I do?
[Text vector V ']
{I1: [w = TF-IDF (PC), c = {TF-IDF (activation)}], t = {question: 1}}} * i1 = PC

[4.5.2 Similarity]
When calculating the similarity between sentence vectors V and V ', the number of dimensions in the vector space is (V
It is equal to the number of dimensions of the word space of (V ′).
Therefore, V ■ in the example is a one-dimensional vector space having only PC as an axis, but here it is expanded to a word space of (V∪V ′).
Similarly, the co-occurrence vector c and the sentence type vector t are also expanded from the sentence vector V ′.
It is as follows.

[Text vector V '']
{I1: [w = TF-IDF (PC), c = {Buy: TF-IDF (buy), Enter: TF-IDF (Enter), Start: TF-IDF (Start)], Do: TF-IDF (Yes )}, T = {Question: 1, Denial: 0}]
i2: [w = TF-IDF (power supply), c = {buy: TF-IDF (buy), enter: TF-IDF (enter), start: TF-IDF (start), do: TF-IDF (do) }, T = {Question: 0, Denial: 0}]
i3: [w = TF-IDF (what), c = {buy: TF-IDF (buy), enter: TF-IDF (enter), start: TF-IDF (start), do: TF-IDF (do)] }, T = {question: 0, negation: 0}]} * i1 = PC, i2 = power supply, i3 = what Here, the similarity is calculated according to the following equation (5). Further, (t · t ′) here returns 1 if the sentence types match and 0 if they do not match.

[5. Operation]
FIG. 13 is a block configuration of the answer support system according to this embodiment, and FIG. 14 is an operation flowchart of the answer support system according to this embodiment. Note that the block configuration shown in FIG. 13 is an example, and a plurality of module configurations can be adopted as will be apparent to those skilled in the art. Here, although the operation subject is clearly shown, from the hardware viewpoint, the computer and the processor are the operation subjects.

The questioner creates a question mail containing the question text on the user computer 600 and sends it to the address of the respondent computer 100.
The respondent computer 100 receives a question mail from the user computer 600 via a plurality of mail servers by a mailer. The mail server directly accessed by the respondent computer 100 may have a function of appropriately distributing mail with a predetermined mail address to the registered respondent computers.

The user gives an instruction to import the question mail received by the mailer into the answer support system.
The input unit 1 receives an instruction from the user and takes in the inquired question mail (S100).
The pre-processing unit 2 performs pre-processing such as full-width (half-width) character conversion and alphabet upper-case (lower-case) conversion.
The morpheme analysis unit 3 performs morpheme analysis on the preprocessed question sentence (S200).

The TF-IDF document vector unit 41, the co-occurrence vector unit 42, and the sentence type document vector unit 43 of the document vector creation unit 4 obtain respective vectors of the body of the question mail that has been taken in.
The similarity calculation unit 6 reads out the average document vector of each question category, and obtains the similarity using equation (5) from the read average document vector and each vector of the obtained question mail (S300, S400).
After the similarity calculation unit 6 obtains the similarity to each question category, the output unit 8 reads each question category and displays the question categories in a list in order of similarity (S500).

  The selection of the question category from the user is accepted, the answer document specifying unit 7 reads out the answer sentence recorded in association with the question category from the identification information of the question category, and the output unit 8 displays it (S600).

The correction to the answer sentence displayed by the output unit 8 is accepted.
When the approval from the user is accepted, an answer sentence is returned to the questioner via the mailer (S700).

  Further, the question sentence transmitted this time through the question answering database construction support system, the answer sentence and the average document vector are recorded (S800). Here, the question category has already been determined, and the recording process is quickly performed without accepting the selection of the question category from the user.

In order to confirm the accuracy of category determination, three types of experiments are performed and the results are evaluated.
(Category judgment experiment)
As experiment data, 629 categories having three or more question sentences in one category are used as data A among classified question / answer data. Further, 145 categories having 12 or more question sentences in one category are used as data B. The total number of data A is 6536 and has 83 upper categories. The total number of data B is 4023 and has 52 upper categories. With respect to these data, the data is divided into three parts, two of them are used as learning data, and the remaining test data is subjected to a three-fold cross-validation test, and the correct answer category of the question sentence, We investigated how many high-ranking categories of correct answers were judged.
Assume that the correct answer category and the higher-ranked category of the correct answer are evaluated for the question sentence.

カテゴリ判定の実験結果である。
データＡでの判定結果は、データＢに比べると悪い。その理由として、学習データの不足が考えられる。データＡのカテゴリ数は６２９個であるが、その大半が１カテゴリ内に３個か４個程度のデータしか持っていない。学習データの数に比べて、カテゴリ数が非常に多いため、判定ミスが増加したものと考えられる。

It is an experimental result of category judgment.
The determination result for data A is worse than that for data B. The reason may be a lack of learning data. The number of categories of data A is 629, but most of them have only about 3 or 4 data in one category. Since the number of categories is very large compared to the number of learning data, it is considered that the number of determination errors has increased.

(Weighing by TF-IDF and comparison experiment with proposed method)
By using the experimental data B in the determination experiment, the document vector using only simple weighting by TF-IDF, the co-occurrence of body words / uses as the proposed method, and the similarity of the document vector considering the sentence type The judgment accuracy was compared.
Assume that the correct answer category and the higher-ranked category of the correct answer are evaluated for the question sentence.

It is the result of the comparison experiment by the document vector of TF-IDF and the proposal method.
The proposed method is slightly more accurate, but the improvement rate is about 10%. The reason for this is that as a result of averaging the vectors, the category document vectors have become very characteristic. When the similarity between the category document vectors was calculated, the values were almost close to zero. When there is a characteristic word, it reacts strongly and other factors are not considered much.

(Comparison experiment of category judgment by KNN method and mean vector method)
Using the experimental data B in the determination experiment, the accuracy of category determination by the KNN method and category determination by the average vector method were compared. In the average vector method, a document vector of an unknown question sentence is input, and the category is determined using the similarity to the average vector of the category. In the KNN method, the similarity between the input vector and the document vector in all learning data is determined. The category to which the input vector belongs is determined from the category to which k document vectors belong from the highest similarity.

In data B, since it is guaranteed that all categories in the learning data have at least 8 data, the value of k is set to 8.
The similarity was calculated using a document vector that takes into account the co-occurrence of body and predicate, which is the proposed method, and the sentence type.
Assume that the correct answer category and the higher-ranked category of the correct answer are evaluated for the question sentence.

It is a result of a comparative experiment between the KNN method and the average vector method.
Although the KNN method gives a slightly higher value in the first rank determination of the upper category, the average vector method gives better accuracy especially in the determination within the third rank.
The reason for the judgment error of the KNN method is that there are 583 out of 4023 cases where there is no data that belongs to the correct category among k pieces of data with high similarity, and the data distribution is quite large. It overlaps in the range.

  Although the present invention has been described with the above embodiments, the technical scope of the present invention is not limited to the scope described in the embodiments, and various modifications or improvements can be added to these embodiments. . And embodiment which added such a change or improvement is also contained in the technical scope of the present invention. This is apparent from the claims and the means for solving the problems.

It is a principle figure (similarity of a document) of invention. It is a principle figure (co-occurrence vector, sentence type vector) of invention. It is a principle figure (document-document group similarity) of the invention. It is a principle figure (answer assistance) of invention. It is a system screen which concerns on embodiment of this invention. It is a structure of the system screen of FIG. It is an example of the network structure to which the computer which built the reply assistance system which concerns on embodiment of this invention belongs. 4 is a tree structure of a question answering database according to an embodiment of the present invention. It is a screen shot of the question answering database construction support system concerning the embodiment of the present invention. 1 is a configuration of a mail answering center question answering database construction support system according to an embodiment of the present invention. It is an operation | movement flowchart which produces the reply sentence of the question mail which concerns on embodiment of this invention. It is a vector conceptual diagram in each question category concerning the embodiment of the present invention. 1 is a block configuration of an answer support system according to an embodiment of the present invention. It is an operation | movement flowchart of the reply assistance system which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Input part 2 Preprocessing part 3 Morphological analysis part 4 Document vector creation part 41 TF-IDF document vector part 42 Co-occurrence vector part 43 Sentence type document vector part 5 Document vector memory | storage part 6 Similarity calculation part 7 Answer document specific | specification part 8 Output unit 100 Respondent computer 101 CPU
102 RAM
103 ROM
104 HD
105 CD-ROM drive 111 Mouse 112 Keyboard 121 Display 122 Speaker 131 LAN interface 200 Respondent computer 300 Server 400 Printer 500 Network device 600 User computer

Claims (8)

Means for morphological analysis of a sentence of a document consisting of sentences;
Means for obtaining a TF / IDF vector from the morpheme-analyzed document, with the weight of the index word appearing in the document as a factor of TF / IDF;
Means for obtaining a co-occurrence vector from the morphological-analyzed document, with the frequency of the prescriptive co-occurrence in the sentence in which the manifestation appears for the manifestation appearing in the document,
Find the TF / IDF document vector and co-occurrence vector of the first document,
Find the TF / IDF document vector and co-occurrence vector of the second document,
Document similarity derivation for determining the similarity between the first document and the second document from the obtained TF / IDF document vector and co-occurrence vector of the first document and the TF / IDF document vector and co-occurrence vector of the second document apparatus. A means for morphological analysis of a sentence consisting of sentences,
Means for obtaining a TF / IDF vector from the morpheme-analyzed document, with the weight of the index word appearing in the document as a factor of TF / IDF;
Means for determining a sentence type of a sentence in which an index word appearing in the document appears from a document subjected to morphological analysis, and obtaining a sentence type vector having the frequency of each sentence type as an element,
Find the TF / IDF document vector and sentence type vector of the first document,
Find the TF / IDF document vector and sentence type vector of the second document,
Document similarity derivation for determining the similarity between the first document and the second document from the obtained TF / IDF document vector and sentence type vector of the first document and the TF / IDF document vector and sentence type vector of the second document apparatus. Means for morphological analysis of a sentence of a document consisting of sentences;
Means for obtaining a TF / IDF vector from the morpheme-analyzed document, with the weight of the index word appearing in the document as a factor of TF / IDF;
Means for obtaining a co-occurrence vector from the morphological-analyzed document, with the frequency of the prescriptive co-occurrence in the sentence in which the manifestation appears for the manifestation appearing in the document;
Means for determining a sentence type of a sentence in which an index word appearing in the document appears from a document subjected to morphological analysis, and obtaining a sentence type vector having the frequency of each sentence type as an element,
Find the TF / IDF document vector, co-occurrence vector and sentence type vector of the first document,
Find the TF / IDF document vector, co-occurrence vector and sentence type vector of the second document,
From the obtained TF / IDF document vector, co-occurrence vector and sentence type vector of the first document, and TF / IDF document vector, co-occurrence vector and sentence type vector of the second document, the first document and the second document Document similarity deriving device for determining similarity. Each means of the document similarity deriving device according to any one of claims 1 to 3,
Obtaining a document vector that is a TF-IDF vector and a co-occurrence vector and / or a sentence type vector for the first document;
Obtaining a document vector of each document of the second document group consisting of a plurality of documents,
An average document vector is obtained from the document vectors of the respective documents of the obtained second document group,
A document-document group similarity deriving device that obtains the similarity between the first document and the second document group from the obtained average document vector of the second document group and the document vector of the first document. Each means of the document similarity deriving device according to any one of claims 1 to 3,
A document vector which is a TF-IDF vector and a co-occurrence vector and / or a sentence type vector is obtained for a comparison target document to be compared,
Find the document vector, co-occurrence vector and sentence type vector of the index TF-IDF of the nth document,
Determining the similarity between the comparison target document and the nth document from the document vector of the comparison target document and the document vector of the nth document;
n is a high-similarity document specifying device that specifies n-th documents having a high similarity among the similarities between each n-th document and a comparison target document. Each means of the document similarity deriving device according to any one of claims 1 to 3,
A document vector that is a TF-IDF vector and a co-occurrence vector and / or a sentence type vector is obtained for a comparison target document,
Obtaining a document vector of each document of the nth document group composed of a plurality of documents;
An average document vector is obtained from the document vectors of the respective documents of the obtained nth document group,
The similarity between the first document and the nth document group is obtained from the obtained average document vector of the nth document group and the document vector of the first document,
n is a high-similarity document group identification device that identifies n-th document groups having high similarity among the similarities between each n-th document group and a comparison target document. Means for morphological analysis of a sentence of a document consisting of sentences;
Means for obtaining a TF / IDF vector from the morpheme-analyzed document, with the weight of the index word appearing in the document as a factor of TF / IDF;
Means for obtaining a co-occurrence vector from the morphological-analyzed document, with the frequency of the prescriptive co-occurrence in the sentence in which the manifestation appears for the manifestation appearing in the document;
From the morphologically analyzed document, determine the sentence type of the sentence in which the index word appearing in the document appears, and let the computer function as a means for obtaining a sentence type vector with the frequency of each sentence type as an element,
Find the TF / IDF document vector, co-occurrence vector and sentence type vector of the first document,
Find the TF / IDF document vector, co-occurrence vector and sentence type vector of the second document,
From the obtained TF / IDF document vector, co-occurrence vector and sentence type vector of the first document, and TF / IDF document vector, co-occurrence vector and sentence type vector of the second document, the first document and the second document A document similarity derivation program that lets a computer find similarities. Means for morphological analysis of a sentence of a document consisting of sentences;
Obtaining a TF / IDF vector from a morphological-analyzed document, with the weight of the index word appearing in the document as a factor of TF / IDF;
Obtaining a co-occurrence vector from the morphological-analyzed document, with the frequency of the prescriptive co-occurrence in the sentence in which the manifestation appears for the manifestation appearing in the document;
Determining a sentence type of a sentence in which an index word appearing in the document appears from a document subjected to morphological analysis, and obtaining a sentence type vector having the frequency of each sentence type as an element,
Determining a TF / IDF document vector, co-occurrence vector and sentence type vector of the first document;
Obtaining a TF / IDF document vector, co-occurrence vector and sentence type vector of the second document;
From the obtained TF / IDF document vector, co-occurrence vector and sentence type vector of the first document, and TF / IDF document vector, co-occurrence vector and sentence type vector of the second document, the first document and the second document A document similarity derivation method further comprising: obtaining similarity.

Images