JP2010282403A - Document retrieval method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a document retrieval method capable of extracting exactly an important document desired by a user. <P>SOLUTION: A document is analyzed in a query document Qn generated in a Q&A system 2 and an a reply document An for replying a query, as objects, to prepare a retrieval index SI. The retrieval index SI is weighted to be weighted with an increased weight or the like, as a contrivance, as to not only a word appearance frequency but also a noun word contained in a sentence with an end auxiliary word indicating a question or the query. A document index DI is also prepared in a document file in a database 3 as an object. The document index DI is weighted to be weighted with an increased weight or the like, as a contrivance, not only depending on a word appearance frequency but also as to a word used in the plurality of documents, in a document group concerned in the preparation of the same person. The retrieval index SI and the document index DI are applied to a retrieval algorithm M to calculate a degree of conformity, and the best document for the query document is extracted out of the database 3. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a document search method capable of appropriately extracting a document file desired by a user by appropriately setting a search source search index and a search destination document index.

  Communication network systems called Q & A systems are widespread on local networks such as the Internet and company networks. Generally, in a Q & A system, when a user inputs a question document on the system, other users who browse the question on the system voluntarily put an answer document on the question on the system based on their own knowledge. The operation of inputting is performed. If the user of a system posts a question about a problem to be solved or information to be discovered, there is an advantage that a certain solution for them can be obtained from the system.

  However, in the above Q & A system, answers can be obtained only from a very limited range of specific answer documents posted in the form of responses to specific question documents. That is, the best answer document for the requested question is not always posted, and there is even a case where no response is given to the question document, and the user often cannot obtain desired information.

  As another method for the user to obtain desired information, there is also a method for setting a keyword that can be recalled with respect to a question event and performing a search on a predetermined database. Although this method can acquire information by a simple operation of inputting a keyword, the search keyword is set depending on the subjectivity of the questioner, so the appropriate keyword for acquiring the best answer is set. In many cases, the desired information cannot be obtained. In the first place, the questioner often does not recognize what is a problem-solving keyword, and appropriate keyword setting cannot be expected from the beginning.

  Patent Document 1 discloses a search method for searching a database using a keyword used in a question document when a new question occurs and extracting a pair of similar questions and answers in the past. Yes. According to this search method, since past Q & A cases for the same type of question are presented, it is certain that the best answer can be easily obtained.

  In such a search method, a search index (keyword) created based on the query document of the search source, and a document index created in advance based on a large number of document files accumulated in the search destination database, A technique is adopted in which a document file having a high similarity is extracted as a hit document by collating using a predetermined search algorithm.

JP 2006-92473 A

  According to the above search method, it is possible to extract a document similar to a question document with a certain degree of accuracy. However, the probability that the answer document that the user really wants can be extracted has not been so high. That is, there is a problem that extraction of an important document that leads to solution of a problem that the user has cannot be performed sufficiently by a conventional search method.

  In view of the above problems, an object of the present invention is to provide a document search method capable of accurately extracting an important document desired by a user.

  A document search method according to one aspect of the present invention that achieves the above object is based on a step of creating a document index based on a large number of document files stored in a first database of a search destination, and on a query of a search source Setting a search index; and collating the search index with the document index to extract a document file corresponding to the query, and creating the document index includes: A step of extracting a document group created by the same person, a step of analyzing each document of the document group, specifying a word used in the document, and one of the document groups Obtaining a degree of appearance of a word specified in a document appearing in another document; Raise the weights for higher word than value, characterized in that it comprises the steps of: setting a weighting of the document index (claim 1).

  According to this method, a document group created by the same person is extracted from the first database, and a degree of appearance in which a word specified in one document of the document group appears in another document. Is required. Generally, a person who describes many documents related to a certain matter (word) can be estimated to be an expert on that matter. Such a document is highly reliable and can be expected to be a complete document. Therefore, when a query including the word is given by increasing the weight for a word that frequently appears in a document related to the creation of the same person and creating a document index, the document file related to the creation of the person, That is, it is possible to increase the similarity of document files related to the creation of experts and to facilitate extraction. Thereby, the extraction probability of the important document desired by the user can be improved.

  In the above method, in the step of specifying the word, it is preferable that the specified word is a noun word. A noun word often expresses that it is directly connected to the matter as compared to other parts of speech. Therefore, this method contributes to the creation of an accurate document index.

  In this case, in the step of setting the weighting, it is desirable to further increase the weighting for the noun word included in the sentence including the connection particle indicating the cause or the reason (Claim 3). For example, a sentence including a connection particle indicating a cause or reason such as “... so” and “... so” is highly likely to contain a main description of a certain matter. Therefore, by increasing the weight of noun words included in such a sentence, it is possible to easily extract a document file including such a sentence.

  Alternatively, in the step of setting the weighting, it is desirable to further increase the weighting for the noun word existing immediately before the connection particle indicating the cause or reason. In general, there are many cases in which a noun word that is a key of a matter is arranged immediately before a connection particle indicating a cause or reason. Therefore, by increasing the weight for such noun words, it is possible to easily extract a document file including such noun words.

  In any one of the above methods, the query of the search source is one question document posted to a second database different from the first database, and in the step of setting the search index, the search index includes: The question document can be created based on a word identified by document analysis. According to this method, it is possible to extract useful answer documents that are latent in the first database, starting with the user posting a question document in the second database.

  Alternatively, the search source query further includes one or a plurality of answer documents posted to the second database with respect to the question document, and in the step of setting the search index, the search index includes the question document The document and the answer document can be created based on the word identified by document analysis (claim 6).

  In general, the questioner often has little knowledge about the question event, is difficult to recall an appropriate keyword, and the words used in the question document are often not appropriate. On the other hand, the respondent has relatively detailed knowledge about the question event (and therefore submits the answer document voluntarily), and an appropriate keyword for the question event is likely to be used in the answer document. Therefore, by creating a search index based on the answer document, it is possible to improve the accuracy with respect to the question event.

  In this case, in the question document, it is preferable that the search index is created by increasing the weights of noun words included in a sentence including a question or a final particle indicating a question (claim 7). For example, a sentence such as “Is it?” Or a sentence with a final particle indicating the question is likely to contain an essential description of the question that the user wants to know. Therefore, by increasing the weight of noun words included in such a sentence and creating a search index, it is possible to easily extract a document file that includes the content in line with the question.

  Alternatively, in the question document, it is preferable that the search index is created by increasing the weight for a noun word existing immediately before a question or a final particle indicating a question (claim 8). In general, a noun word that is a key to a question event is often placed immediately before a question or a final particle indicating a question. Therefore, by increasing the weight for such noun words, it is possible to easily extract a document file including such noun words.

  In the above method, it is preferable that the appearance degree of the word is obtained in all of the question document and the answer document, the weight of the word is set according to the appearance degree, and the search index is created. . According to this method, since the appearance frequency of words is evaluated through all of the question document and the answer document, a search index having excellent objectivity can be created.

  Alternatively, when there are a plurality of the answer documents, the appearance degree of the word is obtained for all the answer documents, the weights of the words appearing in the answer documents are increased, and the weight of the search index is set. Desirable (claim 10). Words that appear similarly in multiple answer documents are presumed to make a point in that matter. Therefore, an accurate search index can be created by increasing the weighting of such words.

  Furthermore, when the question document includes a title, it is desirable that the search index is created by increasing the weights of noun words included in the title. In many cases, noun words that are the main points of questions are included in the title, so increasing the weight of such noun words contributes to improving the accuracy of the search.

  ADVANTAGE OF THE INVENTION According to this invention, the document search method which can extract exactly the important document which a user desires can be provided. Therefore, it is possible to increase the utility value of the Q & A site existing on the Internet and activate the site.

FIG. 1 is a block diagram schematically showing a hardware configuration of a network system to which a document search method of the present invention is applied. It is a typical flowchart which shows the outline | summary of the document search method of this invention. It is a schematic diagram for demonstrating the creation method of the conventional search index. It is a schematic diagram for demonstrating the creation method of the conventional document index. It is a schematic diagram which shows roughly the document search method of this invention. It is a schematic diagram for demonstrating embodiment of the production method of a search index. It is a schematic diagram for demonstrating embodiment of the production method of a document index. It is a figure of the table format which shows the example of a search model. It is a functional block diagram which shows the function structure of a network system. It is a flowchart which shows operation | movement of a network system. It is a flowchart which shows the detail of search index preparation operation | movement. It is a flowchart which shows the detail of document index preparation operation | movement. It is a schematic diagram which shows the specific example of a document search.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram schematically showing the hardware configuration of a network system S to which the document search method of the present invention is applied. This network system S includes a search engine 1, a Q & A system 2 that accepts posting of question documents and answer documents between users on the Internet (or a local communication network), a database 3 that can be used on the Internet, and a user The terminal device 4 to be used is connected to be able to perform data communication via the Internet IN.

  The Q & A system 2 is a specific web site developed on the Internet, accepts posting of a question document (input of document data, etc.) of one user and posts it, and other This is a community site where a user's answer document, the one user's re-question document, a thank-you document, etc. are posted. Examples of the same type of system currently in operation include “Yahoo! Wisdom Bag (registered trademark)”, “Teach me! Good”, “OKwave”, and the like.

  The database 3 is a database that is connected to the Internet and stores a large amount of various HTML files and image files. Although shown here in a simplified manner, the database 3 shows a set of a large number of websites each having a domain name. As such a database 3, for example, a database accessible through a portal site such as Yahoo (registered trademark) or Google (registered trademark) can be exemplified.

  The terminal device 4 is a communication terminal 41, 42, 43, 44, 45,... 4n such as a personal computer, a mobile phone, or a portable information terminal held by a large number of users registered as members in the Q & A system 2. It is. The terminal device 4 can access various websites of the database 3 via the Internet IN. A user of the Q & A system 2 posts a question document to the Q & A system 2 via his / her communication terminals 41 to 4n, posts an answer document to the question document, or appropriately responds to the answer document. It is possible to post various comment documents or simply browse these documents.

  The search engine 1 is a server device that is connected to both the Q & A system 2 and the database 3 so as to be capable of data communication, and can execute document search processing, index creation processing, document analysis processing, and the like. The search engine 1 uses a database 3 or a Q & A system including a document or the like (hereinafter sometimes referred to as a “BEST document”) including an optimal answer content for one question document posted by a user to the Q & A system 2. 2 and the database 3 are searched.

  FIG. 2 is a schematic flowchart showing an outline of a document search method executed by the search engine 1. The search engine 1 extracts one question document and one or a plurality of answer documents posted in response to it from the database provided in the Q & A system 2, and first morphologically analyzes these documents. Is performed (step S1). The document to be analyzed may be only the answer document, but it is more desirable to target both the question document and the answer document from the viewpoint of widely picking up words useful for the search.

  Next, the search engine 1 creates a search index based on the words obtained by the document analysis process (step S2). This search index is mainly composed of a group of words closely related to the question event, and each word is given a predetermined weight. Then, using the search index, a document search process is performed on the database 3 including a large amount of data (step S3). A database provided in the Q & A system 2 may be included in the search processing target.

  On the database 3 side as well, document analysis (step S01) and document index creation (step S02) for metadata extraction are performed by periodic processing by the search engine 1. This document index consists of a group of words extracted from a document file, and each word is given a predetermined weight. In the search process in step S3 of the search source, the search index and the document index of the search destination are used, and a document file having a high similarity to the query is extracted from the database 3 by applying a predetermined search algorithm. The result obtained by the search process in step S3 is output to the terminal device 4 of the user who posted the first question document (step S4).

  The above is the outline of the document search method according to the present embodiment. Here, for comparison with the present embodiment, a conventional method for creating a search index and a document index will be described. FIG. 3 is a schematic diagram for explaining a conventional search index creation method, and FIG. 4 is a schematic diagram for explaining a conventional document index creation method.

  FIG. 3 shows a specific example of a question document and an answer document posted to the Q & A system 2. Here, one question document about a question event that the question title is “the modem lamp goes off when a modem is connected to a personal computer” and three answer documents are illustrated. Here, an example is shown in which the answer to the question document is extracted not only from the original answer document posted to the Q & A system 2 but also from a document file group (answer document candidate) in the general-purpose database 3.

  A conventional technique for creating a search index is a method in which, for example, a noun word is extracted from a question document, and the appearance frequency of the word in the question document is created as a weight. For example, paying attention to the word “modem” in the question document, this word appears four times in the question document, so a search index is created with its frequency (weight) = 4. For example, the word “USB” appears only once in the question document, so weight = 1.

  As shown in FIG. 4, the document index is created in the same manner. That is, for example, a noun word is extracted from each document file of answer document candidates included in the database 3, and a document index is created with the appearance frequency of the word in the document file as a weight. In FIG. 4, for example, the word “modem” appears three times and the word “driver” appears once in “A's document”, which is the answer document candidate. Accordingly, the document index is created with the frequency (weight) of “modem” = 3 and the weight of “driver” = 1.

  The matching index between the question document and the answer document candidate is calculated by fitting the search index and the document index thus created to a document matching score calculation formula (search algorithm) called a search model. Then, a list of hit documents is created in descending order of the degree of match.

  Even with such a document search method that employs weighting that depends only on the word appearance frequency, a certain degree of document extraction accuracy can be ensured. However, there is a limit to increasing the extraction accuracy of the BEST document that the questioner really wants and leads to problem solving. One of the factors is that regarding an event related to a question, a document created by an expert about the event and a document created by an amateur are treated in the same line if the appearance frequency of words is the same. Naturally, a document related to the creation of an expert has a much higher probability of being a BEST document for a questioner, but the conventional search method cannot substantially distinguish the two.

  The search index and document index creation method according to the embodiment of the present invention is characterized in that such a problem is solved and weighting is performed to improve the probability that the user can extract the answer document that the user really wants. FIG. 5 is a diagram schematically illustrating a document search method according to an embodiment of the present invention.

  In the document search method of this embodiment, the document analysis is performed on the question document Qn generated in the Q & A system 2 and the answer document An that responds thereto, and the search index SI is created. This search index SI is a search index to which a special weight created in consideration of other elements is added instead of weighting depending only on the word appearance frequency as in the conventional method. Further, a document index DI is created for the document file in the database 3. This document index is also a document index to which special weighting is added in consideration of other factors, not weighting depending solely on the word appearance frequency. The search index SI and the document index DI are applied to the search algorithm M to calculate the degree of match, and the BEST document for the question document is extracted from the database 3.

In the present embodiment, the following items (1) to (8) are considered as elements of the special weighting of the index.
[Search index SI]
(1) Increase the weighting of noun words included in the title of the question document.
(2) The word appearance frequency is calculated including not only the question document but also all the answer documents.
(3) In a question document, weighting is increased for noun words included in a sentence having a question or a final particle indicating a question.
(4) In a question document, weighting is increased for a noun word existing immediately before a question or a final particle indicating a question.
(5) Increase the weight for words appearing in a plurality of answer documents.
[About Document Index DI]
(6) Weighting is increased for noun words included in sentences having a connection particle indicating the cause or reason.
(7) Increase the weight of a noun word that exists immediately before the connection particle indicating the cause or reason.
(8) In a document group related to creation of the same person, weighting is increased for words used in a plurality of documents.
(9) Increase the weighting of noun words included in the document title.

  FIG. 6 is a schematic diagram for explaining an embodiment of a method for creating a search index SI. The question title, question document, and answer document illustrated here are the same as those in FIG. The reason for setting item (1) above is that the noun word that is the main point of the question is generally included in the question title, so increasing the weighting of such noun word is considered to contribute to improving the accuracy of the search. is there. In the example of FIG. 6, for example, a noun word “modem” indicated by reference numeral 51 appears in the question title. When this “modem” appears once, it is not simply counted as frequency = 1, but the weight of the noun word is counted slightly higher to reflect the importance of the noun word in the search index SI.

  The item (2) is an item intended to create a search index having excellent objectivity by evaluating the appearance frequency of words through all of the question document and the answer document. Generally, in the Q & A system 2, a questioner who does not have detailed knowledge about a question event often cannot set an appropriate keyword. On the other hand, since the respondent is assumed to have appropriate knowledge, there is a high possibility that an answer document is created using an appropriate keyword for the event. On the other hand, a word that appears frequently in both the question document and the answer document can be evaluated as having extremely high importance in the event. In the example of FIG. 6, for example, the word “modem” appears not only in the question title and the question document but also in all of the answer documents 1 to 3. Therefore, this “modem” is given high weight.

  The above item (3) includes, for example, a sentence with a final particle indicating a question or a question such as “Is it?”, “Is it?”, “Is it?” This is an item based on the assumption that there is a high possibility that an essential description of the question that the user wants to know is included. In the example of FIG. 6, a sentence 55 including the word “Is it?” Indicated by reference numeral 54 is “Is a modem manufactured by B company not supported?” , 52 and 53, the weights of the noun words “modem”, “Company B”, and “support” are increased. As a result, it is possible to easily extract a document file that includes the content of the question. In this example, the weight of the noun word “modem” is increased in a superimposed manner in addition to the increase due to the items (1) and (2).

  In the above item (4), there are many cases where a noun word as a key of a question event is arranged immediately before a question or a final particle indicating a question. This is an item intended to facilitate extraction of a document file containing words. In the example of FIG. 6, in the sentence 55, the noun word “support” of reference numeral 53 positioned immediately before the word “Is it?” Indicated by reference numeral 54 corresponds. Therefore, “support” is given a higher weight in addition to the weight by the item (3).

  The above item (5) is an item based on the presumption that a word that appears in the same manner in a plurality of answer documents makes a point in the matter. As described above, it is assumed that the answer document is created by a person who has appropriate knowledge about the question event. Therefore, the words that frequently appear in such an answer document may represent the main point of the question event. It can be said that it is expensive. In the example shown in FIG. 6, the noun word “driver” indicated by reference numeral 56 appears in the answer document 1 and the answer document 2, and the weight is increased for this “driver”.

  FIG. 7 is a schematic diagram for explaining an embodiment of a document index DI creation method. In the above item (6), for example, a sentence including a connection particle indicating the cause or reason such as “... so”, “... so,” includes a main description of a certain matter. It is an item based on the estimation that there is a high possibility that FIG. 7 illustrates one document 60 related to the creation of “Mr. A” as one user and another document group 60 </ b> A related to the creation of the same “Mr. A”. Document files of these documents are stored in the database 3. Focusing on the document 60, the sentence 62 including the connection particle “for” indicated by reference numeral 61 is “because the personal computer manufactured by company A does not support the modem manufactured by company B as a standard”. The document index DI is increased by increasing the weights of the noun words “modem” and “support” indicated by the reference numerals 63 and 64 included in the sentence 62, and the noun words such as “Company A”, “Company B”, and “PC”. Created. Thereby, when the search index SI includes the word “modem” with high weight, for example, the document 60 can be easily extracted.

  In item (7) above, there are many cases where a noun word that is the key to a matter is placed immediately before the connection particle indicating the cause or reason, such as “...” above. Item. In the document 60 of FIG. 7, in the sentence 62, the noun word “support” of reference numeral 64 positioned immediately before the connection particle “for” indicated by reference numeral 61 corresponds. Therefore, “support” is given a higher weight in addition to the weight by the item (6). If the document index DI is created in this way, the matching rate with the search index SI created according to the rule of item (4) above becomes high.

  The item (8) is an item based on the assumption that a person who describes many documents related to a certain matter (word) is an expert on the matter. As shown in FIG. 7, “Mr. A” uses the noun words “modem” and “driver” in one document 60 stored in the database 3 as indicated by reference numerals 63 and 65, respectively. Also, in the other document group 60A related to the creation of “Mr. A” stored in the database 3, the noun word “modem” indicated by reference numeral 63 is all of the documents 601 to 603 and the noun word “driver” indicated by reference numeral 65. "Is also used in documents 602 and 603.

  From these documents, a person named “Mr. A” can be estimated as an expert on matters in which the terms “modem” and “driver” are widely used. Highly credible and can be expected to be a complete document. Such a document is likely to be an important document that leads to the resolution of a question event. Therefore, the document index is created by increasing the weights of the noun words “modem” and “driver”. Thus, when a query (search index SI) including the noun words “modem” and “driver” is given, the document file related to the creation of the person, that is, “Mr. A” who is estimated to be an expert It is possible to increase the degree of similarity of document files related to creation and to facilitate extraction.

  The item (9) is based on substantially the same viewpoint as the item (1). In general, a document title includes a noun word that is a gist of the content of the document, and the noun word is increased in weight. This is because there is a high possibility that it contributes to improving the accuracy of search.

  Next, FIG. 8 is a table format diagram showing an example of a search model. Here, a search model applicable as the search algorithm M shown in FIG. 7 is illustrated. The most popular vector space model of this kind is a cosine measure. According to the research of the present inventor, when a query including a plurality of words is given to a cosine measure, the degree of matching tends to increase as the document matches more types of words. That is, there is a tendency to evaluate that a document in which words are evenly matched satisfies the query.

  However, in a Q & A system that tries to obtain a response document for a certain type of problem, a document that matches a specific key word is more useful than a document that evenly matches the word. There are many. In consideration of this point, when an existing search model is tried, the Dice coefficient is high in the degree of satisfying a query if the matching degree of a specific word is high even if there are few types of matching words for a certain document. In other words, the present invention is a search model that can be treated as a higher-ranked document as the match rate between words with higher weights is higher than when the words are evenly matched. Found. Therefore, in the present embodiment, it is preferable to use the Dice coefficient model as a search algorithm from the viewpoint of improving the search accuracy. Of course, other search algorithms can be preferably used as long as a document having a high match rate between words having high weights can be evaluated as a document having a high rank.

  Subsequently, a detailed configuration of the network system S according to the present embodiment will be described. FIG. 9 is a functional block diagram showing a functional configuration of the network system S. As shown in FIG. The search engine 1 is a large computer device including a CPU (Central Processing Unit) that performs the document analysis process and the search process, for example. The CPU operates to include the functional unit shown in FIG. 9 by executing software programmed to perform the above processing. The search engine 1 functionally includes a first processing unit 11, a second processing unit 12, a search processing unit 13, and an output processing unit 15.

  As described above with reference to FIG. 1, the network system S includes the Q & A system 2, the database 3 (first database), and the terminal device 4 in addition to the search engine 1 described above. The Q & A system 2 includes a Q & A site 21 where a user inputs a question document, an answer document, and the like, and a Q & A document database 22 (second database) that stores the question document, the answer document, and the like. The database 3 includes a plurality of websites (web servers) 31, 32,... 3n in which various document data are stored. The terminal device 4 includes an operation unit 41 including a keyboard and a mouse, a display unit 42 including a liquid crystal display panel, and an output unit 43 including a printer.

  The first processing unit 11 of the search engine 1 performs a process of creating a search index based on a question document (query), an answer document, or the like as a search source. The first processing unit 11 includes a document extraction unit 111, a document analysis unit 112, and a search index creation unit 113.

  The document extraction unit 111 extracts a pair of one question document stored in the Q & A document database 22 of the Q & A system 2 and one or more answer documents posted to the Q & A site 21 in association with the question document. To do. The document extraction unit 111 may capture a document of the entire thread formed on the Q & A site 21 starting from the question document, or may capture a document that is directly answered to the question document. .

  The document analysis unit 112 performs a document analysis including at least a morphological analysis on the question document and the answer document extracted by the document extraction unit 111 and performs a process of extracting independent words. Specifically, the document analysis unit 112 performs normalization processing, document structure analysis processing, and the like for each analysis target document. The normalization process is a process for deleting characters and symbols unnecessary for analysis from the analysis target document and unifying full-width and half-width characters so that the document structure analysis can be performed normally. In the document structure analysis process, the classification target documents after the normalization process are each subjected to a morphological analysis to divide the document into words, and a parsing process to perform dependency between words (nouns and verbs). And the like. For such document structure analysis processing, the document analysis unit 12 utilizes a thesaurus (synonym dictionary) that absorbs synonyms and notation fluctuations.

  The search index creation unit 113 performs processing for creating a search index from the words extracted by the document analysis unit 112. For example, when the document analysis unit 112 extracts “modem”, “driver”, “support”, and the like as independent words, the search index creation unit 113 sets the weighting based on the appearance frequency of these words and the items described above. A search index is created by adding the weights added according to (1) to (5). This search index is stored in a memory unit (not shown).

  The second processing unit 12 analyzes a document file serving as a search destination, that is, a document file stored in the database 3, and performs processing for creating a document index. The second processing unit 12 includes a DB document extraction unit 121, a DB document analysis unit 122, and a document index creation unit 123.

  The DB document extraction unit 121 extracts document files stored in the database 3 (websites 31, 32,... 3n) in association with user IDs and the like in units of document groups related to creation of the same person. . This extraction is periodically performed at a predetermined cycle.

  The DB document analysis unit 122 integrates and edits documents existing in a plurality of websites 31, 32,... 3n to share data, and normalizes processing, morphological analysis, and synonyms for these documents. Performs document analysis processing such as word processing. This process is the same as that of the document analysis unit 112 described above. Through such document analysis processing, the document title, author, and independent words (keywords) are derived.

  The document index creation unit 123 performs processing for creating a document index from the words extracted by the DB document analysis unit 122. For example, when the DB document analysis unit 122 extracts “modem”, “driver”, “support”, and the like as independent words, the document index creation unit 123 describes the weighting based on the appearance frequency of these words and the above description. The document index is created by adding the weights added according to the items (6) to (9). This document index is stored in a memory unit (not shown).

  When setting the weighting of the item (8), the document index creation unit 123 identifies the word used in each document of the document group related to creation of the same person, and the word identified in one document The degree of appearance appearing in another document is obtained. In the example of FIG. 7, it is determined whether the word “modem” that appears in the document 60 also appears in the other documents 601 to 603, and the degree of appearance is determined. Then, the weight of the document index is set by increasing the weight for the word whose appearance degree is higher than a predetermined value. In FIG. 7, since the word “modem” appears in all of the other documents 601 to 603, the weight of “modem” is set to a correspondingly high value.

  The search processing unit 13 searches the database 3 as a search target and applies the search index created by the search index creation unit 113 and the document index created by the document index creation unit 123 to the search algorithm to calculate the degree of match. Process. Examples of the search algorithm include a cosine scale, a Dice coefficient, a Jaccard coefficient, a T score, a mutual information amount, and a Simpson coefficient. As described above, a preferable search algorithm is a Dice coefficient. The search processing unit 13 extracts a document file title or the like that is determined by the search process to have a matching degree higher than a predetermined value.

  The output processing unit 14 creates a list of documents (recommended documents) hit by the search of the database 3 by the search processing unit 13. This list is a list arranged in descending order of matches to the query, and is transmitted to the terminal device 4 of the user who posted the question document.

  A list of search hit documents (recommended documents) transmitted from the output processing unit 14 is displayed on the display unit 42 of the terminal device 4. The user can perform an operation of giving a selection instruction to a desired document from the list by using the operation unit 41. When a selection instruction is given, the terminal device 4 directly accesses the database 3 to browse (output request) the selected document and display the content on the display unit 42. Further, when a print request is given from the operation unit 41, the document is printed from the output unit 43.

  The processing operation of the search engine 1 described above will be described based on the flowchart shown in FIG. A question document is extracted from the Q & A document database 22 by the document extraction unit 111 of the first processing unit 11, and the document analysis unit 112 sequentially performs normalization processing (step S 11) and morphological analysis on the question document. Processing (step S12) and synonym processing (step S13) are executed.

  If an answer document is recorded in the Q & A document database 22 in association with the question document, the answer document is also normalized (step S14), morphological analysis process (step S15), and synonymous. Word processing (step S16) is executed. As a result, words that can be search keywords for the question document and the answer document are extracted. The extraction of the question document and the answer document by the document extraction unit 11 may be performed every time the document is input to the Q & A site 21 or may be scheduled (for example, every day at 24:00).

  Next, the search index creation unit 113 adds a weight based on the appearance frequency of the word and the above items (1) to (5) based on the word extracted by the previous document analysis process, and performs a search. An index is created (step S18). This search index is temporarily stored in a memory unit (not shown).

  On the other hand, on the second processing unit 12 side, an index of a document included in the database 3 is created in parallel by a regular process. That is, the DB document extraction unit 121 extracts document files from the database 3 for each user ID, and the DB document analysis unit 122 performs integration / editing processing in order to share the document data in the database 3 (step S21). ). Then, the DB document analysis unit 122 performs document analysis processing including normalization processing (step S22), morpheme analysis processing (step S23), and synonym processing (step S24) on these documents.

  Then, the document index creation unit 123 creates a document index based on the words extracted by the document analysis process (step S25). This document index is created based on the word extracted by the previous document analysis process, with the appearance frequency of the word and weighting based on the items (6) to (9). The created document index is stored in a memory unit (not shown) (step S26).

  Subsequently, the search processing unit 13 performs a document search for the document index created in step S25 using the search index created in step S18 (step S31). A list of documents (recommended documents) hit by the document search is created by the output processing unit 14 (step S32). Then, the list is transmitted to the terminal device 4 of the user who posted the question document (step S33).

  When a selection instruction is given to a desired document from the list by the operation unit 41 of the terminal device 4 (step S34), the terminal device 4 requests the database 3 to output the selected document. In response to the output request, the contents of the selected document are displayed on the display unit 42, or the document is printed by the output unit 43 (step S35).

  FIG. 11 is a flowchart showing an example of the search index creation process (operation of the search index creation unit 113) in step S17 of FIG. First, the counter q corresponding to the question request number given for each posting of each question document is set to q = 0 (step S41). Thereafter, the counter q is incremented to q + 1 (step S42), and the document analysis data of the qth question document and answer document are acquired (step S43).

  Subsequently, the noun word and final particle are specified for each question document and answer document (step S44), and the appearance degree of the noun word is derived (step S45). The value of the degree of appearance obtained by this processing is used for weighting based on the item (2). Further, a sentence including a final particle such as “...?” And a positional relationship between the final particle and the noun word are specified (step S46). Data obtained by this processing is used for weighting based on the above items (3) and (4).

  Next, it is determined whether or not the number of answer documents exceeds 1 (step S47). When there are two or more (YES in step S47), the appearance degree of noun words among a plurality of answer documents is derived (step S48). The value of the degree of appearance obtained by this processing is used for weighting based on the item (5). If the number of answer documents is 1 or less (NO in step S47), step S48 is skipped.

  Thereafter, it is confirmed whether or not a title exists in the question document (step S49). If there is a “question title” as illustrated in FIG. 6, the noun word used in the title is specified (step S50). Data obtained by this processing is used for weighting based on the item (1). If the title does not exist (NO in step S49), step S50 is skipped.

  Thereafter, weighting is set for each noun word (step S51). Of course, words other than noun words may be added to the search index, and weights may be given to them. The details of the weighting are as described in the items (1) to (5). Then, the related data of the search index including this weighting is recorded in the memory (step S52). Thereafter, it is determined whether or not the counter q is final (step S53). If it is final (YES in step S53), the process is terminated. On the other hand, if it is not final (NO in step S53), the process returns to step S42, and the same processing is repeated for the q + 1th question request.

  FIG. 12 is a flowchart showing an example of the document index creation process (operation of the document index creation unit 123) in step S25 of FIG. First, the counter p corresponding to the creator ID assigned to each contributor of the document to the database 3 is set as p = 0 (step S61). Thereafter, the counter p = p + 1 is incremented (step S62), and the process for the p-th creator is performed as follows.

  A counter d corresponding to the document ID assigned to each document posted by the creator to the database 3 is set as d = 0 (step S64). Thereafter, the counter d = d + 1 is incremented (step S64), and the d-th document file data for the creator is acquired (step S65).

  Subsequently, a noun word and a connection particle are specified for the d-th document file (step S66), and the appearance degree of the noun word is derived (step S67). The value of the degree of appearance obtained by this processing is used for weighting based on the item (8). Further, a sentence including a connection particle such as “... For” is specified, and the positional relationship between the connection particle and the noun word is specified (step S68). Data obtained by this processing is used for weighting based on the above items (6) and (7).

  Thereafter, it is confirmed whether or not a title exists in the d-th document file (step S69). If a title exists in the document file (YES in step S69), the noun word used in the title is specified (step S70). Data obtained by this processing is used for weighting based on the item (9). If the title does not exist (NO in step S69), step S70 is skipped.

  Thereafter, primary weighting is set for each noun word (step S71). Details of the weighting are as described in the items (6), (7), and (9). Then, the related data of the document index including this weight is recorded in the memory (step S72). Subsequently, it is determined whether or not the counter d is final (step S73). If it is not final (NO in step S73), the process returns to step S64 and the same processing is repeated for the d + 1-th document file. It is.

  On the other hand, if the counter d is final (YES in step S73), it is confirmed whether or not there is a word that is used in common among all the document files posted by the p-th creator. (Step S74). This is processing corresponding to the item (8). If there is a common word (YES in step S74), secondary weighting is set for the word (step S75). Details of this weighting are as described in the above item (8) and FIG. Then, the related data of the document index including this weight is recorded in the memory (step S76).

  Thereafter, it is determined whether or not the counter p is final (step S77). If it is final (YES in step S77), the process is terminated. On the other hand, if it is not final (NO in step S77), the process returns to step S62, and the same processing is repeated for the document file related to the creation of the (p + 1) th creator.

  FIG. 13 is a diagram schematically illustrating a specific example of document search based on the present embodiment. The search index SI created based on the search source document (question document and answer document) existing in the Q & A system 2 is 1.023, 0.221, and 6.809 for word a, word b, and word c, respectively. Weighting is given. Further, the document index DI created based on the search destination document (document file) existing in the database 3 is also the word a, the word b, the word for the document 1, the document 2, the document 3,. A weight is assigned to c.

  Such a search index SI and a document index DI of each document are input to a document matching degree calculation formula according to a Dice coefficient search model. Then, the degree of coincidence is calculated for each of document 1, document 2, document 3,. Then, the documents are sorted in descending order of the degree of match, and ranking display data of hit documents is created. In the example of FIG. 13, a Dice coefficient that has a high degree of match when the match rate of a word with a high weight is high may be employed, and since the search index SI has a high weight for the word c, the word c As a result, the document 2 in which the weight of “1” protrudes has the highest degree of match.

  According to the network system S according to the present embodiment described above, it is possible to provide a document search method capable of accurately extracting an important document desired by a user not only from the Q & A system 2 but also from a large-scale database 3. Can do. Therefore, the utility value of the Q & A site 21 existing on the Internet can be increased and the site can be activated.

S network system 1 search engine 11 first processing unit 111 document extraction unit 112 document analysis unit 113 search index creation unit 12 second processing unit 121 DB document extraction unit 122 DB document analysis unit 123 document index creation unit 13 search processing unit 14 output Processing unit 2 Q & A system 21 Q & A site 22 Q & A document database 22 (second database)
3 Database (first database)
4 Terminal device 41 Operation unit 42 Display unit 43 Output unit

Claims (11)

Creating a document index based on a number of document files stored in a first database of search destinations;
Setting a search index based on the query from the search source;
Collating the search index with the document index to extract a document file according to the query,
The step of creating the document index includes:
Extracting a document group created by the same person from the first database;
Analyzing each document of the document group and identifying words used in the documents;
Obtaining a degree of appearance of a word specified in one document of the document group appearing in another document;
Increasing the weight for words whose appearance degree is higher than a predetermined value, and setting the weight of the document index;
A document retrieval method comprising:   The document search method according to claim 1, wherein in the step of specifying the word, the specified word is a noun word.   3. The document search method according to claim 2, wherein in the step of setting the weighting, the weighting is further increased for the noun word included in the sentence including the connection particle indicating the cause or the reason.   3. The document search method according to claim 2, wherein, in the step of setting the weight, the weight is further increased for the noun word existing immediately before the connection particle indicating the cause or the reason. The query of the search source is one question document posted to a second database different from the first database,
5. The document search method according to claim 1, wherein in the step of setting the search index, the search index is created based on a word specified by document analysis of the question document. . The query of the search source further includes one or more answer documents posted to the second database for the question document;
5. The search index according to claim 1, wherein in the step of setting the search index, the search index is created based on a word specified by document analysis of the question document and the answer document. Document search method.   The document search method according to claim 5 or 6, wherein in the question document, the search index is created by increasing the weight of noun words included in a sentence having a question or a final particle indicating a question. .   The document search method according to claim 5 or 6, wherein in the question document, the search index is created by increasing the weight of a noun word existing immediately before a question or a final particle indicating a question. Obtaining the appearance degree of the word in all of the question document and the answer document;
The document search method according to claim 6, wherein weighting of the word is set according to the appearance degree, and the search index is created. In a case where there are a plurality of the answer documents, obtain the degree of appearance of the word in all the answer documents,
The document search method according to claim 6, wherein the weight of the search index is set by increasing the weight of words appearing in the plurality of answer documents.   7. The document search method according to claim 5, wherein, when the question document includes a title, the search index is created by increasing a weight for a noun word included in the title. 8.

Images