JP4146393B2 - Label display type document search apparatus, label display type document search method, computer program for executing label display type document search method, and computer readable recording medium storing the computer program - Google Patents

Description

  The present invention relates to a label display type document search apparatus that displays a document label and then displays the document.

  2. Description of the Related Art Conventionally, a label display type document retrieval apparatus that displays a document label once and then displays the document is used as a retrieval apparatus that retrieves a document via a computer network. This eliminates the need for the user to evaluate the entire document content, thereby enabling the desired information to be reached efficiently.

  In order to realize this, the following methods are considered.

・ Classification system usage method Create classification categories by hand in advance and provide documents that are considered appropriate for each category as learning data. By determining the characteristics of the category based on the characteristics of the learning data, and when the search results are given, the similarity between each search result and the category is calculated and related to the category that seems to be the most similar. The results can be classified and presented to the user.

-Document similarity utilization method In advance, a feature expression of each document is acquired using a vector expression or the like. When a search result is given, similar documents are classified into the same classification based on the feature expression, and are presented in a number that is considered to be effective for the searcher.

・ Characteristic term usage method Similar to classification of search results by acquiring characteristic words, compound words, keywords, etc. in search results from documents and data of search results and presenting them as narrowing candidates Present.
JP 2003-203136 A

  However, the above prior art has the following problems.

  The classification system utilization method is a method in which a plurality of labels are manually created in advance, and a document that matches the meaning of the corresponding label is associated with the label, and the correct data for label generation and label meaning is used. The creation is presumed to be performed manually, and there is a problem that the definition of the initial label and the maintenance of the label accompanying the document update become a large cost for the administrator of the information retrieval system.

  In the document similarity utilization method, the trade-off between document classification quality and speed must be considered due to processing time constraints. The technique used there is a technique for determining the number of classifications in advance, such as the K-Means method. This is because if the number of actual topic classifications does not match the determined value, an ambiguous classification will be generated, making it difficult to label the contents of each classification, and the classification of the generated labels at a glance. There is a problem that it may become unclear that the content of the contents cannot be grasped.

  In order to cope with these problems, there is a movement to use a characteristic term utilization method in a scene such as retrieval of a web document in recent years. In this case, by using a word that co-occurs in the query and the document, the user can efficiently expand the search expression, thereby easily narrowing down the search result.

  However, this method is a method that uses only some features of a document such as a keyword, and therefore, one document is often classified from a plurality of viewpoints. Therefore, when the search results are classified using this method, the classification is non-exclusive, and one document is associated with a plurality of classifications. This has the merit of being able to judge document contents from various viewpoints, but there are cases where the number of classifications is excessive, pointers to the same document group even with different labels, and effective document narrowing cannot be performed. is there.

  The present invention has been made in view of the above, and an object of the present invention is to provide a label display type document search apparatus capable of efficiently narrowing down search results.

In order to solve the above problems, the first aspect of the present invention is a document including a body composed of a character string, further including a title of the document and document identification information indicating the document, and a predetermined character string. The document storage unit storing a plurality of documents whose attribute values are included in the body, and the plurality of documents having the attribute value for each attribute value included in at least one of the plurality of documents stored in the document storage unit Statistical information that records the number of appearances in the memory, and stored in a storage means provided in advance, a document search means for searching a plurality of documents from the document storage means, and a plurality of searched documents Search result statistical information generating means for generating and storing search result statistical information in which the number of occurrences of the attribute value in the plurality of documents is recorded for each attribute value included in at least one of the plurality of searched documents of For each attribute value included in at least one of them, for calculating a fitness indicating the degree to which the attribute value is suitable as a label that is a character string representing a plurality of documents forming a part of the plurality of documents, Using the calculation formula using the number of appearances in the statistical information and the search result statistical information of the attribute value, the fitness calculation means for calculating the fitness of the attribute value, and the plurality of retrieved documents The attribute value corresponding to the fitness level is selected as long as the fitness level satisfies a preset condition from the higher fitness level corresponding to each of a plurality of attribute values including attribute values included in at least Label information generating means for selecting a plurality of attribute values forming a part of the plurality of attribute values, using the selected plurality of attribute values as labels, and generating label information including the plurality of labels, For each label included in the label information, document identification information and titles indicating documents that include the character string that is the label and that are any of the plurality of retrieved documents are included for the number of the documents. Cluster information generating means for generating cluster information that includes cluster information and includes the label, and each document identification information in the cluster information including the one label for each of the two labels included in the label information. The ratio of the one set to the union of the one set that is a set of documents and the other set that is the set of documents indicated by the document identification information in the cluster information including the other label. 1 ratio and a second ratio that is the ratio of the other set to the union, and both the first ratio and the second ratio exceed a preset threshold value. If it is determined that the two labels are in an equivalence relationship, and if only one of the two labels exceeds a preset threshold, the label corresponding to the one is included in the label corresponding to the other If it is determined that there is a relationship, and it is determined that there is an equivalence relationship or an inclusion relationship, a label that is a label that includes the two labels and that shows an equivalence relationship or an inclusion relationship between the two labels is the label information. Second cluster information generating means for generating cluster information including the label and the cluster information including the combination of the document identification information and the title included in at least one of the two cluster information, Each label included in the label information is displayed, and when one label is selected, the document identification information and the cluster information including the label are included. Fine title set are respectively displayed, one if the document identification information and the title set is selected, the document display control means for displaying a document containing a set of document identification information and title is read from the document storage means And a label display type document retrieval device characterized by comprising:

A second aspect of the present invention is a label display type document search method performed by a label display type document search device, wherein the label display type document search device includes a document storage unit, a statistical processing unit, a document search unit, and search result statistical information. A generation unit, a fitness calculation unit, a label information generation unit, a cluster information generation unit, a second cluster information generation unit, and a document display control unit, and the document storage unit is a document including a text composed of a character string. A plurality of documents that further include a title of the document and document identification information indicating the document, and a plurality of attribute values that are predetermined character strings are included in the text, and the label display type document search method includes: Statistical information in which the statistical processing unit records the number of appearances of the attribute value in the plurality of documents for each attribute value included in at least one of the plurality of documents stored in the document storage unit. Is generated and stored in a storage unit provided in advance, the document search unit searches for a plurality of documents from the document storage unit, and the search result statistical information generation unit includes the search A search result statistical information generation step for generating and storing search result statistical information in which the number of occurrences of the attribute value in the plurality of documents is recorded for each attribute value included in at least one of the plurality of documents; For each attribute value included in at least one of the retrieved plurality of documents, the fitness level calculating unit is a label that is a character string representing a plurality of documents that form part of the plurality of documents. Using the calculation formula using the statistical information of the attribute value and the number of appearances in the search result statistical information to calculate the degree of suitability indicating the degree of suitability, conformance of the attribute value And the higher the fitness level corresponding to each of a plurality of attribute values consisting of attribute values included in at least one of the retrieved documents. As long as the fitness level satisfies a preset condition, by selecting an attribute value corresponding to the fitness level, a plurality of attribute values forming a part of the attribute values are selected, and a plurality of selected Each of the attribute values is a label, and a label information generating step for generating label information including the plurality of labels, and the cluster information generating means includes a character string that is the label for each label included in the label information Document identification information indicating a document that is one of the retrieved plurality of documents and cluster information including titles as many as the number of the documents and the label A cluster information generating step for generating cluster information including the document information, and the second cluster information generating means, for each of two labels included in the label information, for identifying each document in the cluster information including the one label The ratio of one set to the union of one set that is a set of documents indicated by information and the other set that is a set of documents indicated by document identification information in the cluster information including the other label And a second ratio that is a ratio of the other set to the union, and both the first ratio and the second ratio exceed a preset threshold value. If it is determined that the two labels are in an equivalence relationship and only one of them exceeds a preset threshold, the label corresponding to the one is included in the label corresponding to the other If it is determined that there is an inclusive relationship, and it is determined that there is an equivalence relationship or an inclusive relationship, a label that is a label that includes the two labels and that can know the equivalence relationship or the inclusive relationship between the two labels A second cluster information generation step for generating cluster information including the label, which is included in the information and includes cluster information including a set of document identification information and a title included in at least one of the two cluster information. The document display control means displays each label included in the label information, and a set of document identification information and a title included in the cluster information including the label is selected by selecting one label. If one set of the document identification information and the title is selected, a document including the set of the document identification information and the title is stored in the document storage. With label-type document retrieval method characterized by comprising a document display control step of displaying it is read out from the stage and solutions.

  According to the present invention, a document is retrieved from a document storage unit that stores the document, an attribute value included in the retrieved document is selected as a document label, and label information indicating the selected label is generated and stored. , Generating a label that integrates the labels in the equivalence relation in the label information and including the labels in the label information; and / or generating a label that integrates the labels in the inclusion relation and including the labels in the label information. When the label information including the integrated label is read and the label is displayed by the label information, and the integrated label is designated, at least one of the labels before the integration is included and also included in the searched document. The document to be read is read from the document storage means and displayed, so there is no need to prepare a label in advance and the user can use the integrated label. Selection is facilitated. Therefore, the search result can be narrowed down efficiently.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
FIG. 1 is a block diagram showing a device configuration of the first embodiment. From here, it demonstrates, showing the example when a document is made into a news article.

  The retrieval apparatus 1 is a server computer that retrieves a document, and corresponds to a label display type document retrieval apparatus that executes the label display type document retrieval method of the present invention. The search device 1 can communicate with a browser 2 of a client computer connected via a network (not shown).

  The browser 2 includes a keyword input unit 21 for inputting a keyword via an input device such as a keyboard and a mouse, and a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), or the like (not shown) for a document searched by the keyword. And a document display control unit 22 to be displayed on the display device.

  Before displaying the retrieved document, the browser 2 displays the attribute value included in the document as a label and classified by the attribute name of the attribute value. When any label is specified by clicking, the title of the document including the label (attribute value) is displayed, and when any title is specified, the document with the title is displayed. It is like that.

  The search device 1 includes a communication unit 101 that receives a keyword from the browser 2 and transmits the searched document to the browser 2, and a request processing unit 102 that controls document search using the keyword given from the communication unit 101.

  In addition, the search device 1 includes a setting file 103 in which information used for document search is written. In the setting file 103, attribute names “genre” and “organization” used in the search are written. In the setting file 103, the number of labels to be selected is written. The setting file 103 stores parameters α, β, and γ that are used when determining the fitness as the attribute name to be displayed (referred to as attribute name fitness). Various threshold values and the like are written in the setting file 103.

  In addition, the search device 1 includes an attribute value classified as one of the attribute names written in the setting file 103, but no attribute name (tag) is given to the attribute value. A document is input, and a news article manager or the like inputs a document by generating a document by so-called manual tagging, and an untagged document, and the attribute value is automatically tagged. , A document generation unit 105 that generates a tagged document (also simply referred to as a document), and a document database that stores the documents generated by the document generation unit 104 and the document generation unit 105 (hereinafter, the database is referred to as DB). (Abbreviated) 106.

  In addition, the search device 1 includes a normalization unit 107 that normalizes attribute values included in documents stored in the document DB 106.

  Further, the search device 1 associates a word (which may be an attribute value) included in the document stored in the document DB 106 with document identification information (hereinafter, identification information is referred to as ID) indicating the document including the word. The index generation unit 108 that generates the index 1081 and the document search unit 109 that searches the document DB 106 based on the keyword and the index 1081 are provided.

  Further, the search device 1 stores a first statistical information DB 110 that stores first statistical information generated for each attribute name of the setting file 103, and second statistical information generated for each first statistical information. A second statistical information DB 111 and a statistical processing unit 112 that generates first statistical information and second statistical information are provided.

  The search device 1 also includes a label candidate selection unit 113 that selects a plurality of attribute values as label candidates for each attribute name of the setting file 103, and a fitness level (referred to as a label fitness level) when the label candidate is a document label. ), And a label determination unit 115 that determines a label based on the calculated label suitability.

  In addition, the search device 1 includes a cluster information generation unit 116 that generates cluster information for each determined label, and a label integration processing unit 117 that performs processing for integrating two or more determined labels. In the present embodiment, a cluster means one or more searched documents including one label.

[Pre-search process]
Next, processing performed by the search device 1 before searching will be described.

  FIG. 2 is a flowchart showing processing performed by the search device 1 before search.

  The document generation unit 104 receives, for example, an untagged document as shown in FIG. 3, and further, for example, designates that “International Atomic Energy Agency has determined ***” as a title, or has an attribute value “International Atomic Energy Agency”. 4 is classified into the attribute name “organization” and the attribute value “economic” is classified into the attribute name “genre”, as shown in FIG. A document is generated by giving the contents and a document ID such as “001”, for example, and stored in the document DB 106 (S101).

  On the other hand, when an untagged document is input and there is a title designation or the like, the document generation unit 105 generates a document by automatically attaching a tag to the attribute value, and further assigns a document ID. It is stored in the document DB 106 (S101).

  By such processing, a large number of documents are stored in the document DB 106.

  Next, the normalization unit 107 normalizes attribute values included in the document stored in the document DB 106 (S103). Normalization means, for example, conversion of an attribute value “IAEA” expressed by an abbreviation into an attribute value “International Atomic Energy Agency” expressed in Japanese that is not abbreviated.

  In other words, the normalization unit 107 detects attribute values having synonyms with different expressions while having the same meaning in the document, and makes them the same expression.

  Although there are several methods for detecting synonyms, a method using the co-occurrence pattern shown in FIG. 5 can be adopted.

  By such processing, the attribute value of the document in the document DB 106 is normalized.

  Next, the index generation unit 108 generates an index 1081 in which a word included in a document stored in the document DB 106 is associated with a document ID of a document including the word (S105).

  As shown in FIG. 6, in the index 1081, for example, the document ID “001” of the document including the word is associated with the word “nuclear power”.

  Next, the statistical processing unit 112 generates first statistical information for each attribute name of the setting file 103 based on the document DB 106 and stores the first statistical information in the first statistical information DB 110 (S107).

  As shown in FIGS. 7A and 7B, one attribute name is assigned to one first statistical information.

  One piece of first statistical information is information including one or more pieces of information in which a document ID is associated with an attribute value included in a document with this ID and classified as an attribute name.

  FIG. 7A shows that, for example, the document with the document ID “001” includes an attribute value “economic” or the like classified into the attribute name “genre”. FIG. 7B shows that the document with the document ID “001” includes an attribute name “International Atomic Energy Agency” classified into the attribute name “organization”.

  Next, the statistical processing unit 112 generates second statistical information for each first statistical information and stores it in the second statistical information DB 111 (S109).

  As shown in FIGS. 8A and 8B, one second statistical information attribute name is assigned to one second statistical information.

  One piece of second statistical information is information including one or more items in which attribute values classified into attribute names are associated with the number of appearances of the attribute value in the first statistical information DB 110.

  FIG. 8A shows that the number of appearances of the attribute value “economy” classified into the attribute name “genre” is 100, for example. FIG. 8B shows that the number of appearances of the attribute value “International Atomic Energy Agency” classified into the attribute name “organization” is 70 times.

  The second statistical information may be generated by detecting the correspondence between the attribute value and the document ID in the first statistical information and counting up the number of appearances for each detection.

  The second statistical information may be composed of the attribute value itself and the document ID of the document in which the attribute value appears. Moreover, you may comprise 2nd statistical information by the co-occurrence frequency between each attribute value. The co-occurrence frequency at this time can be defined as co-occurring attribute values appearing in the same document. When a document is automatically generated from an untagged document, it can be defined that attribute values included in the same sentence or paragraph in the untagged document co-occur. Further, the co-occurrence relationship is not expressed by a binary value of [0, 1] depending on the proximity in the document, but a large value may be given between attribute values that appear closer together like the co-occurrence. Good.

  In this way, the document search becomes possible after the processing up to S109 is completed. However, when the document in the document DB 106 is updated, added, or deleted, the attribute value normalization, the index 1081, the first statistical DB 110, The second statistics DB 111 and the like are updated.

[Search process]
Next, a search process performed by the search device 1 will be described.

  For example, when the keyword “nuclear power” is input by the user, the keyword input unit 21 transmits the keyword “nuclear power” to the communication unit 101 of the search device 1. A plurality of keywords can be used to perform an AND search, an OR search, or a more complicated condition, but here, for convenience of explanation, only one keyword is used.

  FIG. 9 is a flowchart of processing performed by the search device 1 that has received the keyword.

  First, the communication unit 101 gives the transmitted keyword “nuclear power” to the request processing unit 102, and the request processing unit 102 gives the keyword to the document search unit 109. The document search unit 109 searches for the document ID associated with the keyword “nuclear power” in the index 1081 and returns them to the request processing unit 102 (S201: document search).

  The request processing unit 102 gives the document ID to the label candidate selection unit 113 (S203).

  The label candidate selection unit 113 generates first search result statistical information for each attribute name of the setting file 103 based on the first statistical information DB 110 and the searched document ID, and temporarily stores the first search result statistical information (S205).

  As shown in FIG. 10, one attribute name is assigned to one first search result statistical information.

  One first search result statistical information is a document ID of a document including the attribute value for each attribute value included in the first statistical information, and a document ID included in the searched document ID. It is a correspondence.

  Next, the label candidate selection unit 113 generates and temporarily stores second search result statistical information for each attribute name based on the first search result statistical information (S207).

  As shown in FIG. 11, one attribute name is assigned to one second search result statistical information.

  One second search result statistical information is obtained by associating each attribute value of one first search result statistical information with the number of document IDs associated with the attribute value as the number of appearances.

  Next, the label candidate selection unit 113 generates third statistical information for each second statistical information based on the second search result statistical information to which the same attribute name as the second statistical information is assigned (S209).

  As shown in FIG. 12, one piece of third statistical information is statistical information including one or more rows included in one piece of second statistical information, and the attribute value of the row corresponds to the second search result statistical information. It is also included in the line.

  Next, the label suitability calculation unit 114 generates and temporarily stores label suitability information for each second search result statistical information based on the second search result statistical information, the third statistical information, and the retrieved document ID. (S211).

  As shown in FIG. 13, one attribute name is assigned to one label suitability information.

  One label suitability information is obtained by associating a label suitability with each attribute value included in one second search result statistical information.

  The label suitability is calculated as follows, for example.

The number of applications corresponding to one attribute value in the second search result statistical information is set as h, the number of applications corresponding to the attribute value in the third statistical information is set as d, and the number of retrieved document IDs is set as | H | The label suitability is calculated by equation (1).

  In the equation (1), h / d represents the completeness of the attribute value in the retrieved document, and | H | / h represents the rarity of the attribute value in the retrieved document.

  Also, in equation (1), h / | H | is substituted for h in the first term, and d / | D | (| D | is the number of documents including the attribute value) instead of d in the first term. Also good.

  Next, the label determination unit 115 temporarily stores the label information obtained by subtracting the combination of the attribute value and the label compatibility from the label compatibility information (S213). The label information is generated and stored for each label suitability information. The attribute value in the label information is a label because it is a label of the document.

  As shown in FIG. 14, the label information is obtained by associating the label suitability with the label. By selecting a label from the label suitability higher in the label suitability information, the label information in the label information The number of sets of label suitability is smaller than the number of sets of attribute values and label suitability in the label suitability information.

  FIG. 15 is a flowchart of label selection performed by the label determination unit 115.

  The label determining unit 115 selects the number of labels written in the setting file 103 from the one with the highest label matching degree (S301). Next, it is determined whether or not to additionally select a label whose label suitability is the next point (S303).

Specifically, C (n) is the lowest selected label fitness level, C (n + 1) is the label fitness level one above, and C (n-1) is the label fitness level of the next point. When 2) is established, the label of the next point is additionally selected (S305), and the process returns to S303.

  However, e is a threshold value written in the setting file 103 or the like.

  In other words, the inclination of the value is evaluated, and a determination is made by applying the concept of using the boundary where the inclination exceeds a certain threshold.

  By this method, it is possible to prevent leakage from the selection of the label even though the label matching degree is close. That is, it can be made not to select only when there is a difference in the label suitability.

  In S301, a label may be selected by comparison with a threshold value of the label suitability written in the setting file 103 or the like.

  Next, the label determination unit 115 generates attribute name fitness information based on the label information and temporarily stores it (S215).

  As shown in FIG. 16, the attribute name suitability information indicates the attribute name suitability for each attribute name.

  For example, the attribute name suitability for the attribute name “genre” is calculated as follows.

  First, the number dl of documents including any label in the “genre” label information is obtained from the first search result statistical information of “genre”. At this time, one document including a plurality of labels is calculated as 1.

And comprehensiveness S1 is calculated | required by Formula (3).

  Here, dr is the number of retrieved document IDs.

  The greater the S1, the greater the extent to which the search result is covered by the label.

Next, the degree of overlap and the clarity of classification S <b> 2 are obtained by Expression (4).

Here, dr is the number of retrieved document IDs, dl _i is the number of documents including the i-th label l _i in the “genre” label information, and the second search result statistical information of “genre” It is obtained from.

  The larger S2 is, the greater the degree to which the search result is clearly classified by the label.

Next, the uniformity S3 of the classification is obtained from Equation (5). Here, S3 is obtained by calculating an average entropy of a cluster to be described later.

Here, dr is the number of retrieved document IDs, and dl _i is the number of documents including the i-th label l _i in the label information of “genre”. dl _i can be obtained from the second search result statistical information.

  The larger this S3, the greater the degree to which the search results are uniformly classified by the label.

Next, the attribute name suitability S is obtained by Expression (6).

  Here, α, β, and γ are parameters written in the setting file 103.

  Next, the request processing unit 102 reads the second search result statistical information, the label information, and the attribute name fitness information, and provides the label information to the cluster information generation unit 116.

  The cluster information generation unit 116 generates cluster information for each label included in the label information and temporarily stores it (S217).

  As shown in FIG. 17, the cluster information corresponds to each label included in the label information, the document ID of the document including the label and also included in the retrieved document ID, and the title of the document. It is attached.

  Next, the label integration processing unit 117 performs processing for further integrating the determined labels (S218).

  FIG. 18 is a flowchart showing details of S218. Although the attribute name “genre” will be described here, other attribute names are processed in the same manner.

  First, the label integration processing unit 117 reads the stored label information, second search result statistical information, and cluster information (S311). A set of labels consisting of two labels in the label information is sequentially processed. Subsequently, a set of unprocessed labels is selected (S313). The label integration processing unit 117 determines whether the two labels in the selected label set have an equivalence relationship or an inclusive relationship (S315).

In S315, first, the ratio R _{A / B} and the ratio R _{B / A} are obtained from the equations (7) and (8).

  As shown in FIG. 19, A is a set of documents including one label, B is a set of documents including the other label, and A∩B is a set of documents including both labels. . The set of documents is determined from the cluster information.

Subsequently, when both the ratio R _{A / B} and the ratio R _{B / A} exceed a preset threshold value (for example, set to 0.8, 0.9, etc.), the equivalence relation It is determined that

It is not determined that there is an equivalence relationship, and one of the ratio R _{A / B} and the ratio R _{B / A} exceeds a preset threshold (for example, set to 0.8, 0.9, etc.) If it is, it is determined that there is an inclusion relationship in which the person who exceeds it is included in the person who does not exceed it.

  Next, in the label information, two labels determined to be in an equivalence relationship, for example, one label / the other label are integrated into one label connected by “/”, and the label is generated. Include in information. Further, the attribute values included in the second search result statistical information that are the same as the two labels are also integrated into one attribute value and included in the second search result statistical information (S317). ).

  Next, in the label information, two labels that are determined to be in an inclusive relationship, for example, an inclusive label (included label), which is integrated into a single label that represents the inclusive relationship in () Generate and include in the label information. Further, the attribute values included in the second search result statistical information that are the same as the two labels are also integrated into one attribute value and included in the second search result statistical information (S319). ).

  Next, two pieces of cluster information corresponding to two labels (labels before integration) determined to be in an equivalence relationship or an inclusion relationship are selected, and a combination of these pieces of cluster information is generated ( S321). The integrated cluster information includes the integrated label, and the document ID and title included in at least one of the two cluster information.

  Then, the process returns to step S313, a set of labels that have not been processed is selected, and the same process as described above is performed. Note that the processing in FIG. 18 ends when all label pairs have been processed. Note that the label information and the second search result statistical information including the integrated label and attribute value are stored again. The integrated cluster information is stored together with the original cluster information.

  Returning to FIG. 9, the request processing unit 102 reads all of the stored second search result statistical information, label information, attribute name matching degree information, and cluster information, and gives them to the communication unit 101. It transmits to the browser 2 (S219).

  FIG. 20 is a flowchart of processing performed by the browser 2 to which these pieces of information are transmitted.

  As shown in FIG. 21, the document display control unit 22 of the browser 2 displays document IDs and titles included in all cluster information (S401), and further displays labels included in the label information (S403). At this time, the number of labels to be displayed is reduced depending on the degree of conformity, and since the labels having the equivalence relation or the inclusion relation are integrated, the user can easily instruct the label. Specifically, FIG. 21 shows a newly generated label of international / international relation with a label having an equivalence relation, and a newly generated label of sports (baseball,...) With a label having an inclusive relation. It is shown.

  Then, for the convenience of the user, for example, labels are displayed together for each attribute name. Further, the label of the attribute name having a high attribute name matching degree in the attribute name matching degree information is displayed so as to be easier to see. In addition, for labels included in one label information, an associated label having a high degree of label matching is displayed so as to be easier to see. In addition, the number of document IDs associated with the second search result statistical information is displayed in association with the label.

  Then, when one label is instructed by the user (S405), the document display control unit 22 deletes the displayed document ID and title, and is included in the cluster information including the label as shown in FIG. The document ID and title are displayed (S407). Of course, the label at this time may be an integrated label.

  Then, when the document ID is instructed by the user (S409), the document display control unit 22 transmits the document ID to the communication unit 101 of the search device 1 (S411). In practice, the document ID can be designated by clicking the position of the document ID and the title.

  As shown in FIG. 22, in the present embodiment, the number of document IDs and titles is less after the label instruction than before the instruction, so the user can easily instruct.

  The communication unit 101 of the search device 1 gives the transmitted document ID to the request processing unit 102. The request processing unit 102 gives the given document ID to the document search unit 109. The document search unit 109 reads the document with the given document ID and returns it to the request processing unit 102.

  The request processing unit 102 gives the returned document to the communication unit 101, and the communication unit 101 transmits it to the browser 2.

  The document display control unit 22 of the browser 2 displays the transmitted document.

  When one label is designated in S405, the document display control unit 22 does not display the document ID and title included in the cluster information including the label (S407), but uses this label as an additional keyword. Then, the search device 1 may transmit to the communication unit 101 and use the additional keyword and the previously transmitted keyword to perform S201 and subsequent steps (AND search) in FIG.

  As described above, according to the retrieval apparatus 1, the document retrieval unit 109 that constitutes the document retrieval unit retrieves the document from the document DB 106 that constitutes the document storage unit that stores the document, and the label that constitutes the label information generation unit. The decision unit 115 selects an attribute value included in the searched document as a label of the document, generates and stores label information indicating the selected label, and the label integration processing unit 117 constituting the label information processing unit performs labeling. Generate a label that integrates the labels that have the equivalence relationship in the information and include it in the label information, and / or generate a label that integrates the labels in the inclusive relationship and include it in the label information, and display the document The request processing unit 102 constituting the control means reads the label information including the integrated label and uses the label information to read the label information. If the integrated label is instructed in the browser 2, the document including at least one of the labels before integration and also included in the searched document is read from the document storage means and displayed. , It is not necessary to prepare labels in advance, and the integrated label facilitates selection by the user. Therefore, the search result can be narrowed down efficiently.

  Further, the cluster information generating unit 116 constituting the cluster information generating means generates cluster information indicating one of the selected labels and also included in the searched document, and the cluster information processing means When the integrated label information processing unit 117 generates the integrated cluster information corresponding to the label before the integration, the document display control unit (102) indicates the integrated cluster information when the integrated label is instructed. The presence of the document indicated by (1) is displayed, and when the document is instructed, this document is read from the document storage means and displayed, so that the search result can be narrowed down efficiently.

  In addition, the label suitability calculating unit 114 constituting the label information generating unit calculates the suitability when the attribute value included in the retrieved document is used as a label, and the label determining unit 115 constituting the label information generating unit. The search device 1 is characterized by selecting, as a label, a smaller number of attribute values than the number of the attribute values from the one with a higher fitness.

  Further, the label suitability calculation unit 114 constituting the label information generating means includes the attribute value of the target for calculating the suitability of the label and the number of documents included in the searched document, and the attribute value A feature of the search device 1 is that the fitness is calculated using the number of documents stored in the document storage means.

  The statistical processing unit 112 is a statistical information generation unit that generates statistical information about attribute values included in the document stored in the document storage unit, and the statistical information storage unit stores the generated statistical information. A feature of the search apparatus 1 is that the label suitability calculation unit 114, which includes the first statistics DB 110 and the second statistics DB 111 and constitutes the label information generation means, calculates the suitability of the label using the statistics information. ing.

  In the above embodiment, the equivalence relation and the inclusion relation are determined for two labels, but may be determined for three or more labels.

  Moreover, in the said embodiment, although processed about both equivalence | correspondence relation and inclusion relation, you may process only about one according to necessity.

  A computer program that causes the search device 1 to execute the label display type document search method described in the above embodiment is stored in a computer-readable recording medium such as a semiconductor memory, a magnetic disk, an optical disk, a magneto-optical disk, or a magnetic tape. Or transmitted through a communication network such as the Internet and widely distributed.

It is a block diagram which shows the apparatus structure of 1st Embodiment. It is a flowchart which shows the process which the search device 1 performs before a search. It is a figure which shows an example of an untagged document. It is a figure which shows an example of a tagged document. It is a figure which shows the detection method of a synonym using a co-occurrence pattern. It is a figure which shows an example of an index. It is a figure which shows an example of 1st statistical information. It is a figure which shows an example of 2nd statistical information. It is a flowchart of the process which the search device 1 which transmitted the keyword performs. It is a figure which shows an example of 1st search result statistical information. It is a figure which shows an example of 2nd search result statistical information. It is a figure which shows an example of 3rd statistical information. It is a figure which shows an example of label compatibility information. It is a figure which shows an example of label information. It is a flowchart of the label selection which the label determination part 115 performs. It is a figure which shows an example of attribute name compatibility information. It is a figure which shows an example of cluster information. It is a detailed flowchart of the process for further integrating the determined label. It is a figure which shows the relationship between the set of documents containing one label and the set of documents containing the other label. It is a flowchart of the process which the browser 2 performs. It is a figure which shows the example of a display before the label instruction | indication by the document display control part. It is a figure which shows the example of a display after the label instruction | indication by the document display control part 22. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,10 ... Search apparatus 2 ... Browser 21 ... Keyword input part 22 ... Document display control part 101 ... Communication part 102 ... Request processing part 103 ... Setting file 104 ... Document generation part 105 ... Document generation part 106 ... Document DB
107 ... Normalization unit 108 ... Index generation unit 109 ... Document search unit 110 ... First statistical DB
111 ... 2nd statistics DB
DESCRIPTION OF SYMBOLS 112 ... Statistical processing part 113 ... Label candidate selection part 114 ... Label suitability calculation part 115 ... Label determination part 116 ... Cluster information generation part 117 ... Label integration processing part

Claims (4)

Document storage including a document including a text composed of a character string, further including a title of the document and document identification information indicating the document, and a plurality of documents in which attribute values that are predetermined character strings are included in the text Means,
For each attribute value included in at least one of the plurality of documents stored in the document storage unit, statistical information recording the number of appearances of the attribute value in the plurality of documents is generated, and the storage unit provided in advance Statistical processing means to be stored;
Document retrieval means for retrieving a plurality of documents from the document storage means;
Search result statistical information generating means for generating and storing search result statistical information in which the number of occurrences of the attribute value in the plurality of documents is recorded for each attribute value included in at least one of the plurality of searched documents; ,
For each attribute value included in at least one of the plurality of retrieved documents, the degree to which the attribute value is suitable as a label that is a character string representing a plurality of documents forming part of the plurality of documents is indicated. A fitness calculation means for calculating the fitness of the attribute value using a calculation formula using the number of appearances in the statistical information of the attribute value and the search result statistical information for calculating the fitness; ,
The fitness level as long as the fitness level satisfies a preset condition from the higher fitness level corresponding to each of a plurality of attribute values including attribute values included in at least one of the retrieved plurality of documents. By selecting an attribute value corresponding to, a plurality of attribute values forming a part of the plurality of attribute values are selected, and the plurality of selected attribute values are used as labels, respectively, and label information including the plurality of labels is displayed. Label information generating means to generate;
For each label included in the label information, document identification information and titles indicating documents that include the character string that is the label and that are any of the plurality of retrieved documents are included for the number of the documents. Cluster information generating means for generating cluster information that is cluster information and includes the label;
For each of two labels included in the label information, one set which is a set of documents indicated by each document identification information in the cluster information including the one label, and the cluster information including the other label A first ratio that is a ratio of the one set to a union with the other set that is a set of documents indicated by the document identification information, and a second ratio that is a ratio of the other set to the union If both the first ratio and the second ratio exceed a preset threshold value, it is determined that the two labels are in an equivalent relationship, and only one of them is a preset threshold value. Is exceeded, it is determined that there is an inclusive relationship that the label corresponding to the one is included in the label corresponding to the other, and if it is determined to be in the equivalence relationship or the inclusive relationship, the label that includes the two labels is included. And the label information includes a label that indicates an equivalence relation or inclusion relation between the two labels, and a combination of the document identification information and the title included in at least one of the two cluster information. Second cluster information generating means for generating cluster information including the label and the cluster information including
Each label included in the label information is displayed, one of the labels is selected, and a set of document identification information and title included in the cluster information including the label is displayed, respectively. A label display type document retrieval apparatus comprising: a document display control unit that reads out and displays a document including the document identification information and the set of titles from the document storage unit when a set of titles is selected . A label display type document search method performed by a label display type document search device,
The label display type document retrieval apparatus includes a document storage unit, a statistical processing unit, a document retrieval unit, a retrieval result statistical information generation unit, a fitness calculation unit, a label information generation unit, a cluster information generation unit, and a second cluster information generation unit. And a document display control means,
The document storage means includes
A document including a text composed of a character string, further including a title of the document and document identification information indicating the document, and a plurality of documents including attribute values that are predetermined character strings included in the text,
The label display type document search method includes:
The statistical processing unit generates statistical information that records the number of appearances of the attribute value in the plurality of documents for each attribute value included in at least one of the plurality of documents stored in the document storage unit; A statistical processing step to be stored in a storage means provided in advance;
A document search step in which the document search means searches for a plurality of documents from the document storage means;
The search result statistical information generating means generates search result statistical information in which the number of appearances of the attribute value in the plurality of documents is recorded for each attribute value included in at least one of the plurality of searched documents. A search result statistical information generation step to be stored;
For each attribute value included in at least one of the searched plurality of documents, the fitness level calculating unit is a label that is a character string representing a plurality of documents that form part of the plurality of documents. The calculation value using the number of appearances in the statistical information and the search result statistical information of the attribute value to calculate the fitness indicating the degree of suitability as A fitness calculation step to calculate;
The label information generation means presets the fitness level from the higher fitness level corresponding to each of a plurality of attribute values consisting of attribute values included in at least one of the searched plurality of documents. As long as the condition is satisfied, by selecting an attribute value corresponding to the fitness level, a plurality of attribute values forming a part of the plurality of attribute values are selected, and the plurality of selected attribute values are used as labels, respectively. A label information generation step for generating label information including the label of
Document identification information and title indicating that the cluster information generating means is a document that includes a character string that is the label for each label included in the label information and is one of the plurality of searched documents. A cluster information generation step of generating cluster information including the label and the cluster information including the number of the documents,
The second cluster information generating means, for each of two labels included in the label information, one set which is a set of documents indicated by each document identification information in the cluster information including the one label; A first ratio that is a ratio of the one set to a union with the other set that is a set of documents indicated by the document identification information in the cluster information including the other label, and the other with respect to the union A second ratio that is a ratio of the set of the two, and if both the first ratio and the second ratio exceed a preset threshold, it is determined that the two labels are in an equivalence relationship; If only one of the thresholds exceeds a preset threshold, it is determined that there is an inclusion relationship in which the label corresponding to the one is included in the label corresponding to the other, and it is determined that there is an equivalence relationship or an inclusion relationship. If so, the label information includes a label that includes the two labels so that the equivalence relation or the inclusion relation between the two labels can be understood, and at least one of the two cluster information includes the label. A second cluster information generating step for generating cluster information that includes cluster information including a set of the document identification information and the title, and including the label;
The document display control means displays each label included in the label information, and one set of the label is selected, and a set of document identification information and a title included in cluster information including the label is displayed. A document display control step of reading out and displaying a document including the document identification information and title set from the document storage means when one set of the document identification information and title is selected;
Labeling type document retrieval method, which comprises a. A computer program for causing a computer to execute the label display type document retrieval method according to claim 2 . A computer-readable recording medium in which the computer program according to claim 3 is stored.

Images