JP2006185368A - Document database update processor, document database retrieval device, document database index preparation method, and document database retrieval method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a document database update processor capable of efficiently updating/retrieving a document DB by suppressing the consumption of storage capacity in update processing of the document database, a document database retrieval device, a document database index preparing method and a document database retrieval method. <P>SOLUTION: The document DB update processor 102 prepares update information having words to be eliminated in a new edition, negative indexes obtained by recording a list of appearance position list groups of the words and a group of positive indexes obtained by recording appearance positions of character strings particularly added and changed in changed indexes and position shift values in updating indexes from the present generation to the next generation in a document DB of a generation control system, and represents the next generation document DB by the update information and the index groups. In addition, the document DB update processor 102 has an update information merge processing function for merging update information of a plurality of generations. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a document database update device, a document database search device, a document database index creation method, and a document database search method, and more particularly to a document database update device, a document database search device, and a document database that perform update processing and search processing of a document database. The present invention relates to an index creation method and a document database search method.

  In recent years, the use of information technology has become active along with the improvement in performance and price of computers and networks, and the development and spread of the Internet communication environment. Among them, databases and database management systems are actively used as a basis for information storage and information retrieval.

  Some of such database management systems introduce a concept of database “generation” and perform generation management. In the generation management type database management system, in the update process, the database of the next generation is created without changing the substance of the update source database stored in the auxiliary storage device or the like.

  As such a conventional generation management type database management system, there is one that manages the generation of a document database. For example, there is a document database management system as described in Non-Patent Document 1. In this document database management system, the update contents of a document database (hereinafter referred to as a document DB) are taken into consideration and divided into a plurality of DBs in the input data period to make differential update more efficient.

  The document DB management system will be described below with reference to FIG. In the document DB management system 2100 shown in FIG. 21, reference numeral 2101 denotes an input update document, a document DB update processing apparatus that creates a generation N using an index and a document of generation N−1, and 2102 a document into which a document is input. The input device 2103 records a plurality of generations of document DB entities permanently in the auxiliary storage device 2110, or records the latest several generations of update information in the main storage device 2109 in order to speed up the search process. A DB holding device, 2104 is a document DB creation device that creates an index of an initial generation from a document, 2105 is provided with an input interface such as a keyboard, and a search keyword for searching a document DB and a search keyword for inputting a generation to be searched An input unit 2106 is a document DB search unit that searches an input search keyword from the document DB, and 2107 is a display of search results. Search result output unit to be displayed on the display device, such as a ray, 2108 search keyword input unit 2105, a document DB retrieval device comprising a document DB search unit 2106 and a search result output unit 2107.

  The operation of the document DB management system 2100 having the above configuration and a specific example of simple update and search will be described with reference to FIG.

  3A to 3C are examples of a document DB storing miscellaneous feelings such as a personal diary as the document DB.

  As shown in an example 301 of the document DB before update in FIG. 3A, the document DB is expressed as a plurality of document records delimited by tags (represented as <doc ID = X> (X: ID number)). Each document record arranges two items of “document ID” and “text” in the order of ID numbers. Particularly, for the document ID, when the document ID is X, <doc ID = X> is used as a tag element. It shall be recorded and expressed as follows. In order to perform a high-speed search and update process, this document DB includes a word appearing in each document record from the document DB, as shown in index example 401 of the document DB before update in FIG. The document ID in which the character appears and the appearance character position are recorded. In the example 301, there are three document records with document IDs 1, 2, and 3. When updating, each miscellaneous document is identified by the document ID, and update processing of addition, change, and deletion is performed. FIGS. 3B and 3C respectively show an update document example 302 and an updated document DB example 303 for the document DB 301.

  In these examples, the change data for the existing document records of document ID 1 and ID 3 and the deletion record of the existing miscellaneous document of document ID 2 (in the case of a miscellaneous document with an empty body, an instruction to delete the existing document specified by the document ID is given And a newly added document record with document ID 4 is included, and when a difference update process is performed using 302 update document as an input, the next generation document DB 303 (after update) Document DB) is created.

  When the document DB update processing apparatus 2101 and the document DB creation apparatus 2104 receive the update document 302 as shown in FIG. 3B as input, data for searching for and updating the contents of the new update document A difference index which is a structure is created as shown in an updated document DB index example 2201 shown in FIG. 22 and stored in the document DB holding device 2103.

  In this document DB management system 2100, by operating in this way, the update contents of the generation management type document DB are efficiently divided into a plurality of document DBs in the input data period in consideration of the update contents of the document DB. It has become.

In addition, as a generation management type data updating apparatus that enables searching by superimposing with a conventional database, for example, there is a concept dictionary management apparatus described in Patent Document 1. With this device, a basic concept dictionary shared by multiple users can be created by creating a conceptual dictionary adjusted for each field or user without changing the contents, and these can be overlaid and searched. Thus, the basic concept dictionary can be expanded and reduced without destroying it.
JP-A-6-075989 Narayanan Shivakumar, Hector Garcia-Molina: Wave-Indices: Indexing Evolving Databases. SIGMOD Conference 1997: 381-392

  However, in the conventional document DB management system of Non-Patent Document 1, all character strings of the document ID in which the change has occurred are changed even in a small update in which only one word of a document ID in the document DB is changed. In order to create an index for all the extracted character strings, it consumes a large amount of auxiliary storage area for storing update information, and the update speed is proportional to the amount of update information. There is a problem that efficiency decreases.

  In addition, in the conventional concept dictionary management device of Patent Document 1, efficient update processing can be performed by combining a small amount of difference concept dictionary related to only the word of the changed part and the original concept dictionary, but the concept dictionary only In addition, since there is no index for searching documents at high speed, generation update processing of the document DB cannot be realized.

  SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and a document database update processing apparatus and a document database search apparatus capable of efficiently updating / searching a document database while suppressing consumption of storage capacity in the update process of the document database. An object of the present invention is to provide a document database index creation method and a document database search method.

  A document database update processing apparatus according to a first aspect of the present invention is a document database update processing apparatus that updates a generation-managed document database, and is configured in units of a plurality of records having IDs that are uniquely identified. A document database record that cuts out a character string for each record from the initial generation document, an index that shows the extracted character string and the character position of the character string as a pair, and records the initial generation document in the document database A document input unit for inputting a document for update, an update document determination unit for determining a part of a character string of a changed portion between the initial generation document and the document for update, and a corresponding character string part by the determination, Positive index creation that creates a positive index using a pair of the extracted character string, its occurrence position, and the difference value of the character string length that occurs when the character string is changed as an index element A negative index creation unit that creates an index element of the first generation to be deleted by the determination as a negative index, and a deletion that creates the document ID as a deletion record table for the record deleted by the determination And a document database update processing unit that updates and registers the created positive index, negative index, and deleted record table as update information of a new generation.

  According to this configuration, when updating the index from the initial generation to the next generation, the negative index that records the index element of the initial generation that should be deleted in the new version, and the appearance position of the character string that has been added or changed Update information having a positive index that records a pair of character string length difference values and a deleted record table that records document IDs of deleted records is created. By representing the document database, the amount of update information is proportional to only the changed character string portion, so the storage area for update information can be reduced, and at the same time, update information is created in proportion to the reduction in the amount of update information Processing time can also be shortened.

  A document database update processing device according to a second aspect of the present invention is a document database update processing device that updates a generation-managed document database, and is configured in units of a plurality of records having IDs that are uniquely identified. Document database recording section for recording documents from the initial generation to generation N and index information including negative index, positive index, and deleted record table in the document database, and document input for inputting generation N + 1 update document Update to determine the portion of the character string at the change location from the update document of the generation N + 1, and the index information including the negative index, the positive index, and the deletion record table, the document from the initial generation to the generation N The character string length generated in the document determination unit and in the change of the character string cut out, its appearance position, and the character string for the corresponding character string part by the determination. A positive index creation unit that creates a positive index using a set of difference values as an index element, a negative index creation unit that creates an index element of the initial generation to be deleted by the determination as a negative index, and the determination A deleted record table creation unit for creating a deleted record table for the deleted record, and a positive index, a negative index, and a deleted record table created by the determination are updated with generation N + 1 update information. And a document database update processing unit to be updated / registered.

  According to this configuration, when updating the index from the initial generation to the generation N, the negative index in which the index element of the initial generation to be deleted in the new version is recorded, and the appearance position of the character string that has been added or changed Generation N + 1 update information having a positive index that records a set of character string length difference values and a deleted record table set that records document IDs of deleted records is created. By expressing the generation N + 1 document database with the amount of update information proportional to only the character string portion changed for each generation, the storage area of update information for each generation can be reduced, and at the same time, the amount of update information In proportion to the reduction, the update information creation processing time can also be shortened.

  According to a third aspect of the present invention, in the document database update processing apparatus according to the second aspect, the generation from the initial generation to the generation N, the negative index, the positive index, and the generation based on the update document In the update process of i + 1 (0 <i <N), the index element to be deleted based on the negative index of generation i to generation N, the positive index, and the deletion record table, the added / changed index element, and the deletion By providing a plurality of update information interpreting units for interpreting the recorded records, a configuration for creating the generation N + 1 update information is adopted.

  According to this configuration, when updating the index from the initial generation to the generation N, generation N + 1 update information is generated by interpreting the negative index, the positive index, and the deletion record table elements of the generations i to N. Since the update information creation process is performed only for the character string portion added / changed up to generation N, the efficiency of the update information creation process can be improved.

  According to a fourth aspect of the present invention, in the document database update processing device according to the first or second aspect, the updated document determination unit is configured to perform an update process from the pre-update generation index to the post-update generation index. It is determined whether or not the number of character strings to be updated in the update target record is greater than an arbitrary threshold value. If there are many, the index is created by regarding the record as a change record, and the record number is stored in the deleted record table. The configuration to record is taken.

  According to this configuration, when the number of character strings to be changed exceeds a predetermined threshold value, it is recorded in the deletion record table as a record to be deleted, so that the update process is always performed by applying update information. Efficiency can be improved.

  According to a fifth aspect of the present invention, in the document database update processing apparatus according to the second aspect, an update information merge process for performing a process of combining the update information of a plurality of generations accumulated by the update over the plurality of generations into one update information The structure which further comprises a part is taken.

  According to this configuration, it is possible to improve the processing speed in subsequent generation update processing and search processing by combining update information over a plurality of generations into one update information.

  According to a sixth aspect of the present invention, in the document database update processing apparatus according to the fifth aspect, the update information merge processing unit deletes update information that is no longer needed.

  According to this configuration, it is possible to efficiently use the storage area for storing the document database by deleting unnecessary update information.

  According to a seventh aspect of the present invention, in the document database update processing apparatus according to the fourth aspect, the updated document determination unit specifies a document record to be compared from the update document, the document record, A difference character string list between the corresponding document records of the initial document is obtained, and it is determined whether or not the number of elements in the difference character string list is greater than the threshold value.

  According to this configuration, the determination process of the record to be deleted can be efficiently executed by determining the size of the number of elements of the difference character string list and the predetermined threshold value.

  A document database search device according to an eighth aspect of the present invention includes a search character string input unit that inputs a character string to be searched, update information that includes a plurality of generations of positive indexes, negative indexes, and delete record tables. A document database holding unit that stores document information of each generation, and the input character string is analyzed and divided into character strings, and each divided character string is updated from the document database holding unit over a plurality of generations. A configuration is provided that includes a document database search unit that searches using information, an index and a document of an initial generation, and a search result output unit that outputs a record set obtained by the document database search unit.

  According to this configuration, since the search target character string is searched from the update information including the positive index, the negative index, and the deletion record table, the search target can be searched from each element recorded in the index. The search processing efficiency for the database can be improved.

  According to a ninth aspect of the present invention, in the document database search device according to the eighth aspect, generation i (i) is performed using the positive index of generation N + 1 and update information from generation 0 and generation 1 to generation N + 1. = 1 to N + 1), a negative index element from generation N + 1 to generation i + 1, an index element deleted based on an element of the deletion record table, and a negative index that interprets the deleted record cumulatively A deletion record table interpretation unit, wherein the document database search unit searches the generation N + 1 positive index for each divided character string in the generation N + 1 index search and if there is a corresponding character string The character string is set as a search candidate, and in the generation N index search, if there is a character string corresponding to each of the divided character strings, the character string is output to the negative index / delete record table interpretation unit, and negative If the character string corresponding to the character string input from the document database search unit is in the negative index of the generation N + 1, the lookup / deletion record table interpretation unit does not search the character string, and the generation N + 1 If the document data of the record number registered in the deletion record table is interpreted, and there is an element of the input character string, a configuration is adopted in which that element is not a search target.

  According to this configuration, a search target can be efficiently searched from a document database created by an update process over a plurality of generations.

  According to a tenth aspect of the present invention, in the document database search device according to the ninth aspect, a position shift that accumulates a position shift value of each element from the positive index of the generation N + 1 to the positive index of the generation i + 1 An accumulator; and the document database search unit adds the position shift value accumulated by the position shift accumulator to the appearance position of the character string retrieved from the positive index of the generation N + 1. Then, it is determined whether or not the character string searched before the character string is connected, and if it is connected, the searched character string is set as a search target.

  According to this configuration, it is possible to perform a search from a document database created by an update process over a plurality of generations in consideration of the connection relationship between the appearance position after the position shift of the search target and the appearance position of another search target. The search object can be searched accurately.

  An eleventh aspect of the present invention is the document database search device according to the ninth or tenth aspect, wherein the document database search unit is configured to generate the generations 0 to N + 1 for each of the divided character strings. The process of searching for the corresponding character string from the positive index is repeatedly executed, the concatenation is determined for each searched character string, and the generation N + 1 positive index is determined for all the divided character strings. A configuration is adopted in which a search target is searched from all elements up to the positive index of the generation i + 1.

  According to this configuration, when searching for a character string to be searched from a document database created by update processing over a plurality of generations, processing can be performed only by addition / subtraction operations on the appearance position set of each element of the index. Accurate and efficient search processing can be realized without reducing the processing speed.

  A document database index creation method according to a twelfth aspect of the present invention is a document database index creation method for creating an index of a generation-managed document database, and is composed of a plurality of record units each having a uniquely identified ID. A character string is cut out for each record from the generated initial generation document, an index showing the extracted character string and the character position of the character string as a pair, and a document that records the initial generation document in the document database A database recording step, a document input step for inputting an update document, an update document determination step for determining a portion of a character string to be changed between the initial generation document and the update document, and a corresponding character string portion by the determination For the index element, a pair of the extracted character string, its occurrence position, and the character string length difference value that occurs when the character string is changed A positive index creation step for creating an index, a negative index creation step for creating an index element of the initial generation to be deleted by the determination as a negative index, and a document ID for the record deleted by the determination A deletion record table creation step for creating a deletion record table, and an update / registration step for updating / registering the created positive index, negative index, and deletion record table as update information of a new generation. I did it.

  According to this method, when updating the index from the initial generation to the next generation, a negative index that records the index elements of the initial generation to be deleted in the new version, and the appearance position of the character string that has been added or changed Update information having a positive index that records a pair of character string length difference values and a deleted record table that records document IDs of deleted records is created. By representing the document database, the amount of update information is proportional to only the changed character string portion, so the storage area for update information can be reduced, and at the same time, update information is created in proportion to the reduction in the amount of update information Processing time can also be shortened.

  A document database index creation method according to a thirteenth aspect of the present invention is a document database index creation method for creating an index of a generation-managed document database, and is composed of a plurality of record units each having a uniquely identified ID. A document database recording step for recording documents from the initial generation to generation N and index information including a negative index, a positive index, and a deletion record table, respectively, and a generation N + 1 update document are input. Document input step, document from the initial generation to generation N, index information including negative index, positive index, and deleted record table, and determination of character string portion of changed portion from update document of generation N + 1 An updated document determination step to be performed, and for the character string portion corresponding to the determination, the extracted character string, its appearance position, and the change of the character string are changed. A positive index creation step for creating a positive index using a set of character string length difference values generated as an index element, and a negative index for creating an initial generation index element to be deleted by the determination as a negative index For the record deleted by the determination step, the deletion record table generation step for generating the document ID as a deletion record table, and the generation of the positive index, the negative index and the deletion record table generated by the determination An update / registration step for updating / registering as N + 1 update information is provided.

  According to this method, when updating the index from the initial generation to the generation N, the negative index in which the index element of the initial generation to be deleted in the new version is recorded, and the appearance position of the added or changed character string Generation N + 1 update information having a positive index that records a set of character string length difference values and a deleted record table set that records document IDs of deleted records is created. By expressing the generation N + 1 document database with the amount of update information proportional to only the character string portion changed for each generation, the storage area of update information for each generation can be reduced, and at the same time, the amount of update information In proportion to the reduction, the update information creation processing time can also be shortened.

  A document database search method according to a fourteenth aspect of the present invention includes a search character string input step for inputting a character string to be searched, update information comprising a plurality of generations of positive indexes, negative indexes, and deletion record tables. A document database holding step for storing the document information of each generation, the input character string is analyzed and divided into character strings, and update information for a plurality of generations from the document database for each divided character string; A document database search step for searching using an index and a document of the initial generation, and a search result output step for outputting a record set obtained by the document database search step.

  According to this method, since the search target character string is searched from the update information including the positive index, the negative index, and the deletion record table, the search target can be searched from each element recorded in the index. The search processing efficiency for the database can be improved.

  According to the present invention, when updating the index from the initial generation to the next generation, the negative index that records the index element of the initial generation to be deleted in the new version, and the appearance position of the character string that has been added or changed Update information having a set of a positive index that records a set of character string lengths and a deleted record table that records the document IDs of deleted records is created, and the next generation document database is created using this update information and the set of indexes. Since the amount of update information is proportional to only the changed character string portion, the storage area for the update information can be reduced, and at the same time, the update information creation processing time is also proportional to the reduction in the amount of update information. It can be shortened.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram showing an overall configuration of a generation management type document database processing system according to an embodiment of the present invention. In FIG. 1, a document database processing system 100 includes a document input device 101, a document DB update processing device 102, a document DB holding device 103, a document DB search device 104, main storage devices 108 and 110, and an auxiliary storage device. 109.

  The document input device 101 includes a keyboard or the like for inputting document data, and inputs the document data to the document DB update processing device 102.

  The document DB update processing apparatus 102 creates an initial index by cutting out character strings in record units from the initial document (generation 0) input from the document input apparatus 101, and sets the initial document and the initial index as the generation 0 document DB 103a. Records in the document DB holding device 103.

  In addition, when a new update document is input from the document input device 101, the document DB update processing device 102 obtains a record unit from character string difference information between the update document and the initial document of the generation 0 document DB 103a. In addition, an index (a positive index and a negative index described later) and a deleted record table described later are created. These and the input update document are recorded in the document DB holding device 103 as the update information 103b of the generation 0 → 1.

  Further, the document DB update processing apparatus 102 updates the new N-1 generation update document input from the document input apparatus 101 and the update information (not shown) of the previous generation (N-2 generation). After interpreting the update information index (positive index, negative index) recorded in each generation, which is the difference information of the character string between the document for use and the initial document, the index corresponding to this update (positive index) , Negative index) and create a delete record table. The created index (positive index, negative index), deletion record table, and input update document are recorded in the document DB holding device 103 as update information 103c of generation N-2 → N-1.

  In addition, when a new Nth generation update document is input from the document input device 101, the document DB update processing device 102 receives an index (positive index, negative index) and a deletion record as described above. A table is created and recorded together with the input update document in the document DB holding apparatus 103 as update information 103d of generation N-1 → N.

  Further, the document DB update processing apparatus 102 has an update information merge processing function for merging multiple generations of update information recorded in the document DB holding apparatus 103. This will be described later.

  The document DB holding apparatus 103 permanently holds the generation 0 document DB 103a and holds intergenerational update information 103a to 103d created each time an update document is input by the document DB update processing apparatus 102. . The generation 0 update information 103a shown in FIG. 1 indicates that an initial document 1031 and an initial index 401 are recorded.

  The main storage device 108 stores a processing program for the document DB update processing executed in the document DB update processing device 102, and each index element (search target character string) to be compared in the update information merging processing described later. Etc. are temporarily stored.

  The auxiliary storage device 109 stores an index or the like created from the update information index of multiple generations in the update information merge process.

  The document DB search device 104 includes a search keyword input unit 105, a document DB search unit 106, and a search result output unit 107.

  The search keyword input unit 105 includes a keyboard or the like for the user to input a search character string for searching the document DB, and outputs the input search character string to the document DB search unit 106.

  When a search keyword is input from the search keyword input unit 105, the document DB search unit 106 includes update information of multiple generations recorded in the document DB holding device 103 for each character string included in the search character string, and generation 0. And a search function for searching for update information of generation N, and outputs the search result to the search result output unit 107.

  The search result output unit 107 displays the search result input from the document DB search unit 106 on a display or the like.

  The main storage device 110 stores a processing program of the document DB search process executed in the document DB search device 104 and stores a search result of the document DB search process.

  Next, with respect to the update information creation processing executed in the document DB update processing apparatus 102, the block diagram of the document DB update processing apparatus 102 shown in FIG. 2, the update example of the document DB shown in FIG. 3, the pre-update shown in FIG. Description will be made with reference to an index example of the document DB, an index example of the updated document DB shown in FIG. 5, an example of the deleted record table shown in FIG. 6, and a flowchart shown in FIG.

  In this update information creation process, the difference character string between the initial generation document (generation 0 document) that is the first input document and the generation 1 update document that is input next as the update document. In contrast, a case where update information is created will be described.

  FIG. 2 is a diagram showing a configuration of the document DB update processing apparatus 102 that creates the next generation, that is, the generation 1 document DB from the initial generation document DB created from the initial document.

  2, the document DB update processing apparatus 102 includes an update document determination unit 201, a document DB update processing unit 204 including a positive index creation unit 202, a negative index creation unit 203, and a deleted record table creation unit 205, an index And a creation unit 206.

  FIGS. 3A to 3C are diagrams illustrating an example of updating a document in which personal miscellaneous feelings are recorded as the document DB, and is a document DB having the same contents as those described above.

  As shown in FIG. 3A, the document DB 301 before update is expressed as a sequence of a plurality of document records delimited by tags (represented as <doc Id = X>). "And" text "are described side by side in document ID order. In particular, when the document ID is X, the document ID is recorded and expressed as a tag element such as <doc Id = X>. In order to perform high-speed search and update processing, this document DB includes words appearing in the document from each document record, such as an index (initial index) 401 of the document DB before update shown in FIG. The document ID in which the word appears and the appearance character position are recorded.

  In the pre-update document DB 301 in FIG. 3A, there are three document records with document IDs 1, 2, and 3. When updating, each document record is identified by the document ID, and added / changed / deleted. Each update process is performed. FIGS. 3B and 3C are an update document 302 and an updated document DB 303 for the document DB 301 before update, respectively.

  In this example of the update document 302, change data for the existing document records of document ID1 and ID3, and deletion data of the existing document record of document ID2 (the illusory document whose body is empty is an existing illusory document specified by the document ID). 3) and a new additional document record with the document ID 4 is included, and when the difference update process is performed in the document DB update processing apparatus 102, the following as shown in FIG. A generation miscellaneous document DB, that is, an updated document DB 303 is created.

  In the document DB update processing unit 204 in FIG. 2, the update document 302 or the initial document shown in FIG. 3B or the initial document (this is the case of the first document data, not the update process) is input. In order to retrieve and update the document DB 301 before update as shown in FIG. 3A at high speed, the update information which is the initial index or the difference index from the initial index for the character string cut out from the document ID and the text Is recorded in the document DB holding device 103 together with the updated document or the initial document.

  In FIG. 2, the index creation unit 206 reads the initial document 1031 (the document record included in the document DB 301 before update in FIG. 3A) from the generation 0 document DB 103a, and is known in the conventional full-text search DB creation. The character string is cut out from each document record included in the initial document using a method such as an N-gram division method that is a method of the above, or a method such as a method of dividing each document record into words using a word dictionary, The appearance position is recorded, and an index of the document DB before update as shown in FIG. 4, that is, an initial index 401 is created. Then, the index creating unit 206 records the created initial index 401 in the generation 0 document DB 103a.

  Next, the next generation (generation 1) update document 302 (see FIG. 3B) is input from the document input device 101, and the initial document 1031 in the generation 0 document DB 103a is updated to the next generation 1. The update information creation process for creating the update information 103b of the generation 0 → 1 when doing so will be described with reference to the configuration diagram of FIG. 2 and the flowchart shown in FIG. 5A and 5B are diagrams showing an example of a positive index 501 and an example of a negative index 502 created in the update process by the document DB update processing apparatus 102 in FIG.

  In FIG. 2, a generation N update document 302 is input to the document DB update processing apparatus 102 (step S701). The update document determination unit 201 determines the total number of records in the update document 302, specifies a document record j (j: record number, ie, document ID) to be compared, and determines the total number determined by the specified document record j. It is determined whether or not the number is equal to or less than the number of records (step S702).

  If the updated document determination unit 201 determines that the designated document record j is equal to or less than the total number of records (step S702: YES), the updated document determination unit 201 compares the designated document record j with the corresponding document record j of the initial document 1031, and records A difference character string list L (not shown) is obtained (step S703).

  Next, the updated document determination unit 201 determines whether or not the number of elements in the difference character string list L exceeds a preset threshold value ε (step S704). If the updated document determination unit 201 determines that the number of elements does not exceed the threshold ε (step S704: NO), the updated document determination unit 201 notifies the document DB update processing unit 204 of the difference character string list L.

  Next, when receiving the notification of the difference character string list L from the update document determination unit 201, the document DB update processing unit 204, based on the character strings that are the elements of the difference character string list L, Then, the character strings Word (l) to Word (M) are divided and extracted from the document record j of the update document 302 by a conventional index creation method such as a maximum word extraction method (step S705).

  Next, the document DB update processing unit 204 designates the first character string Word (i) (i = 1) from the extracted character strings Word (l) to Word (M) (step S706: YES). Then, it is determined whether or not there is a character string to be deleted by the element Word (p) of the generation 0 initial index 401 at the appearance position of the character string Word (i) (step S707).

  When the document DB update processing unit 204 determines that there is no character string Word (p) to be deleted at the appearance position of the character string Word (i) (step S707: NO), the document DB Word (i) document Information on the difference value of the ID, the appearance position, and the character string length is passed to the positive index creation unit 202. The positive index creating unit 202 combines the information of the difference value between the document ID, the appearance position, and the character string length of the character string Word (i) passed from the document DB update processing unit 204 as a position posting as shown in FIG. A positive index 501 as shown in a) is recorded (step S708).

  In step S707, when the document DB update processing unit 204 determines that there is a character string Word (p) to be deleted at the appearance position of the character string Word (i) (step S707: YES), the character string Word. The document ID and the appearance position of (p) are passed to the negative index creation unit 203. The negative index creation unit 203 records the document ID and appearance position pair of the character string Word (p) passed from the document DB update processing unit 204 in the negative index 502 as shown in FIG. (Step S709).

  Thereafter, the document DB update processing unit 204 sequentially designates (i ++) the character string Word (i) from the extracted character strings Word (l) to Word (M) (step S706), and extracts the extracted character string. All of Word (i) are designated, and similarly, the document DB update processing unit 204 repeatedly executes the processing of Step S707 to Step S709 for each designated character string Word (i) to obtain a positive value. An index 501 and a negative index 502 are created.

  As a result of the above processing, the document DB update processing unit 204, for one document record indicated by the document ID, extracts the difference character string L from the initial document, cuts out the character string from the difference character string L, Each creation process of the index 501 and the negative index 502 is completed, and the positive index 501 and the negative index 502 are recorded in the generation information 0 → 1 update information 103b shown in FIG.

  Next, when the update document determination unit 201 finishes specifying all of the extracted character string Word (i) (i = M is satisfied) (step S706: NO), the document record j to be compared next is selected. In order to specify, j is added (j ++) (step S710).

  Next, the updated document determination unit 201 specifies a document record j to be compared based on j added in step S710, and determines whether the specified document record j is equal to or greater than the determined total number of records ( Step S702).

  If the updated document determination unit 201 determines that the designated document record j is equal to or less than the total number of records (step S702: YES), the updated document determination unit 201 compares the designated document record j with the corresponding document record j of the initial document 1031, and records A difference character string list L between them is obtained (step S703), and the processes after step S704 are repeatedly executed.

  If the updated document determination unit 201 determines in step S704 that the number of elements in the difference character string list L exceeds the preset threshold value ε (step S704: YES), the updated document determination unit 201 specifies the initial document and the update document. Since there are too many differences between the document records j, information on the designated document record j is notified to the deletion record table creation unit 205. When the information of the designated document record j is notified from the update document determination unit 201, the deletion record table creation unit 205 records the document record j in the deletion record table as shown in FIG. 6 (step S711).

  Further, the update document determination unit 201 notifies the document DB update processing unit 204 to process the document record j of the update document 302 as an additional record. When the document DB update processing unit 204 receives a notification from the update document determination unit 201 to process the document record j of the update document 302 as an additional record, the index of the document record j whose position posting is a positive index of all 0s. Is created (step S712).

  The processes in steps S711 and S712 described above are executed for all the document records j of the update document 302, the creation of the deletion record table 601 is completed, and generation 0 → 1 is updated as shown in FIG. It is recorded in the information 103b.

  In step S702, when the update document determination unit 201 determines that the designated document record j exceeds the total number of records (j> total number of records) (step S702: NO), all the document records of the update document 302 are recorded. Since the process for j is completed, the update information creation process is terminated.

  As a result of the above update information creation processing, each document record of the initial document 1031 recorded in the document DB 301 before update shown in FIG. 3A and each document record of the update document 302 shown in FIG. From the difference character string, the positive index 501 in FIG. 5A is created, the negative index 502 in FIG. 5B is created, the deletion record table 601 in FIG. 6 is created, and generation 0 in FIG. 2 is created. 1 is recorded in the document DB holding device 103 of FIG.

  Here, the creation process of the update information 103b based on the update information creation process is specifically described with reference to FIGS. 3 (a), 3 (b), 4, 4, 5 (a), (b) and FIG. explain.

  First, an index (hereinafter referred to as an initial index) 401 of the document DB before update shown in FIG. 4 has been created for the document DB 301 before update shown in FIG. This initial index 401 indicates a character string of four elements cut out from the document record indicated by the document ID <doc ID = 1> in the document DB 301 before update. In the figure, (1, 1), (1, 2), (1, 3), (1, 6) are the former numerical value indicating the document ID, and the latter numerical value is the element appearance position (number of character digits). ).

  Then, when the update document 302 of FIG. 3B is input, the update document determination unit 201 determines that the total number of records in the update document 301 is “4”, and is a document record to be compared. 1 (<doc ID = 1>) is designated, and a difference character string list L between the document record 1 and the document record 1 of the initial document 1031 is obtained.

  In this case, based on the elements of the document record 1 recorded in the initial index 401, the difference character string between the document record 1 of the initial document 1031 and the document record 1 of the update document 302 is shown in FIG. It is each underline part shown in (b). That is, the difference character string list L includes “Despite being hot”, “hot”, “temperature”, “30 degrees” of the document record 1 of the initial document 1031, and “ “,” “Approaching”, “ga”, “still”, “hot”, “highest temperature”, “35 degrees” are recorded as elements.

  Next, the updated document determination unit 201 determines whether or not the number of elements in the difference character string list L exceeds a preset threshold value ε (for example, 20). In this case, since the number of elements of the difference character string list L shown in each document record 1 in FIGS. 3A and 3B is 11, it is determined that the number of elements does not exceed the threshold ε.

  Then, the updated document determination unit 201 notifies the document DB update processing unit 204 of the difference character string list L. Next, when the document DB update processing unit 204 receives the difference character string list L from the update document determination unit 201, based on each element of the difference character string list L, a conventional N-gram, a maximum word extraction method, and the like. According to the conventional index creation method, from the document record 1 of the update document 302, “to”, “approaching”, “but”, “not yet” corresponding to the character strings Word (l) to Word (M). “,“ Hot, ”,“ Maximum temperature ”,“ 35 degrees ”.

  Next, the document DB update processing unit 204 designates “to” of the document record 1 corresponding to the character string Word (i) from the extracted character strings Word (l) to Word (M). It is determined whether or not there is a character string Word (p) to be deleted in the document record 1 of the initial document 1031 at the appearance position “6”.

  In the document record 1 in FIG. 3A, since the character string “Despite being” exists at the appearance position “6”, the document DB update processing unit 204 has the character string Word (p) to be deleted. Then, the document ID “1” and the appearance position “6” of the character string “Daidano” are passed to the negative index creation unit 203. The negative index creation unit 203 creates a pair (1, 6) of the document ID “1” and the appearance position “6” of the character string “Despite being passed” from the document DB update processing unit 204 as shown in FIG. Record in the negative index 502 shown in b).

  In addition, the document DB update processing unit 204 has the character string “ni” shorter by five characters than the character string “despite it” of the document record 1 of the initial document 1031. The information of the document ID “1”, the appearance position “6”, and the character string length difference value “−5” of the character string “ni” is passed to the positive index creation unit 202. The positive index creating unit 202 combines the information of the document ID “1”, the appearance position “6”, and the character string length difference value “−5” of the character string “i” passed from the document DB update processing unit 204. The position posting (1, 6, -5) is recorded in a positive index 501 as shown in FIG.

  Further, the document DB update processing unit 204 designates the next character string “approaching” from the extracted character strings Word (l) to Word (M), and sets the initial document at the appearance position “7”. It is determined whether or not there is a character string Word (p) to be deleted in the document record 1 of 1031.

  In the document record 1 in FIG. 3A, the character string Word (p) to be deleted does not exist at the appearance position “7”. Therefore, the document DB update processing unit 204 passes the document ID “1”, the appearance position “7”, and the character string length “6” of the character string “approaching” to the positive index creation unit 202. The positive index creation unit 202 receives the information of the document ID “1”, the appearance position “7”, and the character string length difference value “6” of the character string “approaching” passed from the document DB update processing unit 204. In combination, position posting (1, 7, 6) is recorded in a positive index 501 as shown in FIG.

  The above processing is also performed on the other difference character strings of the document record 1 and the difference character strings of the other document records 2 to 3, thereby creating a positive index 501 as shown in FIG. Is done.

  Next, the update document determination unit 201 designates the next document record 2 (<doc ID = 2>) of the update document 302, and the difference between this document record 2 and the document record 2 of the initial document 1031. The character string list L is obtained in the same manner, but since there is no document in the document record 2 of the update document 302, the number of difference character strings exceeds the threshold ε, and the information of the document record 2 is stored in the deletion record table. The data is passed to the creation unit 205.

  Further, the update document determination unit 201 obtains a difference character string list L between the document record 3 of the update document 302 and the document record 3 of the initial document 1031, as shown in FIGS. 3 (a) and 3 (b). Since the underlined portion is a difference character string and the threshold ε exceeds “20”, the information of the document record 3 is transferred to the deletion record table creation unit 205.

  The deletion record table creation unit 205 creates the deletion record table 601 shown in FIG. 6 based on the information of the document records 2 and 3 passed from the update document determination unit 201, and updates generation 0 → 1 in FIG. It records in the information 103b.

  As described above, when the next generation update document is input to the initial document DB 103a, the document DB update processing apparatus 102 creates a positive index, a negative index, and a deletion record table as update information. Is recorded in the document DB holding device 103 of FIG. Also, the updated document DB 303 shown in FIG. 3C is created.

  Therefore, the positive index means that a newly added character string is cut out from each document record of the initial document 1031 in each document record of the input update document, and its document ID, appearance position, and position posting are combined. It is for recording as an index.

  The negative index is a combination of the document ID and the appearance position of the character string that is no longer necessary in each document record of the initial document 1031 and recorded as an index for each document record of the input update document. Is for.

  The deletion record table is used to record a document record deleted from the initial document 1031 or a document record greatly changed in each document record of the input update document.

  In the above update information creation process, the case of creating update information from generation 0 to generation 1 has been described. However, the document DB update process shown in FIG. FIG. 9 is a block diagram of the apparatus 102, an example of updating the document DB shown in FIG. 9, an example of an update document shown in FIG. 10, an example of an index of the initial document DB shown in FIG. 11, an example of update information of the update document DB shown in FIG. This will be described with reference to the flowchart shown in FIG.

  FIG. 8 is a diagram showing a configuration of the document DB update processing apparatus 102 that creates update information from generation 0 to generation 1, generation 2 and later. The same components as the document DB update processing apparatus 102 in FIG. , Are given the same reference numerals.

  8, the document DB update processing apparatus 102 includes an update document determination unit 201 including a multiple update information interpretation unit 801, a document DB update processing unit 204 including a positive index creation unit 202 and a negative index creation unit 203, A deletion record table creation unit 205.

  The multiple update information interpretation unit 801 executes the multiple update information interpretation processing shown in the flowchart of FIG. 13, and when generation N update document i is input, each generation 0 → 1 to generation 1 of generations 1 to N−1. The record of the document record number to be deleted described in the deletion record table i + 1 included in each update information of N-2 → N-1 is interpreted as already deleted (steps S1301 and S1302).

  FIGS. 9A to 9C show the document DB 301 before generation 0 shown in FIG. 3A and the updated document DB 303 or updated document DB 303 shown in FIG. 3C. In this example, the update document DB 901 of the generation 2 is created by further updating from the above.

  FIG. 10A shows the generation 1 update document 302 shown in FIG. 3B and FIG. 10B shows the generation 2 update document 1001.

  FIG. 11 shows a generation 0 document DB 103a including the initial index 401 shown in FIG. 4 and the initial document 1031 shown in FIG. 12A shows the positive index 501 shown in FIG. 5A, the negative index 502 shown in FIG. 5B, the deletion record table 601 shown in FIG. 6, and FIG. The update information 103b of the generation 0 → 1 including the update document 302 shown in b) is shown.

  FIG. 12B shows a positive index 1201a, a negative index 1201b, a deletion record table 1201c, and an update document 1001 included in the update information 1201 of generation 1 → 2.

  From the generation 0 document DB 301 in FIG. 9A to the generation of the generation 1 update document DB 303 in FIG. 9B, the update information creation processing based on FIG. ) Update document 1001 is input, the document record with document ID 3 is deleted, and the content of the document record with document ID 4 is changed. Created as FIG. 12B is a diagram illustrating a case where generation information 1 → 2 update information 1201 is created by the document DB update processing unit 204 of FIG. 8 in the example of FIG.

  In the update information 1201 of the generation 1 → 2 in FIG. 12B, the document record with the document ID 3 in the update document 302 in FIG. 10A is to be deleted by the update document 1001 in FIG. Therefore, it is recorded in the deletion record table 1201c. Also, in the update from generation 1 to generation 2, the document record with document ID 4 is updated by the update information determination unit 201 for each of the update document 302 in FIG. 10 (a) and the update document 1001 in FIG. 10 (b). When the document records are compared and the number of difference character strings is determined, it is determined that there is almost no change in this example, and the determination result is notified to the document DB update processing unit 204.

  When the normal update process described with reference to FIG. 7 is executed in the document DB update processing unit 204, a character string is cut out like the positive index 1201a shown in FIG. The character string to be deleted is recorded as a negative index 1201b.

  In addition, by executing the multiple update information interpretation process shown in the flowchart of FIG. 13 in the multiple update information interpretation unit 801, the deletion record table 1201c shown in FIG. 12B is recorded.

  As shown in the example of FIGS. 9 to 12 above, the update information 103b of the generation 0 → 1, the update information 1201 of the generation 1 → 2, and the initial index 401 of the generation 0 are paired, so that FIG. The generation 2 update document DB 901 shown in FIG. The index relationship between the generations in these multiple generations is expressed by a conceptual expression as follows.

(Positive index N) = F (positive index N−1) − (negative index N) − (deleted record table N)
N: Generation number (1 <= N)
F (index n): a function that reflects the position shift value in the positive index N of generation n

  Therefore, in the update information creation processing according to the present embodiment, in the generation management system document DB, a list of a set of a word to be deleted in the new version and its appearance position list when updating the index from the current generation to the next generation , A negative index that records the occurrence position and position shift value of the character string that was added or changed, and a deleted record table that records the document ID of the deleted record By creating update information having a set of the following and expressing the next generation document DB with this update information and the set of indexes, the amount of update information is proportional to only the changed character string portion. The storage area can be reduced, and at the same time, the update information creation processing time can be shortened in proportion to the reduction in the amount of update information.

  In the update information creation process according to the present embodiment, the character string segmentation method is a known technique used when creating an index for conventional full-text search, such as an N-gram segmentation method or segmentation based on dictionary words. Any method can be applied as long as it is an index in which information is recorded by a cut character string and its appearance position.

  In addition, in the update information creation process of the present embodiment, document records that have many character string portions to be changed when creating update information require more overhead in both update and search processes than in the conventional method. For this reason, in the update information creation process of the present embodiment, when the number of character strings to be changed exceeds a predetermined threshold value ε, it is recorded in the deletion record table as a record to be deleted and the index is updated as usual. Thus, the processing efficiency can be improved as compared with the case where update processing is always performed by applying update information.

  Note that the threshold value may be determined based on the empirical value of the administrator, or it may be determined empirically based on the value of the update process in advance according to the nature of the update data. Yes, the user can also determine the optimal threshold.

  Next, update processing between generations created each time an update document is input by the update information creation processing according to the present embodiment, in particular, merge processing that is a means for streamlining update information over multiple generations. explain.

  FIG. 14 shows an update information merge processing unit 1401 that combines a plurality of pieces of update information into one update information as a new function in the document DB update processing apparatus 102, and a plurality of pieces of update information according to the merge processing. It is a block diagram which shows the structure of the document DB update processing apparatus 102 which newly provided the update information deletion part 1402 to delete. FIG. 15 is a flowchart showing an update information merge process executed in the document DB update processing apparatus 102 to combine the update information of the generation j to the update information of the generation k into one update information.

  Note that FIG. 16 illustrates two pieces of update information (generation 0 → 1 update information 103b and generation 1 in the generation 0 to generation 2 update information described with reference to FIGS. 12 (a) and 12 (b). FIG. 6 is a diagram illustrating an example in which update information 1201) of 2 is combined into one update information and updated information 1601 of generation 0 → 2 in the update information merge processing unit 1401 of the present embodiment.

  First, the update information merge processing unit 1401 stores all elements of the positive index 1201a and the negative index 1201b, which are the update information 1201 of generation 1 → 2, in the main storage device 108 such as a non-volatile memory. (Not shown) and a cumulative negative element set (not shown) are recorded (step S1501). Next, the update information merge processing unit 1401 records the elements of the deletion record table 1201c as a cumulative deletion record set in the main storage device 108 (step S1502).

  Next, the update information merge processing unit 1401 executes the processing after step S1504 while incrementing the value of the temporary variable i by 1 until it reaches j (step S1503). The update information merge processing unit 1401 until the condition “i <j” is satisfied prepares and initializes a storage area for a positive index element set in the main storage device 108, which is called a temporary primary element set. All elements of the positive index 1201a are recorded in the temporary primary element set (step S1504). However, at this time, the update information merge processing unit 1401 does not record the element having the document ID recorded in the deletion record table 1201c, that is, the document record having the document ID of 3, in the temporary primary element set.

  Next, the update information merge processing unit 1401 takes out each element MWord (m) of the cumulative negative element set (step S1505), and checks whether MWord (m) exists in the temporary positive element set (not shown). It investigates (step S1506). If it exists (step S1506: YES), the update information merge processing unit 1401 deletes MWord (m) from the temporary positive element set and deletes MWord (m) from the cumulative negative element set (step S1507).

  In the example of the update information 1201 of the generation 1 → 2 in FIG. 12B, the information currently recorded in the negative index 1201b is recorded in the cumulative negative element set, and the “M appliance ( 4, M) (4, 1, 0) "and" M electric device (4, 1, 0) "of the temporary positive element set exist together," M electric device (4, 1, 0) "is deleted from the temporary positive element set. The

  Next, when the update information merge processing unit 1401 finishes comparing all the elements of the cumulative negative element set (step S1505: NO), the update record merge processing unit 1401 converts the elements of the generation 0 → 1 deletion record table 601 into a cumulative deletion record set (not shown). ) In the main storage device 108 (step S1508). Further, the temporary positive element set is additionally recorded in the cumulative positive element set (step S1509), and the elements of the negative index 502 of the generation 0 → 1 are additionally recorded in the cumulative negative element set (not shown) (step S1510).

  Thereafter, when the comparison formula in the processing of step S1503 is not satisfied (step S1503: NO), the update information merge processing unit 1401 uses the cumulative negative element set obtained so far as the update information 1601 of generation 0 → 2 shown in FIG. The cumulative positive element set is recorded as a negative index (0_2) 1603, the positive index (0_2) 1602 and the cumulative deleted record set is deleted as a deleted record table (0_2) 1604 (step S1511), and the update process is terminated. .

  In this way, by combining update information over a plurality of generations into one update information, the processing speed in subsequent generation update processing and search processing can be improved.

  When the merge process ends, the update information deletion unit 1402 updates the generation information 0 → 1 update information 103b and the generation information 1 → 2 update information 1201 according to the conditions specified in advance by the administrator of the document database. The data can be deleted from the DB holding device 103, and the storage area of the document DB holding device 103 can be used effectively. Further, when deleting the data, the data can be deleted more safely by performing backup writing to another storage medium, for example, an external storage medium such as a disk medium, and deleting the data.

  In addition, in the update information creation process of the present embodiment, when update is performed a plurality of times and the update information is accumulated, the update speed and the search speed are reduced in proportion to the update information. On the other hand, since update information merge processing is performed, it is possible to mitigate a decrease in update and search processing speed. At the same time, it is possible to efficiently use the storage area for storing the document DB by deleting update information that is no longer necessary after the merge processing is completed.

  Up to this point, the update process of the document DB has been described. Hereinafter, the operation of the document DB search apparatus 104 that searches the document DB created by the above operation will be described.

  FIG. 17 is a block diagram showing the configuration of the document DB search device 104. 17, the document DB search apparatus 104 includes a search keyword input unit 105, an index search unit 1701, a position shift accumulation unit 1702, a negative index / deleted record table interpretation unit 1703, a search result output unit 107, Consists of

  FIG. 18 is a flowchart showing document DB search processing executed in the document DB search apparatus 104. The search process will be described with reference to these drawings.

  A search character string is input from the search keyword input unit 105 by the user (step S1801). The document DB search apparatus 104 cuts out the input search character string by keyword division processing based on the dictionary word (step S1802).

  Next, as an initial preparation, the document DB search device 104 uses a negative index element, a temporary negative set (not shown) that stores elements of the deleted record table, and a character string obtained by the keyword division process. When a search is possible with a positive index, a list of head appearance positions is stored cumulatively over a plurality of generations (not shown), and for each element of the cumulative result set Then, a position shift accumulation set (not shown) for accumulating position shift values in a plurality of generations of positive indexes is initialized (step S1803).

  Thereafter, the document DB search device 104 performs processing for each divided character string until i exceeds M (step S1804). Next, j, which is used as a subscript representing the generation, is substituted with the generation N to be searched first for j, and while this is decreased by 1, the process is continued while j is 0 or more (step S1805).

  If j is greater than or equal to 0 (step S1805: YES), the index search unit 1701 determines whether or not a search can be performed with the positive index j with respect to the extracted character string Word (i) (step S1806). . If it is determined that the search is possible (step S1806: YES), the process proceeds to step S1807. Note that “0 to N” representing a generation is substituted for j used as a subscript of the positive index j. The same applies to the subsequent negative generation j or deleted record table j.

  In step S1806, the position shift accumulation unit 1702 calculates a value of the position shift accumulation set. At this time, the negative index / deletion record table interpretation unit 1703 causes the character string Word (i) searched by the index search unit 1701 to have the same character appearance position as the negative index element registered in the temporary negative set. Whether or not the record has an element of the deleted record table is determined.

  In step S1806, the negative index / deletion record table interpretation unit 1703 determines that the searched character string Word (i) has the same character appearance position as the negative index element registered in the temporary negative set, or the deletion record. If the record corresponds to a record having an element in the table, the processing is carried out assuming that the position is not a search target, and the character string Word (i) as the corresponding element is deleted from the temporary negative set. As a result, whether to move to step S1807 or to move to step S1808 is determined as the next processing depending on whether or not a searchable positive index element exists.

  The index search unit 1701 determines whether the appearance position is connected to each element s of the cumulative result set and the element p of the positive index element set obtained by the above-described index search process. The position shift value is added to the element e (p) of the position shift accumulation set. Also, the index search unit 1701 deletes the element s from the cumulative result set in the case of the initial index where j == 0 and there is no concatenated p (step S1807).

  Next, the position shift accumulation unit 1702 accumulates and stores the negative index j and the set of deleted record tables j in the temporary negative set in the temporary negative set (step S1808). In step S1806, if there is no searchable positive index element (step S1806: NO), the position shift accumulation unit 1702 makes a transition to perform the process in step S1808.

  In step S1805, when the condition 0 <= j is not satisfied (step S1805: NO), the document DB search apparatus 104 determines whether or not the cumulative result set is an empty set (step S1809). If there is no search result (step S1809: YES), the result record set is returned and the search process is terminated (step S1810).

  If the document DB search apparatus 104 is not an empty set (step S1809: NO), the processing of steps S1804 to S1809 is performed on the next cut character string as before. In addition, the document DB search apparatus 104 repeatedly performs the processing of steps S1804 to S1809 for all the extracted character strings, and outputs the obtained accumulated result set to the search result output unit 107 as a result set.

  By such a search process, the document DB update processing apparatus 102 can efficiently search the document DB created by the update process over a plurality of generations.

  Next, a specific example of character string search processing will be described with reference to FIGS.

  In the example of FIG. 19, in response to the input of the search character string 1901 “H company”, the index search unit 1701 reads the characters “H company (1, 7, 0)” from the initial index 1902 a of the generation 0 document DB 1902. The column 1903 is cut out (step S1910).

  Next, the index search unit 1701 searches the positive index 1904a of the update information (generation 0 → 1) 1904 and does not hit “Company H” (step S1911), and then searches the negative index 1904b. (Step S1912), "H Company (1, 7, 0)" has been hit, and this is registered in the cumulative negative set B1905 (Step S1913).

  Next, the index search unit 1701 hits the seventh occurrence position of “H Company”, which is the character string 1903 searched from the initial index 1902a, but there is a hit in the same appearance position in the cumulative negative set B1905. Therefore, “Company H” does not hit at this position. That is, the character string 1903 cut out from the initial index 1902a shown in the figure is deleted from the cumulative negative set 1905 (step S1914), and as a result, the set φ1906 is output.

  Next, the example of FIG. 20 shows an example of search processing when a search character string 2001 “MEI company and SO company” is input to the generation 0 document DB 1902 of FIG.

  First, the index search unit 1701 cuts out the search character string 2001 such as “MEI company”, “TO”, “SO company”, and “GA”.

  Next, as in the example of FIG. 19, the index search unit 1701 searches the positive index 1904a (step S2010), and “ME company” and “SO company” appear. Store in a set (not shown).

  Next, the index search unit 1701 stores all elements of the negative index 1904b in a cumulative negative set (not shown) (step S2011). Thereafter, the index search unit 1701 searches the initial index 1902a of generation 0 (step S2012), and the elements registered in the cumulative negative set and the elements registered in the initial index 1902a have the same character string and the same A character string hit in generation 0 is searched for after omitting the appearance position from the search target.

  In this case, first, since “MEI company” does not exist in the initial index 1902a, the cumulative position set is recorded with the head position “4” of “MEI company (1, 4, 2)” hit in the positive index 1904a. The position shift value “2” is recorded in the position shift accumulation set.

  Next, the index search unit 1701 does not search the character string “to” cut out from the search character string in the positive index 1904a and does not record it in the negative index 1904b. Sometimes “and (1, 6, 0)” hits for the first time. Next, in determining whether to concatenate with only one “MEI company” stored in the accumulation result set, the position shift value “3” recorded in the position shift accumulation set “ The connection is determined after adding “2”.

  With this processing, as indicated by 2002 in the figure, the appearance position of “to” is “8”, the appearance position of “MEI company” is “4”, and the character string length is “4”. This position remains in the cumulative result set.

  Similarly, a search process is performed for the cut-out character string “ga”, and the appearance position and the position shift value are added to determine the connection with “SO company” hit in the positive index 1904a. The position remains in the cumulative result set.

  In this way, by repeating the process for each extracted character string, it is possible to finally perform a search so as to hit the portion where the appearance character position where the document ID is “1” is “4”. .

  Therefore, in the search process performed by the document DB search apparatus 104 according to the present embodiment, in addition to the conventional search process, a process for interpreting the update information occurs. This process is only an addition / subtraction operation for the character string appearance position set. Therefore, the search processing speed does not decrease, and the search processing of the character string without change may be performed by using a conventional index, so that the search speed does not decrease.

  The document database update processing device, the document database search device, the document database index creation method, and the document database search method of the present invention can reduce the amount of update information of the generation management type document database and can improve the efficiency of the character string search. Therefore, it can be applied to a document database processing system or the like.

The block diagram which shows the structure of the document database processing system which concerns on embodiment of this invention The block diagram which shows the structure of the document DB update processing apparatus which produces document DB of the generation 1 which concerns on this Embodiment (A) according to the present embodiment is a diagram showing an example of a document DB before update, (b) is a diagram showing an example of a document for update, and (c) is a diagram showing an example of a document DB after update. The figure which shows the example of an index of document DB before the update which concerns on this Embodiment (A) is a diagram showing a positive index example of the updated document DB, and (b) is a diagram showing a negative index example of the updated document DB according to the present embodiment. The figure which shows the example of the deletion record table of document DB after the update which concerns on this Embodiment Flowchart showing update information creation processing according to the present embodiment The block diagram which shows the structure of the document DB update processing apparatus which produces the update information from the generation 0 to the generation 1, the generation 2 or later concerning this Embodiment (A) is a diagram showing an example of a generation 0 document DB, (b) is a diagram showing an example of a generation 1 update document DB, and (c) is an example of a generation 2 update document DB according to the present embodiment. Illustration (A) which shows the example of the document for an update which concerns on this Embodiment, (b) is a figure which shows the example of the document for another update The figure which shows the example of document DB of the generation 0 which concerns on this Embodiment (A) which shows the example of the update information of generation 0-> 1 which concerns on this Embodiment, (b) is a figure which shows the example of update information of generation 1-> 2 Flowchart showing multiple update information interpretation processing according to the present embodiment The block diagram which shows the structure of the document DB update processing apparatus which performs the update information merge process which concerns on this Embodiment The flowchart which shows the update information merge process which concerns on this Embodiment The figure which shows the example of the update information of the generation 0-> 2 produced by the update information merge process which concerns on this Embodiment The block diagram which shows the structure of the document DB search apparatus concerning this Embodiment. Flowchart showing document DB search processing according to the present embodiment The figure which shows the specific example of the character string search process which concerns on this Embodiment The figure which shows the specific example of the other character string search process which concerns on this Embodiment Block diagram showing the configuration of a conventional document DB management system The figure which shows the example of an index of the document DB after the conventional update

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Document database processing system 101 Document input apparatus 102 Document DB update processing apparatus 103 Document DB holding | maintenance apparatus 103a Generation 0 document DB
103b Update information of generation 0 → 1 103c Update information of generation N−2 → N−1 103d Update information of generation N−1 → N 104 Document DB search device 105 Search keyword input unit 106 Document DB search unit 107 Search result output unit 108, 110 Main storage device 110 Auxiliary storage device 201 Update document determination unit 202 Positive index creation unit 203 Negative index creation unit 204 Document DB update processing unit 205 Deleted record table creation unit 206 Index creation unit 801 Multiple update information interpretation unit 1401 Update information merge processing unit 1402 Update information deletion unit 1701 Index search unit 1702 Position shift accumulation unit 1703 Negative index / delete record table interpretation unit

Claims (14)

A document database update processing device for updating a generation-managed document database,
A character string is cut out for each record from an initial generation document composed of a plurality of records having a uniquely identified ID, and the extracted character string and an appearance character position of the character string are shown in pairs. An index, and a document database recording unit for recording the initial generation document in a document database;
A document input section for inputting the update document;
An update document determination unit for determining a part of a character string of a changed portion between the initial generation document and the update document;
A positive index creation unit that creates a positive index for the corresponding character string portion by using a pair of a character string length that is cut out and a character string length difference value that occurs when the character string is changed as an index element When,
A negative index creation unit that creates, as a negative index, the index element of the initial generation to be deleted by the determination;
The record deleted by the determination includes a deletion record table creation unit that creates the document ID as a deletion record table, and the created positive index, negative index, and deletion record table are created for a new generation. A document database update processing apparatus comprising: a document database update processing unit that updates and registers as update information. A document database update processing device for updating a generation-managed document database,
Documents that record documents from the initial generation to generation N composed of a plurality of record units each having a uniquely identified ID, and index information composed of a negative index, a positive index, and a deleted record table, respectively, in a document database A database recording unit;
A document input unit for inputting generation N + 1 update documents;
An update document determination unit that determines a part of a character string at a changed portion from the update document of the generation N + 1, and index information including a negative index, a positive index, and a deletion record table, from the initial generation to the generation N When,
A positive index creation unit that creates a positive index for the corresponding character string portion by using a pair of a character string length that is cut out and a character string length difference value that occurs when the character string is changed as an index element When,
A negative index creation unit that creates, as a negative index, the index element of the initial generation to be deleted by the determination;
A deletion record table creation unit that creates a document ID of the record deleted by the determination as a deletion record table, and generates a positive index, a negative index, and a deletion record table generated by the determination as generation N + 1. And a document database update processing unit for updating and registering as update information of the document database.   In the update process of the generation i + 1 (0 <i <N) based on the documents from the initial generation to the generation N and the negative index, the positive index, and the update document, negative numbers of the generation i to the generation N Updating the generation N + 1 by providing a multiple update information interpretation unit that interprets an index element to be deleted based on an index, a positive index, and a deleted record table, an added / changed index element, and a deleted record 3. The document database update processing apparatus according to claim 2, wherein information is created.   The update document determination unit determines whether or not the number of character strings to be updated in the update target record is larger than an arbitrary threshold when updating from the pre-update generation index to the post-update generation index. 3. The document database update processing apparatus according to claim 1 or 2, wherein the record is regarded as a change record, an index is created, and the record number is recorded in the deletion record table.   3. The document database update processing apparatus according to claim 2, further comprising an update information merge processing unit that performs a process of combining the update information of the plurality of generations accumulated by the update over the plurality of generations into one update information.   6. The document database update processing apparatus according to claim 5, wherein the update information merge processing unit deletes update information that is no longer needed.   The updated document determination unit specifies a document record to be compared from the update document, obtains a difference character string list between the document record and the corresponding document record of the initial document, and calculates the difference character string list. The document database update processing apparatus according to claim 4, wherein it is determined whether or not the number of elements is greater than the threshold value. A search character string input part for inputting a character string to be searched;
Update information consisting of a plurality of generations of positive indexes, negative indexes and deletion record tables, and a document database holding unit for storing document information of each generation,
The input character string is analyzed and divided into character strings, and for each divided character string, a search is performed using update information over a plurality of generations from the document database holding unit, an index and a document of an initial generation A database search unit;
And a search result output unit for outputting a record set obtained by the document database search unit. Elements of negative index from generation N + 1 to generation i + 1 when searching for generation i (i = 1 to N + 1) using the positive index of generation N + 1 and update information from generation 0 and generation 1 to generation N + 1 And an index element deleted based on the elements of the deleted record table, and a negative index / deleted record table interpreter that interprets deleted records cumulatively,
In the generation N + 1 index search, the document database search unit searches the generation N + 1 positive index for each divided character string and if there is a corresponding character string, the character string is set as a search candidate. In the generation N index search, if there is a character string corresponding to each of the divided character strings, the character string is output to the negative index / deleted record table interpretation unit,
The negative index / deleted record table interpretation unit does not search the character string if the character string corresponding to the character string input from the document database search unit is in the negative index of the generation N + 1, and the generation 9. The document database according to claim 8, wherein the document data of the record number registered in the N + 1 deletion record table is interpreted, and if there is an element of the input character string, the element is not searched. Search device. A position shift accumulation unit for accumulating the position shift value of each element from the generation N + 1 positive index to the generation i + 1 positive index;
The document database search unit adds the position shift value accumulated by the position shift accumulation unit to the appearance position of the character string retrieved from the positive index of the generation N + 1, before the character string. 10. The document database search apparatus according to claim 9, wherein it is determined whether or not the appearance position of the searched character string is concatenated, and if the character string is concatenated, the searched character string is a search target.   The document database search unit repeatedly executes a process of searching for the corresponding character string from the positive index of each generation of the generations 0 to N + 1 for each of the divided character strings, and for each searched character string The determination of concatenation is performed, and a search target is searched from all elements from the positive index of the generation N + 1 to the positive index of the generation i + 1 for all the divided character strings. The document database search device according to 9 or 10. A document database indexing method for creating a generation-managed document database index comprising:
A character string is cut out for each record from an initial generation document composed of a plurality of records having a uniquely identified ID, and the extracted character string and an appearance character position of the character string are shown in pairs. An index, and a document database recording step for recording the initial generation document in a document database;
A document entry step for entering the update document;
An update document determination step of determining a part of a character string of a changed portion between the initial generation document and the update document;
A positive index creation step of creating a positive index for the corresponding character string portion by using the character string length difference group generated in the change of the character string extracted and its appearance position and character string as an index element When,
A negative indexing step of creating, as a negative index, an initial generation index element to be deleted by the determination;
For the record deleted by the determination, a deletion record table creation step of creating the document ID as a deletion record table;
An update / registration step of updating / registering the created positive index, negative index and deleted record table as update information of a new generation;
A document database index creation method comprising: A document database indexing method for creating a generation-managed document database index comprising:
Documents that record documents from the initial generation to generation N composed of a plurality of record units each having a uniquely identified ID, and index information composed of a negative index, a positive index, and a deleted record table, respectively, in a document database A database recording step;
A document input step of inputting a generation N + 1 update document;
Update document determination step of determining a part of the character string of the changed portion from the update document of the generation N + 1, the index information including the negative generation index, the positive index, and the deletion record table, the document from the initial generation to the generation N When,
A positive index creation step of creating a positive index for the corresponding character string portion by using the character string length difference group generated in the change of the character string extracted and its appearance position and character string as an index element When,
A negative indexing step of creating, as a negative index, an initial generation index element to be deleted by the determination;
For the record deleted by the determination, a deletion record table creation step of creating the document ID as a deletion record table;
An update / registration step of updating / registering the positive index, negative index, and deleted record table created by the determination as update information of generation N + 1,
A document database index creation method comprising: A search character string input step for inputting a character string to be searched;
Update information comprising a plurality of generations of positive indexes, negative indexes, and deleted record tables, and a document database holding step for storing document information of each generation;
Document database search in which the input character string is analyzed and divided into character strings, and each divided character string is searched by using update information for a plurality of generations from the document database, an initial generation index, and a document. Steps,
A search result output step of outputting a record set obtained by the document database search step;
A document database search method comprising:

Images