JP2007018143A - Document retrieval device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To establish availability improvement and load distribution by a small number of computer resources while maintaining the consistency of data. <P>SOLUTION: A client device 101 of a document retrieval system 100 issues a document retrieval request, and a distribution device 102 distributes a document retrieval request from the client device 101 to document retrieval servers 200 and 300, based on a retrieval range. The distribution device 102 distributes the document retrieval request from the client device 101 to the document retrieval servers 200 and 300 according to the retrieval range and the operating status of the document retrieval servers 200 and 300 so that when one document retrieval server breaks down, the other document retrieval server can function by proxy. Index transferring parts 207 and 307 transfer index information for reflecting a retrieval index generated by index generating parts 201 and 301 on auxiliary retrieval index storage parts 308 and 208 of the other document retrieval servers 300 and 200. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a document retrieval technique, and in particular, attempts to improve availability and load distribution while maintaining data consistency as much as possible, and to use a limited computer resource without using special hardware. Is.

  Conventionally, various systems have been proposed to improve the availability and load distribution of databases and search systems.

  Patent Document 1 discloses a technique for realizing hot spare redundancy in a very large database management system. This document discloses that the transaction logger commits the transaction of the secondary system. Hot spare redundancy is common in the pursuit of data consistency and is important in database systems. However, since the hot spare concept requires a redundant secondary database, it is wasted as a non-trouble computer resource.

  On the other hand, Patent Document 2 discloses optimizing the timing of index reflection for applications in which some errors are allowed in synchronization. This attempts to balance the conflicting factors of synchronization error and equipment load. Although this method has an effect that load distribution can be expected depending on individual situations such as low update frequency, there is a possibility that the latest data may not be obtained, and there is a problem from the viewpoint of consistency.

  As described above, each of the conventional techniques has drawbacks, and combinations of them cannot cover the drawbacks.

  FIG. 1 shows an example in which the technique of Patent Document 1 is applied to a document search system. In this example, the documents in the cabinets 1 and 2 of the repository A are searched by the search systems A (master) and A ′ (standby), and the indexes 1 and 2 of the cabinets 1 and 2 are stored in the storage system (NAS / There is redundancy by SAN and RAID, which are held in Network Attached Storage / Storage Area Network, Redundant Arrays of Inexpensive Disks). Further, the documents in the cabinet 3 of the repository B are searched by the search systems B (master) and B ′ (standby), and the index 3 of the cabinet 3 is similarly held in the storage system. Normally, the search systems A and B are used, and when the search systems A and B fail, the standby search systems A 'and B' are used. In the figure, the solid line indicates the operation of the operation system, and the broken line indicates that the system is on standby. The example in the figure shows a situation in which the search system A has failed and the search system A 'is performing a search instead.

  FIG. 2 shows an example of a document search system realized by introducing another conventional category of redundant servers. In the example of FIG. 2, the master search system A distributes the search request to the slave search systems A1 and A2, and the slave search systems A1 and A2 use the indexes 1 and 2 to perform the search. In this example, for example, when the slave search system A1 fails, the remaining slave search system A2 processes all search requests.

There was no search method that could achieve both availability improvement and load balancing with less computer resources while maintaining data consistency.
JP 11-327991 A JP 2001-14336 A

  The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide a search system that can achieve both improvement in availability and load distribution with a small number of computer resources while maintaining data consistency.

  In the configuration example of the present invention, in order to achieve the above-described object, each of a plurality of prepared servers is set as an active standby, and information is reflected mutually when an appropriate trigger occurs. Each server processes a search request in the original range. Then, index information is acquired from another server at an appropriate timing, and the index of the other server is held. When a predetermined server fails, other servers also execute a search service for the failed server in addition to the original search service.

  The present invention will be further described.

  According to one aspect of the present invention, in order to achieve the above object, a document search device includes: a search index generation unit that generates a search index for a predetermined first document group; Search index holding means for holding the generated search index; search means for searching for a document using a search index held in the search index holding means in response to a document search request from a user; Auxiliary search index holding means for holding a search index of another document search apparatus for the second document group, and the auxiliary search index when the search means has a search request for the predetermined second document group A document search is performed using the search index of the holding means.

  In this configuration, the document search device that is operating normally acts as a search service for other failed document search devices, so it is possible to improve availability and load distribution with fewer computer resources while maintaining data consistency. Can be compatible.

  In this configuration, it is preferable that a search index for the predetermined second document group held by another document search apparatus for the predetermined second document group or a difference thereof is copied to the auxiliary search index.

  The auxiliary search index holding unit is typically a search index holding unit that holds a search index used for document search by another document search apparatus for the predetermined second document group. That is, taking two search devices as an example, one search index storage means operates as the other auxiliary search index storage means.

  According to another aspect of the present invention, in the document search system comprising a plurality of document search devices, each of the plurality of document search devices: a search index generation means for generating a search index for the corresponding document group A search index holding unit that holds the search index generated by the search index generation unit; and a document search using a search index held in the search index holding unit in response to a document search request from a user. And a search request for a document group to be searched by the other document search apparatus. The search means has a search index holding means for holding a search index of another document search apparatus. Sometimes a document search is performed using the search index of the auxiliary search index holding means. Unishi to have.

  Even in this configuration, it is possible to improve availability and load distribution with less computer resources while maintaining data consistency.

  In this configuration, a distribution unit that distributes the search request to one of the plurality of search devices according to the search target document group may be further provided.

  Further, detection means for detecting operating states of the plurality of document search devices is further provided so that a search request for a document group of a document search device that is not in an operating state is sent to another document search device in an operating state. Also good.

  The present invention can be realized not only as an apparatus or a system but also as a method. Of course, a part of the invention can be configured as software. Of course, software products used to cause a computer to execute such software are also included in the technical scope of the present invention.

  These and other aspects of the invention are set forth in the appended claims and will be described in detail below with reference to examples.

  According to the present invention, since the document search apparatus that is normally operated acts as a search service for other failed document search apparatuses, it is possible to improve availability and load distribution with less computer resources while maintaining data consistency. Can be compatible.

  Examples of the present invention will be described below.

  FIG. 3 schematically shows a document search system 100 according to an embodiment of the present invention, and FIG. 4 explains an operation example of the document search system 100. In this example, the document search system 100 includes two document search servers 200 and 300. However, the present invention is not limited to this, and three or more document search servers may be used. The functional blocks constituting the document search servers 200 and 300 are realized by cooperating the hardware resources and software resources of the servers.

  In FIG. 3, the document search system 100 includes a client device 101, a sorting device 102, and document search servers 200 and 300. Each component is appropriately connected via a communication network. The client device 101 issues a document search request, and is realized by various hosts such as a personal computer, for example. The distribution device 102 distributes the document search request from the client device 101 to the document search servers 200 and 300. In this example, as shown in FIG. 4, for example, the document search server 200 provides a search service for the documents in the cabinet 1 and the cabinet 2 and the other document search server 300 searches for the documents in the cabinet 3 and the cabinet 4 in a normal state. Service is provided. When one document retrieval server fails, the other document retrieval server acts as a substitute. The distribution device 102 distributes the document search request from the client device 101 to the document search servers 200 and 300 according to the search range and the operation status of the document search servers 200 and 300. The distribution device 102 can be realized using, for example, a redirect mechanism. When issuing a search request on the web base, a distribution destination can be described by a CGI program.

  The document search server 200 and the document search server 300 complement each other and provide a document search service. That is, the document search server 200 provides a document search service for documents in the document group storage unit 400 as a primary server. The document group storage unit 400 includes, for example, cabinets 1 and 2 as shown in FIG. Holds the document. The document search server 300 provides a document search service for documents in the document group storage unit 500 as a primary server, and the document group storage unit 500 stores documents in cabinets 3 and 4 as shown in FIG. It is to hold. At the same time, the document search server 200 provides a document search service for a document in the document group storage unit 500 as a secondary server when the document search server 300 is out of order. The document search service for the document stored in the document group storage unit 400 is provided as a secondary server. Although the document group storage unit 400 is shown separately from the document group storage unit 500 described later, these may be a single storage unit. In FIG. 4, the repository part is shown as one functional part.

  The document search server 200 includes an index generation unit 201, a search index storage unit 202, a search expression input unit 203, a search unit 204, a search result output unit 205, a heartbeat transmission unit 206, an index transfer unit 207, and an auxiliary search index storage unit 208. Etc. are configured.

  The index generation unit 201 refers to the document in the document group storage unit 400, generates a search index for the document, and stores it in the search index storage unit 202. The search index may be a full-text search index or a related document search index. The search formula input unit 203 receives the search formula of the search request sent from the client device 101 via the sorting device 102. The search unit 204 refers to the search index storage unit 202 to identify a document that matches the search formula and sets it as a search result. The search result output unit 205 supplies the search result to the client device 101.

  The index transfer unit 207 transfers index information so that the search index generated by the index generation unit 201 is reflected in the auxiliary search index storage unit 308 of another document search server 300. Typically, a transaction log for writing a search index to the search index storage unit 207 is recorded, and this is triggered at a predetermined timing and committed to the auxiliary search index storage unit 308 of the document search server 300. The predetermined timing may be the passage of time or the number of documents in which search indexes are newly registered.

  The auxiliary search index storage unit 208 of the document search server 200 receives the search index from the index transfer unit 307 of the document search server 300, and uses the search index for the document in the document group storage unit 500 that is the scope of the search service of the document search server 300. Hold.

  The heartbeat transmission unit 206 transmits a heartbeat signal to the distribution device 102 at a predetermined time interval when the document search server 200 is in operation. When the distribution apparatus 102 receives a heartbeat signal from the heartbeat transmission unit 206 of the document search server 200, the distribution apparatus 102 determines that the document search server 200 is in operation and sends a search request for a corresponding search range to the document search server 200. Send to.

  If the heartbeat signal from the heartbeat transmission unit 206 of the document search server 200 is interrupted and the heartbeat signal from the heartbeat transmission unit 306 of the document search server 300 is continued, the sorting apparatus 102 originally The search request to be transmitted to the document search server 200 is transmitted to the document search server 300. On the contrary, when the heartbeat signal from the heartbeat transmission unit 306 of the document search server 300 is interrupted and the heartbeat signal from the heartbeat transmission unit 206 of the document search server 200 is continued, the distribution device 102. Transmits a search request that should originally be transmitted to the document search server 300 to the document search server 200. When the heartbeat from both of the document search servers 200 and 300 is interrupted, the distribution apparatus 102 returns an error to the client apparatus 101.

  The document search server 200 and the document search server 300 have the same configuration, and each part of the document search server 300 is denoted by a corresponding reference numeral in the 300s and detailed description will not be repeated.

  Next, an operation example of this embodiment will be described. As shown in FIG. 4, a primary server and a secondary server are set for each repository information (search A, search B). Cabinets 1 and 2 are assigned to search A (document search server 200), and cabinets 3 and 4 are assigned to search B (document search server 300). During normal operation, only the primary server is accessed for search. For example, for document search for documents in cabinets 1 and 2, indexes 1 and 2 are referred to via search A. In the registration process, access from the repository is performed only to the primary server, and after the registration to the primary server is completed, a trigger is generated at an appropriate timing to perform the registration process to the secondary server. For example, the search A is registered in the indexes 1 'and 2'. At this time, only the plain text of the text extracted word is registered for the full text search, and only the analysis result after text analysis is registered for the related document search, which is a very light process. An event such as the passage of time or registration of a predetermined number of new documents can be adopted as a trigger.

  When a failure occurs in the document search server 200 or 300, this is detected, the operation target is switched, and the secondary server is accessed. For example, when the search A fails, the search B also searches the cabinets 1 and 2, and the search B refers to the indexes 1 'and 2'. Since the search B (document search server 300) holds the indexes 1 'and 2' that are consistent at the trigger timing in addition to the originally included indexes 3 and 4, the search can be performed as usual. In addition, the consistency of registration can be maintained by holding registered documents in failure.

  In this example, the failure detection is performed by a general heartbeat check. However, it can also be determined that there is no response when performing registration / search processing in the primary server more simply.

  FIG. 5 shows a search service in this embodiment, search index update (index record generation, deletion, modification, etc.) during document update operation (document registration, deletion, editing, etc.), transfer operation of index information to a secondary server, etc. A control example is shown. In this example, the presence or absence of a search request is determined to provide a search service in response to the search request (steps S10 and S11), the presence or absence of document update is determined and the search index corresponding to the document update is updated (step S10). S12, S13). Further, it is determined whether or not the timing for transmitting the index information to the secondary server has arrived, and the index information is appropriately transferred to the secondary server (S14, S15). Furthermore, it is determined whether or not the heartbeat signal transfer timing has arrived, and the heartbeat signal is supplied to the distribution device 102 (S16, S17). When the document search server 200 or 300 fails, the heartbeat signal is not supplied to the distribution device 102, and the primary server is switched to the secondary server accordingly. In the example of FIG. 5, the search index is updated in accordance with the document update. However, the search index may be updated at a predetermined time interval or the like using a time stamp of the updated document according to the schedule. The index information transferred to the secondary server and reflected in the auxiliary search index storage units 208 and 308 is, for example, log information of transactions committed to the search index storage unit 202 of the primary server, and may be sequentially committed in the secondary server. .

  According to this embodiment, except for the critical timing of occurrence of a failure, the latest consistent information is maintained, and computer resources are not wasted during normal operation. In addition, since it can be realized with general hardware, the introduction cost is low.

  Next, a modification of the above embodiment will be described.

  In the above-described embodiment, although the configuration can be simplified, the consistency of registered data immediately after the occurrence of a failure cannot be maintained completely. For this reason, as shown in FIG. 6, the upper search layer has the same mechanism, and the lower storage layer can be shared such as NAS / SAN (Network Attached Storage / Storage Area Network) 600 and has redundancy by RAID. May be. This can increase data consistency when storage can be invested. In FIG. 6, portions corresponding to those in FIG. In FIG. 6, a broken line indicates an operation at the time of failure.

  In this modified example, when one server fails, the active server performs the substitution as shown by the broken line in FIG. 7, and only management information is exchanged between the servers.

  The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the invention. For example, although two servers are used in the above example, three or more servers may be used.

It is a figure explaining a prior art example. It is a figure explaining another prior art example. It is a figure explaining the structure of the Example of this invention. It is a figure explaining operation | movement of the said Example. It is a flowchart explaining the example of control of the above-mentioned Example. It is a figure explaining the modification of the above-mentioned Example. It is a figure explaining operation | movement of the said modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Document search system 101 Client apparatus 102 Distribution apparatus 200, 300 Document search server 201, 301 Index generation part 202, 302 Search index storage part 203, 303 Search expression input part 204, 304 Search part 205, 305 Search result output part 206, 306 Heartbeat transmission unit 207, 307 Index transfer unit 208, 308 Auxiliary search index storage unit 400, 500 Document group storage unit 600 NAS / SAN

Claims (8)

Search index generation means for generating a search index for a predetermined first document group;
Search index holding means for holding the search index generated by the search index generation means;
In response to a document search request from a user, search means for performing a document search using a search index held in the search index holding means;
Auxiliary search index holding means for holding a search index of another document search device for a predetermined second document group,
A document search apparatus, wherein a search is performed using the search index of the auxiliary search index holding means when the search means makes a search request for the predetermined second document group.   2. The document search apparatus according to claim 1, wherein a search index for the predetermined second document group held by another document search apparatus for the predetermined second document group or a difference thereof is copied to the auxiliary search index. .   2. The document search apparatus according to claim 1, wherein the auxiliary search index holding means is search index holding means for holding a search index used by another document search apparatus for the predetermined second document group for document search. In a document search system comprising a plurality of document search devices,
Each of the plurality of document search devices includes:
Search index generation means for generating a search index for the corresponding document group;
Search index holding means for holding the search index generated by the search index generation means;
In response to a document search request from a user, search means for performing a document search using a search index held in the search index holding means;
Auxiliary search index holding means for holding search indexes of other document search devices,
A document search apparatus characterized in that when the search means makes a search request for a document group to be searched by the other document search apparatus, a document search is performed using the search index of the auxiliary search index holding means.   A document search system further comprising distribution means for distributing a search request to one of the plurality of search devices according to a search target document group.   5. The apparatus according to claim 4, further comprising detection means for detecting an operation state of the plurality of document search devices, and sending a search request for a document group of the document search device that is not in an operation state to another document search device in an operation state. 5. The document search system according to 5. A step of generating a search index for a predetermined first document group by a search index generating means;
A search index holding unit storing the search index generated by the search index generation unit;
A search means for performing a document search using a search index held in the search index holding means in response to a document search request from a user;
The auxiliary search index holding means stores a search index of another document search device for the predetermined second document group,
A document search method characterized in that a document search is performed using the search index of the auxiliary search index holding means when the search means has a search request for the predetermined second document group. A step of generating a search index for a predetermined first document group by a search index generating means;
A search index holding unit storing the search index generated by the search index generation unit;
A search means for performing a document search using a search index held in the search index holding means in response to a document search request from a user;
An auxiliary search index holding means is used for causing a computer to execute a step of storing a search index of another document search device for a predetermined second document group; and
A computer program for searching a document, wherein when the search means makes a search request for the predetermined second document group, the search is performed using the search index of the auxiliary search index holding means.