KR100911413B1 - Information System Having Per-Document Basis Transactional Index Maintenance Toward DB-IR Integration, And Method Thereof - Google Patents

Abstract

The present invention relates to an information retrieval system and a method having a document-level dynamic index management characteristic for integrating a database (DB) and information retrieval (IR). An auxiliary index store for storing; An index pulsing device for moving the posting information that reaches a predetermined threshold size among the posting information stored in the secondary index storage device to the main index storage device; And an information searcher for searching for desired information including documents inserted, changed or deleted online by using the posting information stored in the main index storage and the sub index storage. Integrate (IR) document-level dynamic index management.

Database, IR, Integration, Document, Dynamic, Index Management, Posting Information

Description

Information retrieval system with document-level dynamic index management characteristics for database and information retrieval integration and its method {Information System Having Per-Document Basis Transactional Index Maintenance Toward DB-IR Integration, And Method Thereof}

The present invention relates to an information retrieval system and method for dynamically managing a document, and in particular, a Per-Document Basis Transactional Index Maintenance for integrating information retrieval (IR) with a database (DB). An information retrieval system having a characteristic and a method thereof are provided.

One of the indispensable actions for human intellectual activities in the information society is the collection, accumulation, retrieval and use of information based on advanced science and technology. Personally, I seek ways to obtain the necessary information and data more quickly and accurately and use it for R & D or decision making. Socially, various information needs in each sector of society are the most important task in an efficient information society. Here is how to build, maintain and operate an IR system that can be met by IR.

The information retrieval system refers to a system that provides information requesters by quickly finding information that meets their needs from a database that has been collected, processed, and processed in advance in an easy-to-find form. In this form, the search results output from the information system may be reported by telephone, fax, or mailed, and the search results may be sent online to the home or office of the requester who owns the computer via the Internet. .

It is divided into reference search, fact search, and full-text search according to the kind of information accumulated in the database. Reference retrieval is a search of bibliographic citations in the literature that focuses on the subject that the requester wants to know. This is the case of online information retrieval systems such as DIRLOG and BRS.

Fact Search can search general data, numerical data, and fact data, such as Chemical Abstracts Service On-line (CAS) for searching for chemicals, and Electronic Information Service (EMIS) for data of semiconductor materials. Belongs to.

The full-text search allows you to search not only bibliographic data but also all relevant sentences and texts from the database of the full text of the literature as needed. Lexis or Westlaw can search legal information. ), Nexis, where you can search for newspaper articles, and Dow Jones / Retrieval.

Accordingly, the present invention has been made to solve the above problems, and a first object of the present invention is to provide an information retrieval system and method having a document unit dynamic index management characteristic for integrating a database (DB) and information retrieval (IR). There is.

A second object of the present invention is a document-based dynamic index management feature for database (DB) and information retrieval (IR) integration using a method of automatically moving over-index information of an auxiliary index store to a main index store. An information retrieval system having the same and a method thereof are provided.

An information retrieval system having a document unit dynamic index management characteristic for integrating a database (DB) and an information retrieval (IR) according to the present invention for achieving the above object is an online information retrieval system for dynamically managing documents. An auxiliary index store for storing posting information of a document to be inserted, deleted or changed in a transactional manner; An index pulsing device for moving the posting information that reaches a predetermined threshold size among the posting information stored in the secondary index storage device to the main index storage device; And an information searcher for searching for desired information including documents inserted, changed or deleted online using the posting information stored in the main index storage and the sub index storage.

The secondary index store stores posting information of documents entered online in a secondary storage on a disk, processes each document and all index information extracted from the document in transaction units, and records all change records in a log on disk. It is characterized by.

The index pulsing apparatus moves the posting information to the main index storage when the length of the posting information for a specific word of the sub index storage reaches a given threshold size, and deletes the posting information stored in the sub index storage. It features.

The information searcher can accurately search for inserted, deleted, and changed documents online by utilizing both the posting information of the main index storage and the posting information of the secondary index storage for a given query.

In addition, the information retrieval method according to the present invention for achieving the above object comprises the steps of: (a) storing the posting information of the document to be inserted, deleted, changed online in a secondary index storage in a transactional manner; (b) moving the posting information, which has reached a predetermined threshold size, from the posting information stored in the secondary index store to the main index store; And (c) searching for desired information including documents inserted, changed, or deleted online by using the posting information stored in the main index storage and the sub index storage.

The secondary index store stores posting information of documents entered online in a secondary storage on a disk, processes each document and all index information extracted from the document in transaction units, and records all change records in a log on disk. It is characterized by.

In the step (b), when the length of the posting information for a specific word of the auxiliary index storage reaches a given threshold size, the posting information is moved to the main index storage, and the posting information stored in the auxiliary index storage is deleted. It is characterized by.

The information retrieval method may accurately search for inserted, deleted, or changed documents online by using both the posting information of the main index storage and the posting information of the secondary index storage for a given query.

According to an information retrieval system having a document-level dynamic index management characteristic for integrating a database (DB) and an information retrieval (IR) according to the present invention, and according to the method, the over-index information of the sub-index storage is stored in the main index storage. The database and information retrieval (IR) are integrated using an automatic movement method. Therefore, in the present invention, the information searcher can search for desired information including documents inserted, changed or deleted online using the posting information stored in the main index storage and the sub index storage.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Example

First, a brief summary of the present invention is as follows.

While information retrieval (IR) and database (DB) are being developed independently, in information systems such as healthcare systems, bulletin boards, XML data management, and digital libraries, the new need to support both data management and efficient retrieval simultaneously There has been a demand. Recently, DB-IR integration issues have emerged in this field of research.

Due to the large gap between DB and IR, different approaches have been applied to index maintenance for new document processing. While DB doesn't have a pure mechanism for building IR-type indexes, it extends the SQL layer to handle text fields, whereas IR generally doesn't have the concept of integrity constraints. Treated as the logical unit of index management. However, a transaction in the addition or modification of a document for DB-IR integration must provide the management of the postings list accompanying the document at the same time. Include transactional indexing.

The present invention relates to the performance of several schemes for per-document transactions for inserting documents: direct index update, stand-alone auxiliary index and pulsing auxiliary index. Evaluate.

The results tested in the present invention show that the pulsing secondary index is a method in which the posting lists of long secondary indexes are updated in-place with the primary index while the short lists are updated directly with the secondary index. It can be seen that it can be an attractive candidate for indexing text fields in.

1. Introduction

The world of data has been developed from two main perspectives: structured relational data models and unstructured text models. Two unique cultures, databases (DB) and IR, now have a natural confluence on the web, using a semi-structured XML model. As web-style search becomes a ubiquitous tool, the need for integration of these two perspectives becomes more important (Consens and Baeza-Yates [13]).

DB-IR integration is an area of recent interest in the research area of database areas [15, 1, 10] and information retrieval areas [3, 13, 6].

There are several approaches to merging the structure data management of the database and the text search capabilities of the IR. The first approach, the most commonly adopted architecture in many information systems, simply combines existing DB and IR engines at the application or middleware level. This is called the middleware method [10]. In this middleware approach, the DBMS engine manages the documents, and the Information Retrieval System (IRS) is involved in supporting index building and user query evaluation.

Web search engines use a methodology in which documents are gathered by web crawlers, which are DB-type but with very simplified administrative characteristics, and indexes-although generally built offline-are built by individual index servers. Take it. While this middleware configuration is common in the real world of information services, it is very difficult to synchronize DBMS document content and IRS index information, resulting in a document-index gap. Moreover, this approach is not powerful enough to take full advantage of the DB and IR capabilities, for reasons such as a black box from an IR perspective, and so on.

The second approach extends the DBMS by loosely coupling IR functions. This is called 'loose coupling' [30]. Commercial DBMS vendors provide text extensions implemented using higher-level (typically, SQL-level) interfaces. Oracle's data blades, DB2's cartridges, and Informix's extenders are among these classes. Although loose coupling can be easily implemented, this loose coupling violates many of the assumptions inherent in current database systems [1], which allows new data types or functions to be implemented in large databases when high performance is required. Undesired [30].

In the QUIQ engine, another example of loose coupling, the hybrid IR-DB system of the DBMS holds all the base data, and the external index server maintains the index [21]. The TopX engine, which combines search with semi-structured data (XML) using Top-k search techniques, is built on a commercial RDBMS (Oracle) [28]. MonetDB / X100, an experimental relational database engine, has shown similar efficiency and effectiveness by building an inverted list of TREC-TeraByte text data as individual relational tables [20].

The third approach is to extend the DBMS by strongly combining the IR characteristics [30] or vice versa. Therefore, it is called tight-coupling. In this strong combination of DBMS and IR characteristics, the new data types and their corresponding IR behavior are integrated into the core of the DBMS engine in the extensible type layer. A well known example of this category is the Odysseus system [30]. Odysseus is an object-relational database management system (ORDBMS) with strongly coupled IR characteristics. There is a way to integrate some DB functions in the IRS in the opposite direction to the strong combination of IR in the DB. In this way, transactional data management is merged into the core of the IRS engine [17]. In the present invention, this kind of IR-extension engine is called IRMS (Information Retrieval & Management System).

The final approach is to design a DB-IR integrated system from scratch. This approach is discussed in [10, 1]. The new DB-IR unification system scheme has been suggested to have the advantages of structural data independence, generalized scoring, and a flexible and powerful query language [1]. Chaudhuri and colleagues suggest that a storage-level core system with RISC-style functionality in DB-IR integration is the most reasonable configuration [10].

1 is a diagram illustrating an evaluation result of an information system based on a DB and an IR approach. Where IR on DB Tight coupling, DB-IR integration, Odysseus, DB Extender, DB2 cartridge, DB-IR middleware The DB-IR middleware approach, Web search engine, etc. are shown, showing that the DB and IR will climb on different ridges to the top of DB-IR integration.

In FIG. 1 a web crawler and web search engines are included for comparison. A web crawler is a program that collects indexed web documents, where web page content is managed by URLs as a key. From a web crawler's perspective, data management is essential for performance, but search is not an important feature for web crawlers. On the other hand, a web search engine is an IRS customized for searching tens of millions to billions of web pages. Web search engines typically rebuild the index from documents collected periodically by web crawlers. Thus, data management is not a major factor, but search characteristics are essential for web search engines.

In web search engines, web page content is handled by web crawlers, which can be regarded as extremely simplified database systems. Thus, web search engines, in short, belong to the DB-IR middleware component class.

The DB (DBMS) employs many additional features, including IR features, that were not of interest in the traditional database world [18]. IRS faces user needs such as transaction processing, joins, materialized views, triggers, and the like. Although DB and IR have different starting points, they will eventually converge on a single goal of DB-IR integration.

To reach the goal, you will have to go through a long and difficult journey with many problems to solve (see [10, 18, 3, 1] for more details on these problems). One of the fundamental problems with DB-IR unification is dynamic index management, where its performance on text fields is a major obstacle in both IR and DB. Research on index management will be discussed at the end of this section.

From an IR perspective, in a dynamic search environment where both index updating and retrieval are performed simultaneously, index management for text search has been extensively studied over the past few years [25, 24, 9, 23]. In order to implement an index update scheme in IR, it is easy to bundle and index multiple new documents and merge them with existing indexes [24, 23]. By buffering the index for new documents, the document cost of updating the index can be significantly reduced because they will share many terms. These approaches manage the posting list for incoming documents in an in-memory data structure, and if they meet certain critical conditions, such as given memory size, time interval, and number of new documents, The in-memory index is merged into the on-disk main index.

As in the DB-IR middleware approach, IRs typically transfer the responsibility for data management to other tools or software such as a DBMS. Moreover, text search systems do not have the concept of integrity constraints because they can extract the index from source documents at any time. Thus, in IR, updates of the index to input documents do not necessarily have to be concurrent, and most of the IR accesses consider a bunch of new documents as the logical unit of dynamic indexing in index management.

But things are changing urgently. While dealing with the IR field for more than a decade, the present invention faces user demands for rapid failure recovery of IR content as well as rapid intermediate index updates. The document stack approach does not meet this requirement because in-memory postings can be lost in the event of a failure, and rebuilding them can be too time consuming. Also, because site managers typically do not know deeply about the text system they are using, communicating this process to site managers can also be a very complex task.

In addition to this practical aspect, the academic consideration of DB-IR integration in the present invention emphasizes that there must be an integrity constraint between the document and the accompanying index. If a part of the index is likely to be lost in some circumstances, it can be said to be far from data integrity because data integrity can be defined as simply preserving data for their “intended use”. Since retrieval can be the primary intended use of the data, document-index integrity must be ensured in DB-IR integration. To support document-index integrity, DB-IR integration will have to support document-level transactions, where the unit of work has ACID characteristics that include one document and its index information. And all transactions must log, like traditional DBs, that can be replayed back into the system correctly before failure or interruption occurs.

In the present invention, such a policy will be referred to as document-level transactional index management.

The present invention proposes two on-disk subindex schemes for document-based transactions of index management and compares their performance with direct update schemes. Transactional index management is expected to perform poorer than bulk documenting because it inevitably burdens you with heavy disk access to numerous posting updates and on-disk transaction logs.

Auxiliary indexing is an on-disk auxiliary data structure that includes a posting list of newly arrived documents and modified documents. The first subindex, an independent subindex, accumulates an unlimited list of incoming documents. The second approach, a pulsing secondary index, accumulates a posting list of secondary data structures until a certain threshold is reached. When a posting list builds up to a threshold, the posting list is merged into the main index.

These approaches were tested by KRISTAL-IRMS, which is bibliographic data for journal articles, where each data contains more than 100 unique words. Here, KRISTAL stands for Knowledge Retrieval In Science and Technology Affiliated Literatures. This ultimately aims for an IRMS for full DB-IR integration. The source of KRISTAL-IRMS is available at http://www.kristalinfo.com .

The secondary approaches outperform the direct update method, and the pulsing method outperforms the single secondary index method. As expected from frequent disk-access overhead, the results in the present invention show a document insertion speed of 2 to 5 seconds in on-disk assisted access, which is significantly lower than approximately 0.1 seconds of the document stack approach. This leaves much room for future research.

Section 2 provides a brief overview of relevant research on online index management. Section 3 describes transaction management, in which the unit of transaction is a single document to be entered, rather than the document stack of a traditional index management approach in the IR field. Experimental methods and results are described in sections 4 and 5, respectively. Finally, in section 6, the conclusions of the present invention and suggestions for future studies will be made.

2. Related Research

It has already been proven that inverted files are the most efficient data structure in high performance retrieval systems for large text data [33], and there has been extensive research so far in devising inverse list indexes [16, 31, 32]. There are also studies that devise inverted lists that can efficiently handle document insertion, deletion or modification [29, 21, 25, 24, 23, 9]. These studies approached the index management problem by treating as many documents as possible as units of work for efficiency.

A pulsing technique as an in-memory buffering scheme, in which short lists are kept in lexical structure and long lists are pulsed in on-disk auxiliary heap structure, was introduced in [14]. The system and Spider IR system disclosed in [8] proceed with index changes in-place. The Gold Mailer system [4, 5] periodically makes changes. Lucene [7], a publicly available search engine framework, applies changes based on memory thresholds. The studies in [29, 27] studied space allocation schemes in in-place schemes, where short lists are grouped into fixed-size buckets to reduce the space management problem for many small station lists. And the overallocation method was further extended in [26]. Granularity in delivering index updates to disk was considered in [12]. A technique called the landmark-diff update method has been introduced to improve index update performance for existing documents whose content has changed [25].

Most index management schemes work by building an in-memory reverse file by storing postings for incoming document bundles in main memory. In order to pay off the disk-access cost, the operation of writing or delivering these in-memory postings to the existing on-disk main index is delayed to some predetermined threshold. The QUIQ engine, a DBMS extender for text fields, moves index changes to on-disk indexes based on a fixed time interval [21]. Work in the IR generally delivers accumulated change information to disk by events that exceed the memory threshold or the number of documents updated [23, 9]. As such, the changed documents share the index words that need to be updated, so disk access can be greatly reduced in the index-managed indexing scheme, thereby providing improved indexing performance. However, in the event of a failure, the data integrity between documents and indexes can be compromised in these document stack approaches.

The Chunk method, in which a document set is divided into "chunks that order documents", has been newly introduced to process a sequence query for structural data values of a relational database under a document-intensive environment [19]. However, because the chunking method is designed for numerical data types and only document-intensive systems, it is difficult to apply to text data types and document-intensive environments. The index data structure of HYB was introduced to support context-dependent prefix retrieval: The block index HYB is composed of many (word_id, document_id) pairs and can be augmented using the locations of words, but in online index management The performance of the HYB data structure is still unclear.

3. Transactional  Index management ( Transactional Index Maintenance )

Most IR systems have the opportunity to consider new blocks of documents-for efficiency-as a unit of index management, because there is no concept of integrity constraints. However, for DB-IR integration, and to meet field requirements from database administrators in the text services area, index management must be supported at the on-disk storage level, not in-memory. This means that the logical unit of the index management process should consist of each document (not a large number of documents) and the accompanying index information. In addition, the DB-IR integrated system must be able to record logs during index management to support the ACID characteristics of the DBMS.

In the present invention, the pulsing secondary indexing method, which is a hybrid method for three methods, that is, a direct index update, a single secondary index, and the two, is tested for document-based index management that considers each document and its index as a transactional unit. It was.

In all experiments of the present invention, all changes in index data are logged to disk in document-by-document transactions. If the transaction is successful, the log is discarded. Otherwise, if the transaction fails for some reason, the log is used to return to the immediately preceding transaction state. In the context of the present invention, all station lists contain all location information, ie accurate information for every occurrence of a given term. Since the secondary approach in the present invention includes two separate on-disk indexes, the initial index for existing documents is called the primary index, and the added or modified index for newly entered documents is called the secondary index.

3.1. Manual index update ( Direct Index Update )

The way to update the index directly is to simply modify the index by list, which contains information about the new document. Manually updating the main index is an intuitive and very simple index update. Each time a document is entered into the system, a posting list for each term is added to the main index. This usually requires relocating existing on-disk posting lists for each term in the new document. Since new documents generally have a number of terms, this kind of direct in-place update to the main index requires a fairly time-consuming relocation of the main index's posting list. Overallocation of the posting list in the main index-leaving some free space after every posting list-has been proposed to reduce this costly operation [9]. However, if the over-allocation size is unpredictable in a real database and there are few updates and additions, or if there is a huge amount of documents inserted and accumulated in the allocated free space of the posting list, then there is no over-allocation. The over-allocation policy could invalidate. Therefore, the present invention does not consider the over-allocation scheme in this experiment.

In the direct update method, query processing performance is similar to that of a completely rebuilt database, because the posting list is guaranteed to be continuous. However, frequent relocation of the main index's posting list will significantly reduce update performance. In general, a document contains a number of terms, so index updates will involve a lot of disk access, in which case you can afford to pay only if the database is very small or the frequency of updates is very low.

3.2. Exclusive secondary index ( Stand - alone Auxiliary Index )

The direct update method is expected to cause frequent relocation of long posting lists. If each relocation size is reduced to be small enough, the overall update performance can be improved. In order to reduce the size of each relocation, the present invention introduces an on-disk auxiliary index for input documents. Each time a new document arrives, the indexing result is added to the secondary index instead of the primary index. As shown in Fig. 2, the deletion is added to the deletion list, and the modification of the existing document is performed by (deletion + insertion) using the correction list. Since the secondary index is empty in the initial state of the database creation, the posting list from the input documents is short and can be easily inserted into the secondary index. However, as new documents accumulate, the secondary index's posting lists grow in size, and their performance decreases due to frequent rearrangement of such large posting lists.

In the present invention's implementation of the exclusive secondary index scheme, the data structure of the secondary index is the same as the primary index using B ⁺ -tree for lexicographical ordering of key storage. This is inevitable to support range operations such as (PUBDATE> = 20010101 and PUBDATE <= 20061231) and right truncations or wild-carding such as TITLE: (index * AND strateg *). It is a choice. Since the secondary index has the same data structure as the primary index, updating the new posting list of the secondary index can be worse than the primary index when a lot of new documents are accumulated. At this point, the entire posting list of the sole subindex must be merged into the main index and empty to ensure update performance. In the present invention, this bulk process is called index optimization. KRISTAL-IRMS uses an in-place update method for this purpose [29, 24, 23]. During the in-place update of the secondary index to the primary index, query processing stops.

In the index management schemes in the present invention, the deletion of a document is simply adding to the deletion list, and the modification of the existing document is handled by deletion + insertion-since the cost of deletion can be ignored, the modification is performed in the same operation as the insertion. Can handle Therefore, the present invention focuses only on the insertion of a new document.

FIG. 2 is a diagram showing a pulsing subindex with a pulsing threshold value DF = 3. For clarity, a detailed structure including a term frequency and a physically separated location list are omitted. Omitting the pulsing function is equivalent to a single subindex without in-place updates.

The pulsing secondary index shown in FIG. 2 includes a document table, a main index, an in-place update, a delete list, an update list, and an auxiliary index. ), And so on.

3.3. Pulsing  Secondary index ( Pulsing Auxiliary Index )

The advantage of introducing an auxiliary index is that the auxiliary index itself has a small size. However, as new documents accumulate, their size increases to the point where index optimization is needed to support correct index management for documents that will be entered as well as future subindexes. If the size of the subindex can be kept compact enough throughout the processing of the input documents, the index update performance will not be significantly degraded, as in the initial stage of the single subindex approach.

In both the direct update method and the exclusive subindex method, the most expensive task is to relocate the long posting list. Moreover, since high document frequency terms such as a "," of ", and the" usually occur simultaneously in one document in English documents, direct index updates and stand alone assist methods are more important than relocation of long posting lists. Will occur frequently. The pulsing subindex is a hybrid of direct index update and single subindex. In this way, to reduce the number of long posts being processed simultaneously in the secondary index, all secondary posting lists longer than the given threshold are updated in place with the primary index (this process is called pulsing) and then discarded in the secondary index. Since terms with high frequency usually occur at the same time, pulsing long posting lists with the main index is evenly distributed throughout the transactions, and the frequency of occurring simultaneously in one transaction is greatly reduced. Since this approach is similar to the pulsing technique by Cutting and Pedersen [14], according to their research, we have named this approach the pulsing secondary index.

For example, suppose the pulsing threshold for long posting lists is document frequency 500. And enough documents have already been indexed, so the posting lists of some of the terms in the secondary index are large enough to reach the pulsing threshold. For example, suppose the document frequency in the secondary index increases to 497 for a ', to of 498, and the' to 499. For new documents containing a ',' of ', and the', the pulsing method directly updates the posting lists of subindexes for a 'and' of '(current document frequency is 498 and 499, respectively). Because the value 599 has not been reached). In the case of the word 'the', the document frequency has reached the threshold of 500, so after updating the post list of 'the' of the secondary index to the main index's posting list, the posting list of 'the' of the secondary index is Empty This means that processing of terms with high document frequency has overhead to update in place as the primary index, but each posting list of secondary indexes is limited to remain below the threshold. In addition, since high frequency terms do not occur exactly at the same time in one document, in-place updates to the main index are evenly distributed throughout the insertion of new documents.

Since all secondary posting lists longer than the given threshold are updated with the main index, the size of the pulsing secondary index will be almost constant regardless of the number of documents inserted. There is no posting list longer than the threshold, and the updating of the posting lists for all unique terms in a document becomes much smaller than the direct updates and the sole subindex schemes. Thus, a single document insertion task must handle a large number of words, but consists of updating small posting lists, compared to direct updates or a single subindex.

4. Experiment ( Experiments )

The equipment used in this experiment was a Linux machine with a dual Pentium Xeon 3GHz CPU, 8GB of memory and RAID-5 SCSI storage. Experiments were performed without disks, memory, or other processes that were over-using the CPU, i.e. with little load.

The data used in the experiment was bibliographic information of the journal, and three kinds of document sets were prepared according to the number of documents from 1 million bibliographic information, and 10,000 additional bibliographic information was prepared as a test set for the insertion test (Fig. 3). ). According to the number of documents, 10K-SET (12MB) with 10,000 citations, 100,000 100K-SET (121MB), and 1 million 1M-SET (1,310MB). Each document set was previously built with bulk loading. In this state, an experiment was performed to insert 10,000 documents of the test set online. (Direct update experiments for 100K-SET and 1M-SET were omitted due to heavy time constraints). The number of documents differing according to the document set was selected to investigate the cost of the three index management methods tested in the present invention according to the number of documents. The final indexes, including global location information for all terms in the 10K, 100K, and 1M datasets, were 29 MB, 246 MB, and 2,373 MB, respectively. The pulsing threshold in the pulsing auxiliary index method was set to a document frequency of 500.

3 shows semi-structured sample data.

FIG. 3 is the first and second bibliography, consisting of @DOCUMENT (record delimiter), TITLE, AUTHOR, JOURNAL, VOLUME, NUMBER, PAGE START, PAGE END, PUBDATE (issue date), ABSTRACT and KEYWORDS. Table 1 shows the average term count for all sections of a document set (one section is synonymous with a field or column in the database world). To keep the dictionary order of terms in the inverted file, it is common to separate the lexical data structure and the posting list from each other. At KRISTAL-IRMS, each posting list physically separates the frequency / document identifier information and the exact location information within the document.

Referring to Table 1, it can be seen that one document contains more than one hundred non-overlapping unique terms. This means that KRISTAL has access to three data structures for each word in the input document—a vocabulary list, a posting list, and a location list—so that adding one document requires access to the on-disk index more than 300 times. .

Table 1 below shows a brief database overview and statistics of index terms per document.

As shown in Table 1, KRISTAL-IRMS is a variable string (synonymous with var_char but has a 2GB length limit; typically, the DB has a 4K limit; KRISTAL denotes KSTRING), a fixed string ((KCHAR []), It supports data types such as Integer (KINT), and Floating Point (KFLOAT), and its index types are INDEX_AS_IS (index the entire section value as a string), INDEX_NUMERIC (index to floating point), INDEX_BY_CHAR (single index), You can choose between INDEX_BY_TOKEN (word index), several indexing types for Korean and Chinese characters, and several other types related to biological sequences such as DNA and proteins.

In the sub-index method, since the posting list for one word is distributed and stored in the main index and the sub-index, the posting access speed may be lowered compared to the continuous posting lists. To test the trade-off between query processing and index management of auxiliary methods, a set of three words with different document frequency ranges in 1M-SET-27,920 terms with (100 <= DF <1,000), (1,000 <= 7,071 terms with DF <10,000), and (DF> = 10,000) were extracted. We created a KRISTAL query that searches these words for the TITLE, ABSTRACT, and KEYWORDS sections-(TITLE: term) OR (ABSTRACT: term) OR (KEYWORDS: term). To evaluate the tradeoff between query evaluation and discontinuous posting lists of secondary indexes, a 1M + 10,000 document database in which the index was built by the rebuild method [23], 10,000 indexed using a single secondary index. For a 1M database with two additional documents, and 1M indexed using the pulsing assist method, the time to process these queries in the BOOLEAN MODEL was compared.

Finally, for 10K-, 100K- and 1M-SET, we evaluated 2,994 complex queries currently used as subject queries in the bibliographic search service (YesKISTI: http://www.yeskisti.net). These complex queries are composed by experts and the formula is very complicated. The search target section for these complex queries was constructed in the same way as the previous experiment (TITLE: compound query) OR (ABSTRACT: compound query) OR (KEYWORDS: compound query). Some examples of the 2,994 compound query are as follows.

yellow * / N8 (polyurethane * OR urethane *)

silicon AND (optic * / N8 signal *) AND module *

food * / N3 (wastewater * OR (waste / W1 water *)) AND treat *

ceramic * AND (bulletproof * OR (bullet / W1 proof *) OR (bullet / W1 resist *) OR (bullet * / N2 (protect * OR resist *)))

wood * / N5 (substitut * OR replacement *)

(catalyst * OR catalyzer *) / N5 (regenerat * OR ((precious OR valu * OR noble *) / N2 metal * / N5 recover *))

Because each compound query is frequently used by proximity operations (NEAR and WITHIN operations—in the form of / N n and / W n , respectively) with Boolean operators such as AND and OR, both posting lists and location lists This is useful for evaluating the access performance for a file (KRISTAL separates the location lists from the posting lists). Moreover, access efficiency to lexical structures from frequent right cut operations will also be reflected in the evaluation of composites. Compound queries were used to evaluate query processing performance against 10K, 100K, and 1M databases with an additional 10,000 documents inserted using the Rebuild, Single Secondary, and Pulsed Secondary indexing schemes.

5. Results

Results for the three index update schemes are shown in FIGS. 4, 5, and 6. Each point in the figures was obtained by averaging insertion times every ten insertions, in order to reduce the effect of biases in document lengths. All experiments were performed when the equipment was under light load, that is, while other important tasks were not accessing disks, memory, or CPUs.

4 shows the insertion time averaged for every 10 out of 10,000 input documents for a 10K set, showing the results of entering 10,000 documents into a 10K database.

Here, 10,000 documents have been indexed in bulkload mode before the experiment. As mentioned in section 3.1, Figure 4 shows that updating the index information of new documents directly to the primary index is very poor compared to the secondary index schemes (averaging 10,000, direct index update, sole secondary index and The pulsing subindex scheme represents processing speeds of 12.0, 4.37 and 1.97 seconds per document, respectively). This means that the direct update method of the main index is not suitable even in small text databases.

On the other hand, a single secondary index can improve the update efficiency, but slower as the document is inserted. The pulsing subindex, a hybrid method that applies a single subindex for short posting lists and direct updates for long posting lists, performs much better than a single subindex. Moreover, this approach shows almost stable performance even with many insertions. Thus, while index optimization (the bulk-mode in-place update of secondary posting lists to the primary index) is necessary for a single secondary index, it is not necessary for a pulsing secondary index.

As the input documents accumulate, the post lists of the sub-index become longer. There is no restriction on the length of the post list for a single secondary index. On the other hand, all posting lists of the pulsing subindex do not exceed the length of the pulsing threshold. Except for post lists that are pulsed by the primary index, all posting lists to be processed by the pulsing secondary index are shorter or equal in length than the posting lists of the sole secondary index. For example, suppose the word the 'occurs in all input documents, and the pulsing threshold is document frequency 500. For the 10,137th new document, the term the 'posting list in a single aid is 10,137 times the posting data structure, whereas only 137 times in the pulsing auxiliary index. In the worst case where the same document is entered repeatedly n times, the length of the total posting list to be processed in the single subindex becomes larger than that of the pulsing subindex (at least a multiple of n / pulsing threshold), which is shown in FIG. It makes a difference in performance.

FIG. 5 shows the insertion time averaged for every 10 of 10,000 input documents for a 100K set, with the time-limited direct update test omitted.

In FIG. 5, the experiment was performed on 100K-SET without the direct update method experiment due to heavy time constraints of 3 minutes or more per document under the same conditions. Averaging insertion times per document for 10,000 insert transactions for a 100K database, the isolated and pulsing subindex schemes yield throughputs of 4.62 and 2.26 seconds per document, respectively. In this experiment for 100K-SET, the single and pulsing auxiliary index approach is very similar to the experimental results for 10K-SET (FIG. 4).

In FIG. 5, as the addition of new documents accumulates, the performance of the single secondary index method is degraded, while the pulsing secondary index method shows a stable bias point that tended to be stable but was not observed in FIG. Apart from these biases, the pulsing method is superior to the single method, and its performance is stable regardless of the number of inserts of new documents.

FIG. 6 shows the insertion time averaged for every 10 of 10,000 input documents for the 1M set, with the time-limited direct update test omitted.

6 shows the results for the 1M data set. When averaging 10,000 insert transactions for a 1M database, the sole and pulsing secondary indexes show insertion rates of 4.71 and 4.38 seconds per document, respectively. As we observed the performance bias of the pulsing method for the 100K data set in this invention (FIG. 5), a much wider performance bias of the pulsing method is observed in the processing time plot for the 1M database (FIG. 6). In the pulsing method, once the term's document frequency reaches the pulsing threshold, the posting list of the term is added to the corresponding posting list of the main index. However, the main index of 1M is 100 times 10K and 10 times 100K. In general, this means that posting lists are on average 100 or 10 times longer. The main index's posting list can range from tens to hundreds of megabytes. Pulsing the secondary posting list by the primary index will obviously cause a relocation of that posting list in the main index. As in 1M-SET, when the main index becomes very large, the unit of pulsing operation becomes very large, and it takes a lot of time for logging as well as relocation. Therefore, larger biases are observed as the main index grows, which is due to the pulsing of the posting list's main index for high frequency words.

FIG. 7 is a comparison of average insertion time per document for updating 10,000 new documents for 10K, 100K, and 1M sets using a single subindex and a pulsing subindex.

The experimental results shown in FIGS. 4, 5, and 6 are summarized again in FIG. 7. For all three data sets, the performance of the pulsing subindex method outperforms the single subindex method. The average performance on a single subindex is almost constant regardless of the main index size-since the single subindex is independent of the main index: while the pulsing method is associated with the size of the main index due to pulsing. Decreases as the main index increases in size. Therefore, unfortunately, the performance of the pulsing sub-index will be worse than the single sub-index for document tables larger than 1M-SET (this situation can be avoided if you split the table into multiple tables so that the document collection is smaller than 1 million documents). have).

8 shows the average query processing time of terms with variable DF range after adding 10,000 new documents to 1M set, and FIG. 9 after adding 10,000 new documents to 10K, 100K, and 1M set. , Which shows the average query processing time for complex queries.

FIG. 8 shows low frequency terms (100 <= DF <1000), medium frequency terms (1000 <= DF <10,000), for a database constructed as a reconstruction (baseline), pulsing auxiliary index, standalone auxiliary method. And posting approach patterns of high frequency terms (DF> = 10,000). In all cases, the sole secondary index approach is significantly lower than the rebuilt database, while the pulsing approach is close to the rebuilt database (the monolithic approach is 2.6-3.7 times slower than the rebuild database, but the pulsing aid is Posting access performance is only 23% lower than the rebuild DB).

As with the secondary index approach with discontinuous attributes of the posting list, index management schemes inevitably include a tradeoff between index management performance and query processing performance. 8 shows that the tradeoff may be significant in a single secondary index but not in a pulsing secondary index. This is again demonstrated in the experiment evaluating the complex queries shown in FIG. For 1M databases, the query processing performance of the standalone and pulsing secondary indexes is 56% and 81% of the rebuild. By making the posting list contiguous, you minimize the number of disk accesses required to get the posting list, thereby maximizing query processing performance. The rebuild guarantees this situation. However, in the standalone and pulsing subindex schemes, the posting lists are divided into two parts, the main index and the subindex, and require more disk access than the rebuild scheme, which results in poor query processing performance (FIG. 9). ). However, the pulsing secondary index method of keeping the posting small has only 19% overhead in query processing, while the sole secondary index generates 44% overhead.

6. Conclusion and future research Conclusion and Future Work )

The present invention provides an evaluation of an index management method that supports several document unit transactions for dynamic index management. Experimental evaluation in the present invention shows that the pulsing secondary index method can be a competitive candidate for text indexing in actual DB-IR integration. On the other hand, the pulsing method also exhibits the disadvantages of biased index management time and slightly degraded query processing performance. To date, research has shown that the indexing efficiency of pulsing subindexes is still unsatisfactory compared to the document-level indexing schemes. The batch-based approach of keeping indexes of new documents in memory shows a management speed of 0.1 seconds per document, while the pulsing method averages 4.4 seconds per document on average for 1 million document sets.

An important issue with the pulsing secondary index method of in-place update and secondary indexes is the expensive relocation of long posting lists. As with previous studies designed to avoid the relocation of long posting lists in information retrieval [9], the performance of transactional index management can be significantly improved if this expensive operation can be avoided. As part of this work, it may be useful to introduce over-allocation schemes for secondary indexes. The index compression algorithm [31, 2] may further improve this performance by reducing the length of the posting lists.

Although KRISTAL-IRMS has evolved from the other direction of IR, many of the design principles in KRISTAL system construction are based on DB-IR storage-level core systems and DBs with RISC-style functionality in the DB-IR integration claimed in [11, 10]. The view of IR integration is in agreement. KRISTAL-IRMS, which employs Berkeley DB as a storage engine, supports full concurrent control and recovery, and storage-level single-table queries. KRISTAL targets IRMS (an evolutionary form of IRS-see Figure 1), which supports text data types at the storage level in addition to the various structural data types. KRISTAL's query hierarchy is implemented on a storage hierarchy that extends storage-level single table queries into multi-tables. In the query hierarchy, KRISTAL supports sorted query evaluation, structured query evaluation, classification, and efficient XML retrieval, while several types such as SQL type query language, table joins, formation views, ranking of structured data values, triggers, and query optimization DB characteristics still remain a problem to be solved.

Today we live in a time of extreme change where all design assumptions for all systems are reevaluated. Information retrieval and database researchers have long distanced each other, thereby separating them from separate, fixed worlds, IR and DB. However, customers want table joins and triggers at IR, and DB has begun requesting unstructured text processing and sorted query evaluation.

Example

10 and 11 are configuration diagrams of an information retrieval system according to a preferred embodiment of the present invention.

As shown, the information retrieval system according to the present invention includes an auxiliary index storage unit 10, a main index storage unit 20, an index pulsing device 30, and an information retrieval unit 40. As shown in FIG.

The secondary index storage unit 10 stores posting information of a document that is inserted, deleted, or changed online in a transactional manner. In addition, the posting information of documents entered online is stored in a secondary storage on disk, each document and all index information extracted from the documents are processed in a transaction unit, and all change records are recorded in a log on disk.

The main index storage unit 20 stores the posting information reaching a predetermined threshold size among the posting information stored in the sub index storage unit 10 by the index pulsing apparatus 30.

The index pulsing device 30 serves to move the posting information that reaches a predetermined threshold size among the posting information stored in the secondary index storage to the main index storage. In addition, the index pulsing apparatus 30 moves the posting information to the main index storage unit 20 when the length of the posting information for a specific word of the secondary index storage system reaches a given threshold size. The posting information stored in the sub index storage unit 10 is deleted.

Finally, the information searcher 40 searches for desired information, including documents inserted, changed or deleted online, using the posting information stored in the main index storage 20 and the sub index storage 10. do. In addition, the information searcher 40 can accurately search for inserted, deleted, and changed documents online by utilizing both the posting information of the main index storage and the posting information of the secondary index storage for a given query.

The present invention is not limited to the above-described embodiments, and various modifications and changes can be made by those skilled in the art, which should be regarded as included in the spirit and scope of the present invention as defined in the appended claims. will be.

[references]

[1] S. Amer-Yahia, P. Case, T. Roelleke, J. Shanmugasundaram, and G. Weikum. Report on the DB / IR panel at SIGMOD 2005.SIGMOD Record, 4 (4): 71.74, December 2005.

[2] V. N. Anh and A. Moffat. Improved word-aligned binary compression for text indexing. IEEE Transactions on Knowledge and Data Engineering, 18 (6): 857.861, 2006.

[3] R. Baeza-Yates, Y. S. Maarek, T. Roelleke, and A. P. de Vries. Third edition of the .XML and information retrieval. workshop. First workshop on integration of IR and DB (WIRD) jointly held at SIGIR'2004, Sheffield, UK, July 29th, 2004. SIGIR Forum, 38 (2): 24.30, 2004.

[4] D. Barbar´a, C. Clifton, F. Douglis, H. Garcia-Molina, S. Johnson, B. Kao, S. Mehrotra, J. Tellefsen, and R. Walsh. The Gold Mailer. In Proceedings of the Ninth International Conference on Data Engineering, April 19-23, 1993, Vienna, Austria, pages 92.99, 1993.

[5] D. Barbar´a, S. Mehrotra, and P. Vallabhaneni. The Gold Text Indexing Engine. In ICDE '96: Proceedings of the Twelfth International Conference on Data Engineering, pages 172. 179, 1996.

[6] H. Bast and I. Weber. The CompleteSearch Engine: Interactive, efficient, and towards IR & DB integration. In CIDR 2007, Third Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 7-10, 2007, Online Proceedings, pages 88.95, 2007.

[7] A. Bialecki, D. Cutting, S. Ganyo, C. Goller, O. Gospodnetic, M. Harwood, E. Hatcher, D. Naber, Y. Seeley, and S. Siren. The Apache Lucene Project. http://lucene.apache.org/ .

[8] E. W. Brown, J. P. Callan, and W. B. Croft. Fast incremental indexing for full-text information retrieval. In VLDB'94, Proceedings of 20th International Conference on Very Large Data Bases, September 12-15, 1994, Santiago de Chile, Chile, pages 192.202, 1994.

[9] S. B¨uttcher, C. L. A. Clarke, and B. Lushman. Hybrid index maintenance for growing text collections. In SIGIR '06: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, pages 356.363, 2006.

[10] S. Chaudhuri, R. Ramakrishnan, and G. Weikum. Integrating DB and IR technologies: What is the sound of one hand clapping? In Proceedings of the Second Biennial Conference on Innovative Data Systems Research (CIDR 05), pages 1.12, Asilomar, CA, USA, 2005.

[11] S. Chaudhuri and G. Weikum. Rethinking database system architecture: Towards a self-tuning RISC-style database system. In VLDB 2000, Proceedings of 26th International Conference on Very Large Data Bases, September 10-14, 2000, Cairo, Egypt, pages 1.10, 2000.

[12] T. Chiueh and L. Huang. Efficient real-time index updates in text retrieval systems. Technical Report ECSL-TR-66, Computer Science Department, SUNY at Stony Brook, 1999.

[13] M. P. Consens and R. A. Baeza-Yates. Database and information retrieval techniques for XML. In Advances in Computer Science-ASIAN 2005, Data Management on the Web, 10th Asian Computing Science Conference, Kunming, China, December 7-9, 2005, Proceedings, volume 3818, pages 22.27, 2005.

[14] D. R. Cutting and J. O. Pedersen. Optimization for dynamic inverted index maintenance. In SIGIR '90: Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval, pages 405. 411, 1990.

[15] A. P. de Vries and A. N. Wilschut. On the integration of IR and Databases. In Database issues in multimedia; short paper proceedings, 8th International IFIP 2.6 Working Conference on Database Semantics (DS-8), Rotorua, New Zealand, volume 138, pages 16.31, 1999.

[16] W. B. Frakes and R. A. Baeza-Yates. Information Retrieval: Data Structures & Algorithms. Prentice-Hall, 1992.

[17] GIIS. KRISTAL-IRMS: Information Retrieval & Management System. http://www.kristalinfo.com/ .

[18] J. Gray. The next database revolution. In SIGMOD '04: Proceedings of the 2004 ACM SIGMOD international conference on Management of data, pages 1.4, 2004.

[19] L. Guo, J. Shanmugasundaram, K. Beyer, and E. Shekita. Efficient Inverted lists and query algorithms for Structured Value Ranking in update-intensive relational databases. In ICDE '05: Proceedings of the 21st International Conference on Data Engineering (ICDE'05), pages 298.309, 005.

[20] S. H´eman, M. Zukowski, A. P. de Vries, and P. A. Boncz. Efficient and reexible information retrieval using MonetDB / X100. In CIDR 2007, Third Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 7-10, 2007, Online Proceedings, pages 96.101, 2007.

[21] N. Kabra, R. Ramakrishnan, and V. Ercegovac. The QUIQ engine: hybrid IR-DB system. In Proceedings of the 19th International Conference on Data Engineering (ICDE), March 5-8, 2003, Bangalore, India, pages 741., 2003.

[22] D. Knaus and P. Sch¨auble. The system architecture and the transaction concept of the SPIDER information retrieval system. IEEE Data Eng. Bull., 19 (1): 43.52, 1996.

[23] N. Lester, J. Zobel, and H. Williams. Efficient online index maintenance for contiguous inverted lists. Inf. Process. Manage., 42 (4): 916.933, 2006.

[24] N. Lester, J. Zobel, and H. E. Williams. In-Place versus Re-Build versus Re-Merge: Index Maintenance Strategies for Text Retrieval Systems. In Computer Science 2004, Twenty-Seventh Australasian Computer Science Conference (ACSC2004), Dunedin, New Zealand, January 2004, pages 15.22, 2004.

[25] L. Lim, M. Wang, S. Padmanabhan, J. S. Vitter, and R. Agarwal. Dynamic maintenance of web indexes using landmarks. In WWW '03: Proceedings of the 12th international conference on World Wide Web, pages 102.111, 2003.

[26] W.-Y. Shieh and C.-P. Chung. A statistics-based approach to incrementally update inverted les. Inf. Process. Manage., 41 (2): 275.288, 2005.

[27] K. Shoens, A. Tomasic, and H. Garcia-Molina. Synthetic workload performance analysis of incremental updates. In SIGIR '94: Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, pages 329.338, 1994.

[28] M. Theobald, R. Schenkel, and G. Weikum. An efficient and versatile query engine for TopX search. In VLDB '05: Proceedings of the 31st International Conference on Very Large Data Bases, pages 625.636, 2005.

[29] A. Tomasic, H. Garcia-Molina, and K. A. Shoens. Incremental updates of inverted lists for text document retrieval. In Proceedings of the 1994 ACM SIGMOD International Conference on Management of Data, Minneapolis, Minnesota, May 24-27, 1994, pages 289.300, 1994.

[30] K.-Y. Whang, M.-J. Lee, J.-G. Lee, M.-S. Kim, and W.-S. Han. Odysseus: A high-performance ORDBMS tightly-coupled with IR features. In ICDE '05: Proceedings of the 21st International Conference on Data Engineering (ICDE'05), pages 1104.1005, 2005.

[31] I. H. Witten, A. Moffat, and T. C. Bell. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann Publishers, San Francisco, CA, 1999.

[32] J. Zobel and A. Moffat. Inverted les for text search engines. ACM comput. Surv., 38 (2): 6, 2006.

[33] J. Zobel, A. Moffat, and K. Ramamohanarao. Inverted les versus signature les for text indexing. ACM Trans. Database Systems, 23 (4): 453.490, 1998.

1 to 9 are diagrams for explaining an information retrieval method having a document-based dynamic index management characteristic for integrating a database and information retrieval according to an embodiment of the present invention.

1 is a view showing the evaluation results of the information system according to the approach of the DB and IR,

2 is a diagram showing a pulsing auxiliary index with a pulsing threshold value DF = 3,

3 is a diagram showing sample data in a semi-standard format,

4 shows the insertion time averaged for every 10 of 10,000 input documents for a 10K set,

5 shows the insertion time averaged for every 10 of 10,000 input documents for a 100K set,

6 shows the insertion time averaged for every ten of 10,000 input documents for a 1M set,

FIG. 7 compares the average insertion time per document when adding 10,000 new documents for 10K, 100K, and 1M sets using a single subindex and a pulsing subindex.

8 is a view showing the average query processing time of terms having a variable DF range after adding 10,000 new documents to the 1M set,

9 is a diagram showing the average query processing time for complex queries after adding 10,000 new documents to 10K, 100K, and 1M sets.

10 and 11 is a block diagram of an information retrieval system according to a preferred embodiment of the present invention

[A brief description of the main parts of the drawing]

10: secondary index storage 20: primary index storage

30: index pulsing device 40: information search machine

Claims (8)

In an information retrieval system that manages documents dynamically, An auxiliary index storage unit for storing posting information of a document to be inserted, deleted or changed online in a transactional manner; An index pulsing device for moving the posting information that reaches a predetermined threshold size among the posting information stored in the secondary index storage device to the main index storage device; And And an information searcher for searching for desired information including documents inserted, changed or deleted online by using the posting information stored in the main index storage and the sub index storage. The method of claim 1, wherein the secondary index reservoir is: Information posted on the disk is stored in a secondary storage on the disk, the document is entered online, each document and all index information extracted from the document is processed in a transactional unit, and all the change history is recorded on the disk log information Search system. The device of claim 1, wherein the index pulsing device comprises: When the length of the posting information for a specific word of the secondary index storage reaches a given threshold size, the corresponding posting information is moved to the main index storage, and the posting information stored in the secondary index storage is deleted. system. The method of claim 1, wherein the IR is: And an information retrieval system that accurately searches for inserted, deleted, or changed documents online by using both the posting information of the main index storage and the posting information of the secondary index storage for a given query. In the information retrieval method, (a) storing, in a transactional manner, posting information of a document to be inserted, deleted, or changed online in a secondary index store; (b) moving the posting information, which has reached a predetermined threshold size, from the posting information stored in the secondary index store to the main index store; And (c) searching for desired information including documents inserted, changed, or deleted online by using the posting information stored in the main index storage and the sub index storage. The method of claim 5, wherein the secondary index reservoir is: Information posted on the disk is stored in a secondary storage on the disk, the document is entered online, each document and all index information extracted from the document is processed in a transactional unit, and all the change history is recorded on the disk log information Search method. The method of claim 5, wherein in step (b): When the length of the posting information for a specific word of the secondary index storage reaches a given threshold size, the corresponding posting information is moved to the main index storage, and the posting information stored in the secondary index storage is deleted. Way. The method of claim 5, wherein the information retrieval method is: An information retrieval method for accurately retrieving inserted, deleted or modified documents online by using both the posting information of the main index storage and the posting information of the secondary index storage for a given query.

Images