KR20190089420A - Data construction and management system of sub index storage method - Google Patents

Abstract

According to one embodiment of the present invention, provided is a data construction system of a sub-index storage method, which is an index structure securing a space in which multiple types of data is stored and indexing the space as a storage space of data corresponding to a data value. Specifically, provided is an index storage method which can quickly find specific data from multiple types of data to improve the performance of adding, deleting, and searching data, and allows the multiple types of data to efficiently maintain the alignment of data identifier lines. The technical solution is to store the multiple types of data in a mass storage object, and each sub-index which indexes an identifier to each data is matched and connected.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a data structure management system,

The present invention relates to a data construction and management system of a sub-index storage method. In order to efficiently store and retrieve data in consideration of limited memory resources and other system resources, a space for storing a plurality of data is secured, And indexes the data into the storage space of the corresponding data.

As the Internet develops and the number of users increases, various services are provided through the Internet, and users are using various types of services.

On the other hand, as the utilization rate of services provided by the Internet service provider increases, a huge amount of data is newly created, modified or deleted. As the amount of this information increases, the importance of a system that can efficiently provide information to the service provider and promptly provide information requested by the user is increasing.

Accordingly, the index structure, which is commonly used in the information retrieval system in recent years, is an index method that finds information that appears when a keyword is given. For this purpose, the index maintains data for each keyword.

At this time, since the data is composed of the identifier of certain data in which the keyword occurs and the location information in the data, the contents of the data vary depending on the frequency of occurrence of the keyword. In addition, even if the same keyword is used, the information of the generated position differs for each data, and therefore the contents of each data are not constant either.

Therefore, the index should have a storage structure that can be efficiently updated as data is added, deleted, or modified.

The simplest method for storing an index according to the above-mentioned necessity is a method of using a database table having a record structure of a keyword and data. However, this method is known to not only require a lot of storage space but also deteriorate performance because the same keywords are stored redundantly by the frequency of occurrence.

Therefore, a method of storing an index in an index structure without using a database table, that is, an index for a keyword, and a method for pointing data in a pointer field of a terminal node have been studied extensively.

In FIG. 1, reference numeral 10 denotes an index for a keyword, and reference numeral 11 denotes a storage space in which data is stored.

Therefore, since the inverse index storage structure shown in FIG. 1 is not the above-mentioned database table using method, it can be known through experiments that the storage space efficiency and the performance are better, A lot of research has been conducted on how to allocate the storage space of the number 11 when the size of the data continuously increases.

However, the proposed schemes do not consider data management and retrieval in an environment where data is dynamically updated due to the problem of finding specific data quickly and data addition, deletion, and modification.

As a result, the keyword search performance for specific data drops.

It is an object of the present invention to overcome the above-mentioned problems, and it is an object of the present invention to provide an apparatus and a method for searching for a specific data in a data list in order to improve data retrieval performance, Storage structure.

According to an aspect of the present invention, there is provided an index structure for reserving a space in which a plurality of data lists are stored and indexing the data list into a storage space of a corresponding data list according to a keyword, the index structure storing a data list in a large- Indexes the identifiers in the respective data lists.

In order to achieve the above object, the sub-index is indirectly pointed to by using an offset array to index a data identifier in connection with a data list.

According to another aspect of the present invention, the data list includes a physical layer for storing an object identifier, a data identifier, and generated location information.

According to another aspect of the present invention, there is provided a method of managing a plurality of data lists including a physical layer for storing an object identifier, a data identifier, and occurrence position information in a large-capacity object, A method of inserting new data on an inverse index storage structure using a sub index and a large capacity object in which a plurality of sub indexes are indexed and indexed to each data list, the method comprising the steps of: ; A second step of doubling the size of the current offset array if it is determined that the space for the offset arrangement is insufficient in the first step; A third step of allocating one element of the identifier of the new data to the empty offset array and determining whether the identifier of the new data is larger than the identifiers of the current data; A fourth step of adding new data to the end of the existing data list when it is determined that the identifier is larger than the current identifiers; and if it is determined through the third step that the identifier of the new data is not larger than the current identifiers The fifth step of inserting new data into the byte offset determined through the search of the sub-index And after performing the fourth or fifth step, correcting the value of the offset array elements recording the byte offset of the new data and all the data whose byte offset has been changed, adding the added identifier and the allocated offset array element And a seventh step of inserting the number into the sub-index.

According to another aspect of the present invention, there is provided a method of managing a data structure, the method comprising the steps of: storing a plurality of data lists including a physical layer for storing an object identifier, a data identifier and occurrence position information in a large object, Indexed storage structure using a sub-index and a large-capacity object in which a plurality of sub-indexes indexing a plurality of sub-indexes are connected to each data list, the method comprising the steps of: A second step of correcting a value of the offset array elements recording the byte offset of the data deleted in the first step and all data whose storage location is changed, , And a step of discarding the data deleted in the second step There to a third step of deleting kideon sub-index entries point to the array element. According to another aspect of the present invention, there is provided a method of managing a data structure, the method comprising the steps of: storing a plurality of data lists including a physical layer for storing an object identifier, a data identifier and occurrence position information in a large object, A method of retrieving data on an inverse index storage structure using a sub-index and a large-capacity object in which a plurality of sub-indexes indexing a plurality of sub-indexes are connected to each data list, the method comprising the steps of: A second step of determining whether or not a specific data identifier is present; and a step of, if a specific data identifier is present in the second step, Search for A third step of searching for a specific sub-index, searching for a specific sub-index, reading and returning the specific data retrieved, and, if the specific data identifier does not exist in the second sub-index, It involves having four steps.

According to another aspect of the present invention, there is provided a method of managing a data structure, the method comprising the steps of: storing a plurality of data lists including a physical layer for storing an object identifier, a data identifier and occurrence position information in a large object, Index storage structure using a sub-index and a large-capacity object in which a plurality of sub-indexes are index-indexed to each data list, wherein the sub-index is matched with an inverse index storage structure, the database system comprising: And the database object has a physical form matching the data in the data list using the object identifier and the data identifier in the concept of the address.

The following advantages can be obtained by providing an index storage structure using a sub-index and a large-capacity object according to the present invention operating as described above.

First, it can efficiently support dynamic update of index. By managing the data list of one keyword as one large object and managing it by using the large object management method, the data can be dynamically added, deleted and modified so that the storage space can be managed at a minimum cost even if the length of the data list varies . That is, even if data is inserted or deleted in the middle of a large-capacity object, other data before and after the inserted / deleted data need not be physically moved.

Second, it improves data deletion performance. Using the sub-index, the data containing the identifier of the deleted data can be quickly found and deleted from the data list, thereby enhancing the document deletion function. In particular, since keywords having a long data list are included in most data, data deletion in a long data list is an important factor for data deletion performance.

Third, it improves the data correction performance. Since the modification of the data is to delete the existing data and add the modified data, the correction performance is improved when the data deletion performance is improved. In addition, when inserting modified data into the data list, it should be inserted at the existing data identifier position in order to maintain the order of the data identifiers. In this case, when the data is inserted through the sub-index, the sorting of the data identifiers is automatically maintained, thereby improving the data modification performance.

Fourth, it improves the keyword search performance for specific data. By using the sub-index, specific data can be quickly found in a data list corresponding to a specific keyword, so that it is possible to enhance the search performance as to whether a specific keyword is included in specific data.

Fifth, it enables integrated processing of database search and information search. In a database management system combined with information retrieval, it is necessary to be able to integrate database retrieval with regular data and information retrieval with respect to data contents. In the new index storage structure, a database By maintaining the identifiers of objects, information retrieval and database retrieval can be integrated.

FIG. 1 is a conventional index-type storage structure.
FIG. 2 is a simplified schematic configuration example of a system in which an index storage structure according to the present invention is implemented; FIG.
FIG. 3 is an index-type storage structure according to the present invention; FIG.
FIG. 4 illustrates an example of using an offset method in a sub-index shown in FIG. 3; FIG.
FIG. 5 is a diagram illustrating a process of adding data using a storage structure of an index scheme according to the present invention; FIG.
6 is a diagram illustrating an example of a data deletion process using an index-type storage structure according to the present invention;
FIG. 7 illustrates an example of a data retrieval process using an index-type storage structure according to the present invention;
8 (a) through 8 (b) illustrate examples of data matching relationships between an index-type storage structure and a database system according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First, in order to briefly explain the technical points to be achieved in the present invention, consistent management of data is important in an environment where dynamic addition, deletion, and modification of data are frequently performed. For this purpose, a database management system function Is required.

In addition, the retrieval of the data is performed by retrieving the regular data of the data and the retrieval of the unstructured data. The database management system supports the retrieval of the above-described regular data, and the retrieval of the non- Information retrieval systems are well supported. Therefore, it is necessary to combine data retrieval with database management systems for consistent management and integrated retrieval of data.

Therefore, in the combined system, when the data is dynamically added to, deleted from, or modified in the database, the index structure must be efficiently updated and the data retrieval through the index and the database retrieval for the structured data must be integrated .

The index storage structure according to the present invention proposed by the necessity of the above-described index structure is applied to the hardware environment shown in FIG. 2, Storage structure is shown.

In the index storage structure according to the present invention shown in FIG. 3, a data list is stored in a large capacity object in a conventional index structure, and a sub index using a data identifier is provided for each data list.

In FIG. 3, reference numeral 20 denotes a B + tree index having a keyword as a key value. If a keyword is given, a large object (reference numeral 21) in which a data list of the keyword is stored is searched. A large-capacity object is an object whose size exceeds a disk page size such as full text, image, audio, and video data, and a data list for one keyword is divided into one It is regarded as a large object.

As described above, when a data list is stored using a large-capacity object, management of a data list having various variable length characteristics is automatically performed by a large-capacity object management technique.

Also, reference numeral 22 in the attached FIG. 3 is a sub-index devised in the present invention and constructed for each large-capacity object. The sub-index 22 is an index constructed with a data identifier as a key for a data list stored in a large-capacity object, so that the data list can automatically maintain sorting in the order of data identifiers, .

That is, when the data is modified, the modified data can be quickly inserted into the data list in the order of the data identifiers through the sub index, and the deleted data can be quickly deleted from the data list through the sub index when the data is deleted. When a keyword search for data is performed, data including an identifier of a specific data can be quickly retrieved from the data list through a sub index.

Therefore, if sub-indexes are used, data retrieval and deletion and keyword search performance for specific data are improved. These sub-indexes are not always constructed, but may be constructed only for a data list of a certain length or more in order to reduce overhead, and may be implemented by a B + tree.

Referring to FIG. 4, which shows a detailed storage structure of subindexes in the index structure proposed by the present invention, a subindex key is a data identifier, and a subindex indicates a subindex in a large object It is the location where the data is stored.

However, when the sub-index directly indicates the storage location of each data, when inserting new data or deleting existing data in the middle of a large-capacity object, the storage locations of all the data stored thereafter are changed. .

Accordingly, in the present invention, an offset array is maintained in the front part of the data list in order to solve this problem. Each element of the offset array indicates a storage position of the corresponding data. The storage location is the byte offset value from the end of the offset array. Therefore, even if the size of the offset array changes, the storage location of the data, i.e., the byte offset value, does not change. In addition, the sub index indicates an element of the offset array indicating the storage position of each data, so that the sub index is not affected even if the storage position of the data is changed.

An algorithm for inserting one new data when data is added or modified using the index structure of the structure shown in FIGS. 3 and 4 is shown in FIG. 5 attached hereto.

Referring to FIG. 5, it is determined in step S101 whether a space for offset arrangement for storing new data is insufficient. If it is determined in step S101 that the space for offset arrangement is insufficient, the process proceeds to step S102. In step S102, the size of the current offset array is doubled to secure space for offset arrangement.

Thereafter, if it is determined in step S101 that the space for offset arrangement is not insufficient, or if the size of the current offset array is doubled in step S102, the process proceeds to step S103. In step S103, One of the elements of.

During the process of allocating the new element to the vacant offset array in step S103, it is compared in step S104 whether the identifier of the new data is larger than the current identifiers. If the comparison value is larger than the existing identifiers, We add the data to the end of the list.

If it is determined in step S104 that the comparison value is not larger than the existing document identifiers, the process proceeds to step S105. In step S106, new data is inserted into the byte offset determined through searching of the sub-index. At this time, the byte offset of all the data stored after the inserted data becomes larger by the size of the inserted data. However, the physical location of the data itself is not changed by the management method of the large capacity object.

After the new data is inserted or added through the above-described steps S105 and S106, the value of the offset array elements recording the byte offset of all the data inserted and added with the byte offset in step S107 Modify it.

Thereafter, finally, in step S108, the identifier of the added data and the assigned offset array element number are inserted into the sub-index.

The process of inserting one new data through the above process has been described. The process of deleting one piece of data including the identifier of the deleted data when the specific data is deleted, that is, when the data is deleted, As shown in FIG. 6.

The sub index is searched in step S201 to find the storage location of the data including the identifier of the deleted data, and the corresponding data is deleted in step S202.

At this time, the byte offset of all the data stored after the deleted data becomes smaller by the size of the deleted data. Here, as in the case of insertion, the physical position of the data does not change.

Accordingly, in step S203, the value of the offset array elements recording the byte offset of all the data whose data and storage position are changed is modified. In step S204, the sub-index entry indicating the offset array element of the deleted data is deleted do. At this time, the offset array element indicating the deleted data can be reused when inserting new data.

The process of retrieving data using the index storage structure according to the present invention, which can add or delete data through the process as shown in the above-mentioned fifth and sixth figures, is shown in FIG. 7 .

First, a keyword index is searched using the keyword given in step S301 as a key value, and a large-capacity object storing a data list of a given keyword is searched.

Thereafter, in step S302, if the data identifier is divided into the case where the specific data identifier is given and the case where there is no data identifier, the process proceeds to step S303. In step S303, The sub index is searched to find the given specific data. In step S304, the searched specific data is read and returned.

On the other hand, if the specific data identifier is not given, the flow advances to step S305. In step S305, all the data in which a given keyword is generated should be returned, and the data list stored in the large object is sequentially read and returned.

Through the above process, we have examined the process of adding new data or deleting and searching specific data.

Now, a description will be given with reference to the contents of FIGS. 8 (a) and 8 (b) with a method of integrating information retrieval and database retrieval using a data identifier and an object identifier.

7 (a) shows a matching relationship between a data list and a database object on the database system, and the object identifier, the data identifier, and the generated location information are data List is a database object composed of data identifier, data number, author, creation date and contents.

At this time, the database object uses an object identifier and a data identifier in the concept of an address, and has a physical form matching the data of the data list.

The interconnection and usage relationship between the data list and the database object having the matching relationship as shown in FIG. 8 (a) is illustrated in the accompanying FIG. 8 (b).

The first data 400a and the second data 400b are stored together with the data identifier and the object identifier, and the data identifier 400b and the second identifier 400b Are physically aligned along with the object identifiers, and each object identifier indicates object 1 500b and object 2 500a.

The first object 500b and the second object 500a store data identifiers and their values are the same as the data identifier values of the first data 400a and the second data 400b. When the information retrieval query is processed to obtain the data, the corresponding database object can be read through the object identifier stored in the data to process the database query. On the contrary, if the database object is obtained by processing the database query, Is used as a key value to process the information search query by searching the sub-index, so that the information search and the database search can be integrated.

15: CPU
10: Memory
25: Application
30: Database Management System Area
35: Database query processing area
40: Information Retrieval System Area
45: Database object buffer
50: Data list buffer
55: Database

Claims (7)

An index structure for securing a space in which a plurality of data lists are stored and indexing the space into a storage space of a corresponding data list according to a keyword input, the index structure comprising: a plurality of data lists including a physical identifier for storing an object identifier, Index storage structure using a sub-index and a large-capacity object, wherein a plurality of sub-indexes indirectly indexing a data identifier by using an offset array is connected to each data list.
2. The reverse index storage structure according to claim 1, wherein the sub-index is selectively connectable to a data list having a predetermined length or longer when connected to each data list stored in the large-capacity object.
A plurality of data lists including an object identifier, a physical identifier for storing a data identifier and occurrence position information are stored in a large-capacity object, and a plurality of subindexes indirectly indexing a data identifier using an offset array are matched to respective data lists A method for inserting new data on an index storage structure using a linked sub-index and a large object, the method comprising: a first step of searching for insufficient space for an offset array of new data; A second step of doubling the size of the current offset array if it is determined that the space for the offset arrangement is insufficient in the first step; If it is determined that the space for the offset arrangement is not insufficient in the first step or after the second step is performed, one element of the new data identifier is allocated to the empty offset array, Determining whether the identifier is larger than the identifiers; A fourth step of adding new data to the end of the existing data list when it is determined through the third step that the new data identifier is larger than the current data identifiers; A fifth step of inserting new data into the byte offset determined through the search of the sub-index when it is determined through the third step that the new data identifier is not larger than the current data identifiers; And after performing the fourth step or the fifth step, after correcting the value of the offset array elements recording the byte offset of the new data and all data whose byte offset has been changed, adding the identifier of the added data and the allocated offset array element And a seventh step of inserting the number into the sub-index. The method of inserting new data in an index storage structure using a sub-index and a large-capacity object.
A plurality of data lists including an object identifier, a physical identifier for storing a data identifier and occurrence position information are stored in a large-capacity object, and a plurality of subindexes indirectly indexing a data identifier using an offset array are matched to respective data lists A method of deleting data on an index storage structure using a sub-index and a large-capacity object, the method comprising: a first step of searching for a sub-index to find a storage location of data including an identifier of the deleted data and deleting the data; ; A second step of correcting the values of the offset array elements recording the byte offset of the data deleted in the first step and all the data whose storage location is changed; And a third step of deleting the sub-index entry pointing to the offset array element of the data deleted in the second step. ≪ Desc / Clms Page number 19 >
A plurality of data lists including an object identifier, a physical identifier for storing a data identifier and occurrence position information are stored in a large-capacity object, and a plurality of sub-indexes indirectly indexing a data identifier by using an offset array, A method for inserting new data on an index storage structure using a matching sub-index and a large object, the method comprising: searching a keyword index based on an input keyword and searching for a large object storing a data list corresponding to the keyword; A first step; A second step of determining whether a specific data identifier exists; A third step of searching a sub-index to search for a given specific data and reading and returning the retrieved specific data if a specific data identifier is present in the second step; And a fourth step of sequentially reading and returning the data list stored in the large capacity object when the specific data identifier does not exist in the second step. Way.
A plurality of data lists including an object identifier, a physical identifier for storing a data identifier and occurrence position information are stored in a large-capacity object, and a plurality of sub-indexes indirectly indexing a data identifier by using an offset array, In a structure of a database system matched with an index storage structure using a matching sub-index and a large object, a plurality of database objects including a data identifier, a data number, an author, a creation date, and contents exist in the database system and,
Wherein the database object has a physical form matching the specific object identifier by using an object identifier and a data identifier in the concept of an address. rescue.
A plurality of data lists including an object identifier, a physical identifier for storing a data identifier and occurrence position information are stored in a large-capacity object, and a plurality of subindexes indirectly indexing a data identifier using an offset array are matched to respective data lists A sub index connected to the sub index; There is a plurality of database objects composed of a data identifier, a data number, a creator, a creation date, and contents, and the database object is a concept of an address, data having a physical form matching a specific object identifier A method for operating in a database management system having a base system, the method comprising the steps of: expanding the size of the current offset array twice when there is insufficient space for offsetting new data; The new data is added to the end of the existing data list if it is larger and the new data is inserted into the byte offset determined through the search of the sub-index in the other case, and then the value of the offset array elements recording the byte offset of all the changed data Edit New data insert mode for inserting an offset array element number allocated to the identifier of the added data, and then the sub-index and; The sub-index is searched to find the storage location of the data including the identifier of the deleted data to delete the corresponding data, and the value of the offset array elements recording the byte offset of the deleted data and all the changed data A data erasing mode for erasing a corresponding sub-index entry; And searches for a keyword index based on the input keyword, searches for a large-capacity object in which a data list corresponding to the keyword is stored, determines whether a specific data identifier exists, and if the keyword exists, And a data retrieval mode for sequentially reading and returning the data list stored in the large object when the specific data identifier is not present, and searching the sub index for retrieving the specific data, A method of combining database management system and information retrieval by index storage structure using sub - indexes and large objects.

Images