JP2871755B2 - Split control method in dynamic hash - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Overview] A storage structure using a dynamic hash method
In a data processing system that manages data, a split control method for dynamic hashing, in which the page splitting function is improved by performing page copying at the time of splitting, the overhead of the split processing is balanced in the entire system, and the response and the response in the transaction processing are improved. The purpose is to guarantee the throughput. At the time of splitting, a new page is secured and the contents of the split target page are copied to the new page. A process of rehashing and deleting unnecessary records, so that the two processes can be operated separately.

[Industrial Application Field] The present invention relates to a dynamic hashing method which improves a page splitting function by performing page copying at the time of splitting in a data processing system for managing data by using a storage structure using a dynamic hashing method. For the split control method.

As the storage structure in the database management system,
Research using dynamic hashes has been conducted.
The dynamic hash method is roughly classified into two types. One is a tree based on directory (Tr
ee), and the other is a directory-independent linear hash.

No commercial database has ever been put into practical use of dynamic hashing as a database. Dynamic hashes have several problems in adapting to database systems for large-scale online transaction processing (OLTP). One problem is that
In realizing dynamic hashing, the page splitting function has a significant effect on the response and throughput in transaction processing, and their allowable values cannot be sufficiently guaranteed.

[Conventional technology]

FIG. 5 shows an example of a conventional tree-like hash, FIG. 6 shows an example of a conventional linear hash, FIG. 7 shows an example of a conventional split process, and FIG. 8 shows a conceptual diagram of a conventional split control.

The requirements for a database system include a fast response and a good use of space for storing data. In particular, in online transaction processing, it is important that these are in a state of "stable and stable", that is, "the worst value is within a certain range", rather than "average is good". It is.

As a data storage structure that satisfies these requirements, a dynamic hash approach has been studied. The dynamic hash is described in detail in, for example, the following document.

(References) by RJEnbody, HCDo “Dynamic Hashing Schemes” ACM Computing Surveys, Vol. 20, No. 2, June, 1988 When storing data using hash functions, data is inevitably unevenly distributed and keys are duplicated. Things happen. With the conventional, generally used static hash, the space usage rate deteriorates when trying to prevent an increase in the number of accesses due to uneven distribution of data, and the access number cannot be guaranteed when trying to maintain the space usage rate .

In dynamic hashing, the space used increases or decreases according to the amount of data, so that a trade-off between the space usage rate and the number of accesses is eliminated to some extent.

Typical examples of the dynamic hash include a tree-like hash and a linear hash.

There are differences between a tree-like hash and a linear hash in the method of finding a record from a hash value. In a tree-like hash, an entry identifier in a directory is obtained from a hash value. On the other hand, in the linear hash, a page identifier (a page number, a page address, etc.) is obtained from a hash value. Here, a page is a physical or logical I / O access unit on the secondary storage. Sometimes called a block or bucket.

Both dynamic hashes improve the space efficiency of the storage structure by sharing pages with records of multiple hash values. The method of maintaining a page is to split the page when it is full, as in a conventional binary tree structure, if it is a tree-like hash. FIG. 5 shows an example thereof.

The hash function of this tree-like hash is h _k (key) = key // 2 ^k .

The value of the hash function corresponds to the directory entry number. k is the number of directory entry extensions, ///
Represents the remainder.

It is assumed that two records can be stored in one page. In the state shown in FIG. 5 (a), if there is a storage request for a record having a key of 005, a new page is added, and 002 and
Records with the key of 003 are sorted by re-hashing. Then, as shown in FIG. 5 (b), for a record having a key of 005, a corresponding page is determined by hash and stored therein.

In the linear hash, a page is caused to overflow, and split processing is performed at another arbitrary timing. Sixth
The figure shows an example.

 In the example of FIG. 6, the following hash is performed.

(1) Let H be a function to convert from k to m random bit strings.

(2) According to the number of expansions d, the number of pages in the physical space is
It is assumed that 2 ^d (d = 0... M).

(3) Let h _i (k) = H (k) mod 2 ⁱ be the mapping function for the physical space.

(4) Immediately after expansion or the initial value is a split pointer
SP is the first page of the physical space. In this state, _hd is used for mapping the record to the physical space.

(5) The split pointer SP is moved according to the fixed number or the number L of records, and records that have been moved are redistributed using the mapping function _{hd + 1} to the next physical space. Therefore, in this state, the mapping function h (k) of the record to the physical space is as follows.

When h _d (k) ≧ SP, h (k) = h _d (k) When h _d (k) <SP, h (k) = h _{d + 1} (k) Here, for example, the initial value d is 2, and L is stored in three cases. Two items are stored in the page.

 A specific example of the result of H (k) is as follows.

160,177,194,061,078,095,006,023,
028.

In this case, the linear hash is stored as shown in FIG.

FIG. 6 (a) shows a state where up to the record is stored. Here, when there is an addition of the record of 〜, the storage page is determined by the following function, respectively.

a _{h d (k) = 006mod 2} 2 = 2 h d (k) = 023mod 2 2 = 3 h d (k) = 028mod 2 of ² = 0 of record storage location to the second page, already 2 reviews Since the record is stored, the overflow page OP is secured and stored here. The result of storing the records up to is as shown in FIG.

When the storage of the three items is completed, the split control is started. Since there is an overflow page in the page pointed to by the split pointer SP, a new page is secured.
Re-hash records, and with the new function.

From _{'h d + 1 (k)} = 194mod 2 3 = 2' h d + 1 (k) = 078mod 2 3 = 6 'h d + 1 (k) = 006mod 2 3 = 6 result, and the record Then, as shown in FIG. 6 (c), the page is moved to a new sixth page.

The split pointer SP advances by one and points to the split target at the next split trigger.

Thus, in the linear hash, the page overflows and the split control is performed later.
Called delayed split. Linear split with delay split
In the case of the hash, the overhead of the split processing is increased by the overflow of the page as compared with the tree-like hash. This overhead is an increase in the number of program running steps in the split processing and waiting for resource lock.

In delayed splitting, it is conceivable that the splitting process is controlled by a daemon function that operates in the background. However, if split control is easily performed by the daemon function, there is a danger that the response and the overall throughput will be reduced due to waiting for record (resource) exclusion of the storage structure access function that operates as foreground processing.

In the case of the linear hash described above, the usage rate of the page due to the split changes drastically.
As described in the above-mentioned references, a partially expanded hash having improved space utilization efficiency has been proposed. However, since the overhead due to the split has not been studied, the overhead is essentially the same as that of the linear hash.

The outline of the conventional split processing as described above is described in Section 7.
It is as shown in the figure.

(A) When a split is required, a new page is first secured.

(B) Read the record of the page to be split.

(C) When the record ends, the processing ends.

(D) Re-hash using a new hash function.

(E) As a result of the hash, it is determined whether the storage location is the new page side or the current page side. In the case of the current page, the record is left as it is, the process returns to the process (b), and the process proceeds to the next record.

(F) If the storage location is on the new page, the record is moved to the new page. That is, the record is copied to a new page, and the original record on the current page is deleted. Thereafter, the process returns to the process (b) and the process is repeated.

If there is an overflow page, the processes (b) to (f) are similarly repeated for the overflow page.

In such a conventional split, for example, when the page P2 shown in FIG. 8A is split, all of the records R1 to R4 stored in the record P2 are
Each of them is re-hashed and, as shown in FIG. 8 (b), the movement (including deletion) of the record is executed by a series of processing.

[Problems to be solved by the invention]

In conventional split control in dynamic hashing, when splitting becomes necessary, all records on the page to be split, including overflow pages, are re-hashed, and records that need to be changed in storage location are copied and copied. Since the deletion is performed as a series of processing, there is a risk that the processing of the online transaction may be delayed for a long time due to the exclusive control during this time. In addition, there is a problem that the load on the CPU is concentrated during the execution of the split control.

A necessary condition for the storage structure of the database for online transaction processing is to satisfy the following three points while minimizing the influence of the self-adjustment function of the storage structure change.

(1) The transaction of data processing (foreground) and the storage structure change processing (background) are separated to guarantee the response of transaction processing.

(2) Exclusion waiting between transaction processing of data processing and storage structure change processing is prevented from becoming long.

(3) Overall, the transaction processing cost of data processing and the cost of storage structure change processing are balanced.

The above condition (1) can be realized to some extent by the conventional delay split method. Furthermore, (2),
It is necessary to satisfy the condition (3).

SUMMARY OF THE INVENTION It is an object of the present invention to solve the above problems, to balance the overhead of split processing in the entire system, and to guarantee the response and throughput in transaction processing.

[Means for solving the problem]

 FIG. 1 is a diagram illustrating the principle of the present invention.

In FIG. 1, reference numeral 10 denotes a processing unit comprising a CPU and a memory, etc., 11 denotes a storage structure access processing unit for performing insertion, retrieval, update, and deletion of records, 12 denotes a storage structure change processing unit for performing split control, and 13 denotes Page copy processing, 14 denotes unnecessary record deletion processing, 15 denotes a buffer control unit for reading and writing pages, 16 denotes a secondary storage device such as a magnetic disk device, and 17 denotes a data storage unit having a hash storage structure.

In the page copy process 13, when a split in the dynamic hash becomes necessary, a process of securing a new page and copying the content of the split target page to the new page as it is is performed.

In the unnecessary record deletion process 14, the original page and the new page after the copy process by the page copy process 13 are re-hashed, and one unnecessary record of the duplicate record in the new page and the original page is deleted. .

In the present invention, in particular, the processing of the page copy processing 13 and the unnecessary record deletion processing 14 in the storage structure change processing unit 12 are separated, and if there is an urgent online transaction during this processing, for example, To be executed with priority.

For example, in the state shown in FIG.
Suppose that records R1 to R4 are to be rearranged with the target being a split target.

In the process of the page copy process 13, the content of the page P2 is copied to a new page P6 without any re-hashing of each record, as shown in FIG. If there is an overflow page, the overflow page area is similarly secured and copied as it is.
Therefore, the time from the start to the end of this processing is extremely short.

In the unnecessary record deletion processing 14, at any other time, the records R1 to R4 duplicately stored in the pages P2 and P6 are re-hashed, and whether or not the currently stored location is correct is determined. If it is not correct, it is determined that the record is unnecessary, and the record is deleted as shown in FIG.

[Action]

In the present invention, in a dynamic hash such as a tree-like hash, a linear hash, and a partially extended hash,
The following control is performed.

-When splitting, secure a new page and copy the split target page.

-When retrieving a split page, records that are unnecessary due to duplication are not retrieved because the new hash function is used. However, in the case of sequential search not based on the hash value, re-hash is performed to confirm that the record is a search target.

Unnecessary records are records that are not originally moved due to splitting. Because the page is copied,
Unnecessary pages are copied to new pages.

-In the self-adjustment function, this unnecessary record is detected and deleted by re-hashing.

As described above, at the time of splitting, the process is completed simply by copying a page, and thus the exclusive control period for the process is extremely short. Deletion of unnecessary records by the self-adjustment function eliminates the need to delete all unnecessary records at once, and can be performed sequentially at appropriate times depending on the CPU load, etc., so that the influence on online transactions, etc. is minimized. Can be smaller.

〔Example〕

FIG. 2 shows an example of application of the present invention to a tree-like hash, FIG. 3 shows an example of application of the present invention to a linear hash, and FIG. 4 shows a processing configuration example of a linear hash structure according to an embodiment of the present invention. .

[Example of application to tree-like hash] In the tree-like hash split processing, when a record is stored, if a page is full, a new page is secured and all the contents of the old page are copied to the new page. After that, unnecessary records are deleted from the new and old pages. If the unnecessary record is rehashed, it can be determined whether or not the record should exist on the new page or the old page.

Since the directory entry and the page are obtained from the hash value, unnecessary records due to copying are excluded from the key search. Even if the unnecessary records exist, there is no problem in the key search.

By the way, in an actual database, a forward search may be performed without depending on a hash value. Therefore, at the time of splitting, for example, the display during splitting is set up in page units. Later, when the self-adjustment function is operated to delete unnecessary records and all unnecessary records in the page are deleted records, the display during splitting is turned off.

In the sequential search, the records in the page with the split display are re-hashed and only those that are not unnecessary records are searched.

FIG. 2 shows an application example of the present invention to the same tree-like hash case as in FIG. 5 described as a conventional example.

The hash function of this tree-like hash is h _k (key) = key // 2 ^k .

The value of the hash function corresponds to the directory entry number. k is the number of directory entry extensions, ///
Represents the remainder.

When 002 and 003 are inserted as initial data, the state shown in FIG. The split display F is "0".

Next, when an attempt is made to insert a record having a value of 005, the page is full, so a page copy split is performed. As a result, a state as shown in FIG. (003) and (002) are unnecessary records, but there is no direct display of unnecessary records on the page.
The split display F is set to "1".

When hashing the record of 005 to be inserted,
It can be seen that it should be inserted into the page indicated by the directory entry “1”. Therefore, an unnecessary record of the page is obtained by re-hashing, and a record of 005 is inserted into the area as shown in FIG. Note that the unnecessary records need only be found as long as 005 records can be inserted. At this stage, it is not necessary to search all unnecessary records in the page.

When all unnecessary records are deleted at another time, the display F during split is returned to "0".

[Application Example to Linear Hash] In the linear hash, when a page becomes full, a record is stored in an overflow page and the split is delayed. For example, when a certain amount of records is inserted, the self-adjustment function advances the content of the split pointer SP by one and starts the split processing.

In the split processing, a new page is secured and the contents of the old page are copied. Unnecessary records due to copying are excluded from key search because the page is obtained from the hash value.

In an actual database, a forward search may be performed regardless of the hash value. For this reason, the split display F is set up, and if the self-adjustment function is operated later and all unnecessary records in the page are deleted, the split display F is erased. In the sequential search, records in the page where the split display F is set are rehashed, and only non-unnecessary records are searched.

FIG. 3 shows an application example of the present invention in the case of the same linear hash as in FIG. 6 described as a conventional example.

As shown in FIG. 3, in the reorganization when the split pointer SP is currently on the second page and is moved to the third page, the prime page PP and the overflow page OP of the second page are set as shown in FIG. , Copy as it is. Then, a split display (not shown) is set.
Thereafter, the split pointer SP is advanced so as to point to the third page. This process can be performed at high speed because rehash is not required.

[Application Example to Partially Extended Hash] In the case of partially extended hash, the number of copied pages is determined by the number of groups. When the number of groups is 2, each group and a new page have a configuration of two pages (prime page and overflow page) by each page copy operation. When the number of groups is n, each group and a new page have a configuration of n pages. The following is similar to the linear hash.

FIG. 4 shows a processing configuration of one embodiment of the linear hash structure of the present invention.

The split control daemon processing unit 20 shown in FIG. 4 corresponds to the storage structure change processing unit 12 shown in FIG. 21 is a buffer, 22 is a storage location in the secondary storage device 16 and buffer 2
This shows an address conversion table indicating the relationship with 1 and the like.

The storage structure access processing unit 11 has processing functions such as record insertion, key search and record search by sequential search, record update, and record deletion.

The split control daemon processing unit 20 has processing functions such as prime page copy, overflow page copy, unnecessary record deletion, and overflow page integration.

The buffer control unit 15 has processing functions such as page reading, page writing, new page acquisition, and updating of the address conversion table 22.

The storage structure access processing unit 11 requests the buffer control unit 15 to read a page when accessing a record. The read page is transmitted via the address conversion table 22,
Associated with the secondary storage device 16. Buffer control unit 15
When the insertion page becomes full due to a record insertion request or the like, the overflow page is cut out and the record is stored there.

The split control daemon processing unit 20 operates asynchronously with the storage structure access processing unit 11. If splitting is required, a page to be split is obtained from the split pointer SP. In the figure, it is the P page. This P page currently contains one overflow page (O page).
One exists.

In this split control, the system performance is stabilized by dividing the split process into several processing units as a self-adjustment function of the database for online transaction processing.

The flow of the processing unit of the split control in the present embodiment will be described below. Each phase is processed by the split control daemon processing unit 20, and several simultaneous runs are possible. As for the technique for running the programs simultaneously, various methods are known as task control and process control, and a detailed description thereof will be omitted.

Register a new page entry in address translation table 22, and
Make two duplicate pages.

If an overflow page exists, a new page entry is registered, and one copy page is created.

The process is repeated until copying of the overflow page is completed.

One record in the old page (copy source) and the new page (copy destination) are rehashed, and unnecessary records are deleted.

Performs overflow page integration processing. In this integration process, when the usage rate of pages after deleting unnecessary records falls below a certain standard, for example, 40%,
If there are more than one, they are put together and the overflow page is returned. Omitting this integration, after all overflow pages become empty,
You may return it.

According to the buffer control logic, the above processing result is
This is reflected in the secondary storage device 16.

The above processing units 1 to 3 may be separated or may be put together. For example, can be a split control phase and can be a self-adjustment phase.

In the example of FIG. 4, specifically, the following processing is performed.・ P
The page is copied as page P '. In this new page acquisition process, the buffer control unit 15 acquires a new page on the primary storage.

-Register the P 'entry in the address conversion table 22.

・ Set the split display on the P, P 'page.

Copy the overflow page O to the page O ', and set the display during splitting.

Re-hash records in one P of the split page and delete unnecessary records.

• After all records are processed, the display during splitting disappears.

• The same process as for P is performed for page P '.

• The same processing as P is performed for the O page and the O 'page, but if all pages become empty, the overflow page is returned.

During the above processing, when the sequential search function is used in the storage structure access processing unit 11, if the display during splitting is standing on a page, the records in the page are re-hashed and are unnecessary records. Make sure.

Although the linear hash has been described, a tree-like hash, a partially expanded hash, and the like can be handled by a similar processing configuration.

In the embodiment described above, the overflow page has a link structure, but a linear hash may be used as the overflow structure, as is known as a recursive linear hash. Also, a binary tree structure can be used as the overflow structure.

The split control daemon processing unit 20 shown in FIG. 4 operates asynchronously with the storage structure access processing unit 11, but can also operate in synchronization with the storage structure access processing unit 11. However, in this case, it is not for full-scale online transaction processing.

In the record search of the storage structure access processing unit 11, there is a case where the sequential search function is not provided.

〔The invention's effect〕

As described above, according to the present invention, the possibility that transaction processing operating in the foreground waits for exclusion for a long time can be reduced by the self-adjustment function operating in the background. Further, the costs of the transaction processing and the self-adjustment function for changing the storage structure can be balanced in the entire system.

[Brief description of the drawings]

1 is a diagram illustrating the principle of the present invention, FIG. 2 is an example of application of the present invention to a tree-like hash, FIG. 3 is an example of application of the present invention to a linear hash, and FIG. 4 is an embodiment of the present invention. 5 is an example of a conventional tree-like hash, FIG. 6 is an example of a conventional linear hash, FIG. 7 is an example of a conventional split process, and FIG. 2 is a conceptual diagram of the split control of FIG. In the figure, 10 is a processing device, 11 is a storage structure access processing unit, 12 is a storage structure change processing unit, 13 is a page copy process, 14 is an unnecessary record deletion process, 15 is a buffer control unit, 16 is a secondary storage device, 1
7 represents a data storage unit.

Claims (1)

(57) [Claims] In a data processing system for managing data by a storage structure using a dynamic hash method, a process for securing a new page at the time of splitting and copying the contents of the page to be split to the new page ( 13)
And a processing step (14) for re-hashing the split target page and the new page after the copying processing and deleting unnecessary records, wherein the two processing steps can be operated separately. A split control method for dynamic hashing.