JPH0449132B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Application of the Invention] The present invention relates to a method for storing, retrieving, updating and deleting data in information processing by a computer.

[Background of the invention]

In information processing by electronic computers, records (=
Record: A collection of related information (data)) is saved in a file realized on the main memory or auxiliary storage, and records in the file can be searched and updated as necessary, and new records can be registered in the file. Common actions include editing records and deleting records from files. The methods of registering, searching, and deleting records at this time include searching, registering, deleting, and updating records based on the value of index information (key information) unique to the record; There are forms of file access such as reading records in the order of their value (key information), or scanning all records that have a specified value of index information (key information). Systems are required to realize these various forms of data access quickly and efficiently.

For records with index data (key),
Conventional techniques regarding file storage structures and access methods include the following methods and file configuration methods.

(1) Linear search method To register a record to a file, add it to any position in the file, and to search for a record, repeat matching in order for all records in the file until a record with a matching key is found. It is.

(2) Binary search method Records are arranged in the file in the order of their key values, and the search range is divided into two, narrowing down the target and searching for records with matching keys. You can also add records and
At the time of deletion, all subsequent records are refilled according to the key value of the record to be added or deleted.

(3) Hashing method The Hashing method is when the key of a certain record is given, the search location is determined by the address generated depending on the key (by the hashing function), and records containing the specified key are searched. This is a method to explore. The basic idea is to store records in a distributed manner within the file storage area with as few collisions as possible by appropriately selecting a hashing function that converts keys into addresses, and to reduce memory references during searches. In number of times,
It attempts to search for the desired record.

(4) Search method using tree structure This method constructs a tree structure with nodes consisting of (n-1) keys and n pointers to lower nodes, and searches the branches from the root node of this tree. It attempts to search for a record with a specified key by following the nodes in this direction. When n is 2, it becomes a binary search tree, and the B-tree search method is an application of this to balance the tree when reconfiguring the tree structure when adding or deleting.

In addition to the conventional techniques listed here, there are various methods for modifying these, and these are described in "Information Processing" published by the Information Processing Society of Japan, Vol. 24 No. 4 (April 1983), p. 391 - p. 400.
``The Art of Computer'' by Knuth.DE
Programming VOl.3 (Sorting and Searching)”
(Addison-Wesley, 1973) and others.

However, the conventional method described above has the following problems.

() In methods (1) and (3), when trying to read records in the order of key values, it is necessary to sort all the records in the file, and reading takes a significant amount of processing time.

() With method (2), on average, approximately half of the total records in the file are refilled each time a file is added or deleted, and the overhead of refilling reduces computer processing performance. This results in a decrease in

() In method (4), each node in the tree structure is connected by a pointer, so
part of this pointer is destroyed, or
In the event of a failure, there is a risk that the effect cannot be localized and the entire file will go down. Furthermore, since it is complicatedly configured with pointers, there is a high possibility of failure.

[Purpose of the invention]

An object of the present invention is to provide a fast and efficient file input/output method for data having an index (key).

[Summary of the invention]

The present invention is a method for managing input of data to a storage device, and the present invention is a method for managing data input to a storage device, which identifies whether or not records are registered for each unit area when a continuous storage area has a plurality of storage unit areas for recording and registering data. It is characterized in that it manages and maintains the order of data ordering key information, and inputs and stores data so that empty states are evenly scattered on the storage area.

Specific methods for average scattering include adding records sequentially to the center of the free range, or by using the ratio of the key information values at both ends of the free range to the key information values of the input data to be added. There are many possible methods.

[Embodiments of the invention]

The present invention solves the above-mentioned problems, reads data in index (key) order, searches, registers, deletes, and updates data at high speed, and prevents local storage area destruction and failure. This invention provides an indexed data management method and input/output device that realizes a file configuration method that does not cause failure of the entire file. First, the basic concept and configuration method will be explained with reference to FIGS. 2 and 3.

FIG. 2 is a diagram illustrating the concept of the basic data configuration method in the present invention. 1 is a record storage memory (storage area) for storing records, and can hold one record for each storage unit from 1 to N. Among the storage units, the shaded areas are storage units in which no records are registered (empty), and for each storage unit,
The numbers in the storage unit whose empty/real status is managed by record empty/actual management bitmap memory, etc., shown in 2, indicate the value of the key information of the record stored there, and are continuous. The records are arranged so that they are uniformly scattered on the storage area in the order of the values of their key information. In other words, the characteristics of the data configuration method in the present invention are: (1) records are scattered sparsely on the storage area where records are stored, with as uniform a density as possible; (2) records are distributed sparsely on the storage area where records are stored. , and the records are arranged in the order of the record key values.Additional registration of records to achieve this is performed by the method shown below.

(Record registration method) (1) From the key value of the new record to be registered,
Find an empty storage area where records are arranged in the order of key values. A method similar to the binary search method can be used for this purpose, and by narrowing down the range of storage areas to be searched, free storage areas in which new records can be registered can be found. It is possible to efficiently obtain the range and the key values of the records immediately before and after it.

(2) Determine the storage unit in which the new record should be registered from among the free storage areas found in (1). As a criterion for determining the storage unit, the registration status (empty, actual status) of records in the storage area is made to be as uniform as possible and evenly distributed and sparse. One method for this is to evaluate the key values of the records at both ends of the free storage area and the key values of the newly added record, and then adjust the range of the free storage area depending on the record to be added. divided into proportions. This method is to ensure that the proportion of key values that will be added in the future is as close as possible to the distribution of the divided free space, which will result in the amount of "record movement" described next. Reduce.

(3) In (1), if there is no free space in the storage area to be registered, the records that have already been registered in another free area are stored without changing the order of the key values.
By moving the records in order, a free storage area is created in which to register a new record, and then records are added according to methods (1) and (2). At this time, when moving records, try to keep the number of records to be moved as small as possible, and keep the record registration status (storage area empty/real status) as evenly distributed and sparse as possible. Determine the record group to be moved and the moving position so that

The steps of this record registration method are shown in FIG. In Figure 3, part B is a process for equalizing the record registration status within a limited range of storage area to which new records are to be added, and part A is a process for equalizing the record registration status within a certain limited range of storage area where new records are to be added. This is a process to equalize the empty and real states of storage areas.

In the method of the present invention described above, (a) retrieval of a record with a certain key is performed by two steps.
(b) To delete a record, simply search for the record using (a) and empty the storage unit in which the record is registered. (c) Records can be deleted using the same search method as the minute search method. The addition of the data can be done only by registering data in the free storage area and making the storage unit into a real state, or even if it is necessary to move the record.
Ratio of records registered in the file (rot ratio = (number of registered records ÷ number of storage units in the file))
(d) To retrieve records in order of key value, it is only necessary to physically read the file storage area in order. It has the following characteristics.

Next, the features of the present invention will be described in comparison with the prior art.

First, we will discuss the differences with the hash method as a comparison with the closest prior art at the implementation level.

<Differences from the Hashing method> The Hashing method maps the key value of a record registered in a file to an address on the file area using a specific function to register and refer to the record.The main points are as follows. The purpose is to distribute the mapping as evenly and uniformly over the file area as possible so that multiple records do not correspond to the same address (collision).
That is, in the hashing method, for a set of keys registered in a file at a certain point in time, the result of converting the value into an address within the file is made uniform for the area of addresses within the file. For this purpose, a randomization function is used as a key-to-address conversion function.

On the other hand, in the present invention, the records to be registered in a file are arranged in the storage area of the file in the order of their key values so that the empty and real states are evenly and sparsely arranged as much as possible. In other words, when registering a record, the address of the storage unit to which the record should be added is determined according to the order of the key values, and if there is no free space to add the record, the records in the file are moved sequentially. By rearranging them, they are arranged uniformly and sparsely.

Therefore, the fundamental difference in principle between the hashing method and the method of the present invention is that the hashing method distributes records uniformly over a file statically by using a randomization function that depends on the key values of the records. However, in the method of the present invention,
In the process of adding and deleting records, this is achieved dynamically by gradually moving records so that the records are sparsely distributed.

Next, the differences with the B-tree (or B _- ⁺ tree) method will be described as a comparison with the closest conventional technology at the level of realized functions.

<Differences from the B _- tree method> As already mentioned, the B _- tree method or its variant, the B ⁺ _-tree method, is an application of a search method that uses a tree structure. At the same time, the tree structure is reconfigured, and each node of the tree structure is connected and connected using pointers.
By changing connections and following pointers, records can be read in the order of key values. The difference between the B _- tree method and the method of the present invention is that the B _- tree method manages the order between records using pointers, whereas the method of the present invention physically records records in the order of key values. There is no need to have a pointer to line up the . As a result, in the present invention, the possibility of file structure destruction due to pointer abnormality is extremely low. In addition, in the B _- tree method, records in key order are connected using pointers, and each record is not necessarily physically arranged in the order of key values, so when reading records in key order,
If the file is on a magnetic disk, random access occurs on the disk and takes a long time to access, but with the present invention, the disk can be read sequentially, so accessing records in key order is extremely simple. The difference is that it is more efficient.

Next, the present invention will be described with reference to specific examples. FIG. 1 shows the configuration of an indexed data input/output device according to the present invention. In FIG. 1, numerals 1 and 2 are the aforementioned record storage memory and record/actual management bitmap memory, and the file directory 3 is composed of the record storage memory and the record/actual management bitmap memory. Unique information regarding the file 18 (file size, record size, file start address on memory, etc.) is stored as predefined information. According to the mode specified by the mode register 5, the control device 4, while referring to the information in the file directory 3, changes the status of the record storage memory 1, record blank/actual management bitmap memory 2, from real record to real record, as described above. The arrangement is controlled so that the key values are uniformly and sparsely distributed in the order of the key values. The mode specified by mode register 5 is the registration, registration, and
This is control information that instructs to perform either reference, deletion, or update, and the control device 4: (a) At the time of registration (addition): From the key value input from the key information storage buffer memory 6, The additional position of the record is determined, and the record input from the record storage buffer memory 7 is registered in the file.

(b) When referencing: Search the file for a record with a key equal to the key value input from the index data (key) storage buffer memory 6, read the record from the file, and transfer it to the record storage buffer memory 7. Output.

(c) When deleting: Search the file for a record with a key equal to the key value input from the index data (key) storage buffer memory 6, and delete the record from the file (record blank/real management bitmap memory The corresponding record is empty and the real management flag is turned off (“0”).

(d) When updating: Search the file for a record with a key equal to the key value input from the index data (key) storage buffer memory 6, and store the record in the record storage buffer memory 7.
Update with the contents of the record input from.

In the above, how to add a record (),
4 and 5 show details of the record search method and (). The meanings of each symbol and process name in FIGS. 4 and 5 are as follows.

<Symbols> RN...Register that stores the record number of the desired record (corresponding to the address on the record storage memory) SP...Search pointer BP...Start pointer AP...End pointer WRN, WRNX Work register for storing record number MP...Record Destination pointer IP: Record addition position pointer All of the above are internal registers in the control device 4 in FIG. Figure 4 A and B are steps 101
Consisting of ~123.

Processing step: (a)＜off-bit-search-fo-
rward from (X) → Y> starts from the bit corresponding to the record number specified by Search and store the record number corresponding to the first detected off-bit in Y. This is, for example, a description of steps 107 and 112 in FIG. 4A.

Processing step: (b)＜off-bit-search-ba-
ckward from (X) → Y> is the same as (a). However, this means searching for off-bits in the decreasing direction of the record number (step 105 in FIG. 4A).

Processing step: (c)＜on-pit-search-fo-
rward from (X) → Y> is the same as (a). However, the on-bit is searched in the direction of increasing record number (step 113 in FIG. 4).

Processing step: (d)＜on-bit-search-ba-
ckward from (X) → Y> is the same as (a). However, the on-bit is searched in the decreasing direction of the record number (step 109 in FIG. 4).

Processing step: (e)＜Move from (X) to (Y)
count(Z)> moves Z records from the record indicated by X to the record storage area for Z records from the record position indicated by Y (steps 117 and 119 in FIG. 4).

Processing step: (f) <data write to (X)> writes the contents of the record storage buffer memory to the Xth record storage area on the record storage memory (FIG. 4B).

Processing step: (g) <bit-set-to (X)> turns on (to "1") the Xth bit on the record empty/real management bitmap memory (FIG. 4B).

The above also applies to steps 151 to 166 in FIG.

A functional explanatory diagram of each process a, b, c, and d is shown in FIG. 6, and a functional explanatory diagram of process e is shown in FIG.
Each of these processes a to g can obviously be realized using existing memory operations or existing techniques, and is well known, so a detailed description thereof will be omitted. In addition,
If the bits of the pattern sought by a, b, c, and d do not exist in the memory, (maximum record number + 1) is returned to Y for forward, and 0 is returned to Y for backward.

The "record addition method" and "record search method" shown in FIGS. 4 and 5 will be explained below.

When adding a record, first search for a record that has the same value as the key of the record to be added (block 101 in Figure 4A, described later), and then find the record storage position (BP and BP) in the file to be added.
Find the empty area between the actual records at the record position indicated by AP. At this time, BP and AP
If there is no free space between BP or AP, the real record string is moved in the direction of shortening the distance to the free space adjacent to the continuous real record string (processing in blocks 104 to 120). At this time, the record storage position to which the actual record string is moved is determined according to a predetermined free area allocation rule R (block 115). In this way, it was requested
Within the free area where records can be added, the record storage position where the record should actually be added is determined according to the predetermined free area distribution rule R' (block 120), and the data of the additional record is placed in the area. is written, and the corresponding record empty/real management flag is turned on (“1”) (processing of blocks 120 to 122).

Next, in the record search process, BP and
First, select the search target record area surrounded by AP (BP = 0, AP = (maximum record number + 1)
(Figure 5, blocks 102 and 103), find the search target record number that divides the area according to the predetermined distribution rule R'', and find the key value of the real record in the vicinity. The search target range is narrowed down by comparing with the key value (updating BP and AP: processing of blocks 104 to 115). (If there is no corresponding record in the file with a matching key value), the range is completely empty (if there is no corresponding record in the file with the matching key value).If the corresponding record is detected in the process shown in Figure 5, the record number is In the RN, if a corresponding record cannot be detected, the area where the record should be stored is indicated by BP and AP.

Note that the distribution rules R, R′,
R″ is 2 that specifies at least both ends of a certain area.
This is a function that derives record storage position record numbers within an area that depends on the values of two pointers and divides the area surrounded by two pointers into certain distributions. This function can be arbitrarily defined depending on the characteristics and distribution method of the key values of the records to be stored in the file, but its purpose is to
The purpose is to ensure that the actual records are scattered evenly and sparsely, without being concentrated in one place.

Hereinafter, when an evaluation function is specifically applied to the above-mentioned distribution rule, its characteristics and the transition and behavior of the actual record storage state in the file of the present invention will be explained.

<Specific Example of Distribution Rule> As one of the embodiments of the present invention, consider a case where the distribution rule is defined as follows.

MP←R (WRNX, WRN): MP←
WRNX+WRN/2 IP←R′(BP, AP): IP←BP+AP/2 SP←R″(BP, AP): SP←BP+AP/2 In other words, record addition is done in the center of the free area, and the search The search range is always divided into two parts. Figure 8 shows how the arrangement of records in the file changes as records are added to the file. assumes that the following key values are entered in the order:

Order of keys added: [80, 45, 32, 23, 67, 95, 21, 1, 10, 55,
71, 46, 47, 17, 3, 60, 85, 5, 50, 35] Figure 9 shows the number of records moved at the time of record addition. As is clear from FIG. 9, as long as the number of records registered in the file is small, that is, the load ratio is low, new records can be added with almost no movement of records. As the load ratio increases, the number of records to be moved may increase in some cases, so it can be seen that it is more efficient to use the system while keeping the load ratio to a certain level. Furthermore, records can be searched using a method similar to the binary search method, and the desired record can be searched for efficiently. To delete a record, after searching for a record to be deleted, it is only necessary to turn off ("0") the record empty/actual management flag corresponding to the record number of the searched record.

<Specific Example of Allocation Rule> As a second example, consider a case where the allocation rule is defined as follows.

MP←R(WRNX, RNX): MP←WRNX+WRN/2 IP←R′(BP, AP): IP←[Key−Key(BP)/Key(AP)−Key(BP)(AP−)
BP)〕＋BP SP←R″(BP, AP): SP←[Key−Key(BP)/Key(AP)−Key(BP)・(AP
−BP)] + BP However, Key: Value of the key to be added or searched Key (AP): Value of the key of the record indicated by AP Key (BP): Value of the key of the record indicated by BP In other words, adding a record The record addition position at the time and the search start position at the time of record search are determined by the ratio of the key value of the records at both ends in the free area or search range area and the key value of the record to be added or searched. Select a record storage location that will be split. However, at both ends of the file, the minimum and maximum values that the key can take shall be given in advance. In this example, the key value range is 1 to 100, with the minimum value being 0 and the maximum value being 101. . FIG. 10 is a diagram showing changes in the arrangement of records in a file when the same key strings are added in the same order as in the example under the above conditions, and FIG. It is a diagram showing the correspondence with the number of movement records at that time. As can be seen from this figure, by using the specific example distribution rule for records with key characteristic distributions like this one, the overhead (record movement) at the time of addition can be further reduced. Even in searches, the number of key verifications is mostly 1 or 2 times,
You can search for the records you want. However, this does not mean that the distribution rule of the concrete example always has superior properties, but rather that the concrete example is better in the case of key distribution characteristics as in this example. It's just a thing. That is, according to the present invention, the amount of overhead (record movement) when adding a record is correlated with the load ratio, and increases rapidly when the load ratio exceeds a certain limit value (FIG. 12a). This correlation curve shows that by appropriately selecting the distribution rule according to the characteristics of the keys of the records stored in the file, the limit value of the load ratio can be raised (Figure 12b), and the overall average overhead can be increased. (first
Figure 2c) is also possible.

In the configuration diagram of FIG. 1, if the indexed data input/output device 19 is a processor that can operate independently of the processing device (processor) that issues the processing request 9, the record registration processing request Processing is performed by temporarily retaining the record data in the record storage buffer memory, returning a processing completion signal 10 to the request source, and then performing record movement processing within the file in conjunction with additional record registration. It is possible to improve responsiveness to requests. This is particularly useful in dedicated processor configurations, such as file processors that specialize in file processing and database machines that perform intensive database operations, especially in distributed processing formats.

According to the embodiments of the present invention, (1) Records can be added (registered), referenced, deleted, and updated at high speed using index data (key). Furthermore, by suppressing the load ratio, stable and high response performance can be guaranteed even when adding records.

(2) Records can be read out at high speed in the order of index data (key) values (sorted). In particular, not only is there no need for sorting, but records are arranged in key order in physically contiguous areas within the file, so even if the record storage memory is an auxiliary storage device such as a magnetic disk, It is possible to read data continuously at high speed.

(3) Unlike conventional key-ordered file construction methods, such as the B-tree search method, this key-ordered file does not have a "pointer" that stores the location address of other data as a constituent element. The structure of the file is difficult to destroy, and even if it is destroyed, it will only result in local data loss, and the risk of the entire file going down is extremely low.

〔Effect of the invention〕

According to the present invention, input to or output from a file can be performed at high speed.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an overview of the input/output device for indexed data, FIGS. 2 and 3 show the basic data structure of the present invention and its explanatory diagram, and FIGS. 4A, B,
Figure 5 is an explanatory diagram of adding and searching records, and Figure 6 is an illustration of adding and searching records.
7 and 7 are functional explanatory diagrams of the processing steps, and FIG.
The figure shows the state change diagram of the arrangement of records within a record, and Fig. 9 shows the number of records moved when adding a record.
FIGS. 10 and 11 correspond to FIGS. 8 and 9 and show the case where the distribution rules are changed, and FIG. 12 shows the relationship between the load ratio and the overhead at the time of addition. 1...Record storage memory, 2...Record empty/actual management bit map memory, 3...File directory, 4...Control device, 5...Processing request mode register, 6...Key information storage buffer.
Memory, 7...Record storage buffer memory,
8... File specific information, 9... Processing request signal (mode), 10... Processing completion signal, 11... Key information, 12... Record data, 13... Bitmap address, 14... Record empty Actual information, 15...Record address, 16...Key information, 17...Record data, 18...File, 19...Data input/output device with index.

Claims (1)

[Claims] 1. In a method for managing data input to a storage device, there are multiple storage area units that can be recorded on a continuous storage area, and whether or not records are maintained for each storage unit. A key that identifies and manages the records so that empty states are evenly scattered in the storage area, and the arrangement of the records is index information that identifies the records and is ordered by a predetermined evaluation. A data input management method characterized by inputting and arranging and storing each record in the order of information values. 2. In the data input management method set forth in claim 1, when a new record is additionally registered on the storage area, the value of the key information of the new record and the value of the already registered record are It is characterized by comparing the value of the key information, determining the range of free storage area in which a new record should be registered based on the order relationship of the values, and inputting the record within the range of the determined free storage area. How to manage data input. 3. As stated in claim 2, if there is no free space in the storage area in which the new record should be registered, the record group that has already been registered in another free area on the storage area is transferred to its key. A state in which a free storage area is created in which a new record is to be registered by sequentially moving the information values so that the order does not change, and empty and real storage areas are uniformly scattered within the new storage area. A data input management method characterized by determining a storage unit to which a new record should be added so that the new record is added, and registering the new record in the storage unit. 4. In claim 1, when adding a new record, the order relationship of the values of the key information is maintained, and the range of free storage area in which the new record is to be registered is maintained, and Based on the relationship between the value of the key information of the registered record and the value of the key information of the new record, the storage unit in which the new record should be registered is determined so that the free space is divided according to a certain distribution rule. A data input management method characterized by: 5. In the data input management method according to claims 1 to 4, 1-bit flag information for each storage unit is stored in a bit map that holds the flag information in an order corresponding to the arrangement of the storage units in the storage area. Then, check the flag status (on/off) of the bitmap memory to determine whether a record is registered in the memory unit.
When registering a record, turn on the corresponding flag in the bit map memory, turn off the corresponding flag when deleting a record, and scan the flag on the bit map memory when searching for a free or real memory unit. A data input management method characterized by determining a free storage area or an actual storage area. 6. A data input management method as set forth in claim 1, characterized in that a record for which a new registration request is made is registered in a record buffer area that temporarily stores and holds the record.