JP4813155B2 - Data management apparatus, data management method for data management apparatus, and data management program - Google Patents

Description

The present invention relates to an area range management method in which rows (records) added to a database are separated at an arbitrary point of time, and a data management apparatus in a database or the like that realizes high-speed data access and loading and efficient data management The present invention relates to a data management method and a data management program for a data management apparatus .

  As the amount of data stored and stored in the database increases, the access efficiency to the database decreases and more time is required for searching and referencing. As a countermeasure, an index is generally used for the key used for the search. However, if the data is frequently added, deleted, or updated, or if the amount of data retrieved by the search increases, the access efficiency will be increased again. There was a problem that would decrease.

As a data management method for solving this problem, a method of managing a series of segment groups by creating a bookmark information with time information by dividing the data storage area of the database into segments called segments without providing an index. (Patent Document 1).
JP-A-10-260876

  However, in the data management method of Patent Document 1, data is stored for a certain period of time in a segment, and once stored, the data cannot be changed. For this reason, if the data that arrives within a certain period of time becomes sparse, small data is stored in segments one after another. However, since data cannot be reorganized, the access efficiency for each small data is reduced. In addition, the utilization efficiency of the data storage area also decreases. Furthermore, when searching with a key other than time, it is necessary to search all segments, so high-speed data access is not possible.

  The present invention has been made to eliminate the problems of the prior art such as the above-described problems, and the object of the present invention is to enable optimal data organization even if the data to be reached is sparse, for example. That is.

  A data management apparatus according to the present invention uses a central processing unit as a group of a first storage device having a storage area, a buffer memory having a predetermined size, and a plurality of arbitrary data matching the size of the buffer memory. A data processing unit for processing data, an input / output control unit for inputting / outputting arbitrary data to / from the first storage device in groups, and a plurality of groups stored in a continuous storage area of the first storage device A range entry including group specifying information for specifying a group belonging to the range, arbitrary data specifying information for specifying arbitrary data included in the group belonging to the range, and range identification information for identifying the range. A second storage device stored in the second storage device, and a range in which the management processing of the range entry stored in the second storage device is performed using a central processing unit Characterized in that a management control unit.

  According to the present invention, there is provided a data management device that enables efficient data management by operation in units of ranges such as high-speed data access, loading, deletion, and backup by managing a data storage area in a continuous range. be able to. With this data management device, for example, even if the data that arrives is sparse, optimal data organization can be achieved.

Embodiment 1 FIG.
FIG. 1 is a diagram illustrating an appearance of a data management system 900 according to the first embodiment.
1, a data management system 900 includes a system unit 910, a display device 901, a keyboard (K / B) 902, a mouse 903, a compact disk device (CDD) 905, a printer device 906, and a scanner device 907, which are cables. Connected with.
Further, the data management system 900 is connected to the FAX machine 932 and the telephone 931 via a cable, and is connected to the Internet 940 via a local area network (LAN) 942 and a web server 941.
The display device 901, K / B 902 and mouse 903 constitute the terminal device 2. The system unit 910 is a data management apparatus 1 that performs data management processing.

FIG. 2 is a hardware configuration diagram of the data management system 900 according to the first embodiment.
In FIG. 2, the data management system 900 includes a CPU (Central Processing Unit) 911 that executes a program. The CPU 911 includes a ROM 913, a RAM 914, a communication board 915, a display device 901, a K / B 902, a mouse 903, an FDD (Flexible Disk Drive) 904, a magnetic disk device 920, a CDD 905, a printer device 906, and a scanner device 907 via a bus 912. It is connected.
The display device 901 includes a liquid crystal display, a CRT (Cathode Ray Tube) display, and the like.
The RAM 914 is an example of a volatile memory. The ROM 913, the FDD 904, the CDD 905, the magnetic disk device 920, and the optical disk device are examples of nonvolatile memories. These are examples of a storage device, a storage device, or a storage unit.
The magnetic disk device 920 is an example of a storage device that stores each record data in a database. The RAM 914 is faster in data access than the magnetic disk device 920, stores data input / output between the CPU 911 and the magnetic disk device 920, and increases the access efficiency of the CPU 911 for the data stored in the magnetic disk device 920. It is an example of the buffer memory which improves.
The communication board 915 is connected to a FAX machine 932, a telephone 931, a LAN 942, and the like.
For example, the communication board 915, the K / B 902, the scanner device 907, the FDD 904, and the like are examples of an input device, an input device, or an input unit.
Further, for example, the communication board 915, the display device 901, and the like are examples of an output device, an output device, or an output unit.
The terminal device 2 is a terminal used by the user, and causes the user to input a processing request from the input device, and outputs (displays, prints, sounds, etc.) the processing result input from the data management device 1 to the output device.

Here, the communication board 915 is not limited to the LAN 942 but may be directly connected to the Internet 940 or a WAN (Wide Area Network) such as ISDN. When directly connected to a WAN such as the Internet 940 or ISDN, the data management system 900 is connected to a WAN such as the Internet 940 or ISDN, and the web server 941 is unnecessary.
The magnetic disk device 920 stores an operating system (OS) 921, a window system 922, a program group 923, and a file group 924. The program group 923 is executed by the CPU 911, the OS 921, and the window system 922.

The program group 923 stores a program for executing a function described as “˜unit” in the description of the embodiment. The program is read and executed by the CPU 911.
In the file group 924, in the description of the embodiment, the data described as “˜information”, “˜request”, “˜command”, and the determination result and calculation result at the time of executing the function described as “˜part” are stored. The data shown, the data passed between programs that execute the function described as “˜part”, and the like are stored as “˜file”. Further, data stored by the data management apparatus 1 (for example, record data), management information used for data management processing (for example, a range management table 5 described later) and a backup file (for example, an archive file 300 described later) are files. Group 924 stores “˜file”.
Also, in the description of the embodiments, the arrows in the flowcharts and configuration diagrams mainly indicate input / output of data, and the data for the input / output of the data includes a magnetic disk device 920, an FD (Flexible Disk cartridge), and an optical disk. , CD (compact disc), MD (mini disc), DVD (Digital Versatile Disk), and other storage media. Alternatively, it is transmitted through a signal line or other transmission medium.

  Also, what is described as “˜unit” in the description of the embodiment may be realized by firmware stored in the ROM 913. Alternatively, it may be implemented by software alone, hardware alone, a combination of software and hardware, or a combination of firmware.

  The program for implementing the embodiment may be stored using a storage device such as a magnetic disk device 920, FD, optical disk, CD, MD, DVD, or other storage medium.

FIG. 3 is a configuration diagram of the data management system 900 according to the first embodiment.
In FIG. 3, the data management system 900 includes a data management device 1 and a terminal device 2. The data management device 1 uses the CPU 911 to process processing requests from the terminal device 2 such as data loading, retrieval, backup / restore, deletion, and addition / deletion of ranges described below. Hereinafter, the minimum unit of data loaded from the terminal device 2 to the data management device 1 is referred to as “row”. This “row (an example of arbitrary data)” corresponds to a record in the database, for example. Hereinafter, the data management apparatus 1 will be described as a database. The data management system 900 can load a plurality of rows at a time from the terminal device 2 to the database of the data management device 1 and can extract necessary rows from the data management device 1 by searching.

The data management device 1 is a data management device according to the first embodiment, and includes a data processing unit 3, a range management control unit 4, a data access control unit 6 (an example of an input / output control unit), and a data storage area unit 7 (first 1) and a buffer memory 951. In addition, a storage device (second storage device) that stores the range management table 5 and a storage device (third storage device) that stores the archive file 300 (an example of a backup file) are provided. The range management table 5 and the archive file 300 may be stored in the same storage device as the data storage area unit 7, or may be stored in another storage device. The data storage area unit 7 has an update area 8 for updating stored data and a reference area 9 for not updating stored data. The data storage area unit 7 allocates the update area 8 to the continuous area of the magnetic disk device 920 and allocates the reference area 9 to the continuous area of the magnetic disk device 920 separately from the update area 8. Further, the data storage area unit 7 may allocate the update area 8 and the reference area 9 to continuous areas of the magnetic disk device 920. As a result, a desired row can be accessed by accessing the magnetic disk device 920 at successive locations on the surface of the magnetic disk, and data access can be made without wasting the seek of the disk head. The data storage area unit 7 can be realized even with other storage media such as a semiconductor memory.
Here, the range management table 5 is a table having management information for the range. A plurality of rows conforming to the size of the buffer memory 951 of the data management device 1 (hereinafter referred to as buffer size) are defined as one group, and a plurality of groups stored in a continuous area in the update area 8 or the reference area 9 are included in one range. It is. Hereinafter, management information for one range (an example of a range entry) is referred to as an “entry”.

Each component of the data management apparatus 1 performs the following processing using the CPU 911.
The data processing unit 3 inputs a processing request from the terminal device 2 and outputs a data management command to the range management control unit 4 in response to the processing request. At this time, the data processing unit 3 processes the row stored in the buffer memory 951 and outputs the processing result to the terminal device 2.
The range management control unit 4 inputs a data management command from the data processing unit 3 and performs data management processing such as entry change / addition / deletion / initial setting for the range management table 5 in response to the data management command. A data access command is output to the data access control unit 6.
The data access control unit 6 inputs a data access command from the range management control unit 4 and performs data access processing such as acquisition / addition / deletion of a group according to the data access command on the data storage area unit 7. At this time, the data access control unit 6 stores the row acquired from the data storage region unit 7 in the buffer memory 951 or additionally stores the row stored in the buffer memory 951 in the data storage region unit 7.

FIG. 4 is a conceptual diagram of “range” realized by the data management apparatus 1.
In FIG. 4, the database 30 has three ranges 30a-30c. A range ID (an example of range identification information) is assigned to each range as a number for identifying the range. One range can be defined when the database 30 is generated, and ranges can be added in order by range addition processing.

FIG. 5 is a conceptual diagram when a range is added to the database 30 shown in FIG.
The row to be added to the database 30 is loaded into the “current range” which is the latest range added last. The range 30c in FIG. 4 and the range 30d in FIG. 5 are the current range. The data management apparatus 1 can backup / restore and delete data in units of this range.

FIG. 6 is an example showing the range management information held in the range management table 5.
The range management table 5 has a database generation date 10, a table last write time 11 to the database, a range number 12 held by the database, and a range key number 13. Also, in each entry, as attributes of individual ranges, range ID 14, start group number 15, group number 16, finalized state 17, loaded state 18, finalized time 19, entry last write time 20, minimum row number 21 (minimum number) An example), a maximum line number 22 (an example of a maximum number), a minimum time stamp 23 (an example of minimum addition time), and a maximum time stamp 24 (an example of maximum addition time). In FIG. 6, the line number (minimum line number 21, maximum line number 22) and time stamp (minimum time stamp 23, maximum time stamp 24) are the keys of the range. It is possible to set freely.
The range management control unit 4 sets each range management information of the range management table 5. In particular, management processing such as search, change, addition, deletion, and initial setting is performed for each entry.

Next, the attribute of each range that each entry has will be described.
The range ID 14 (an example of range identification information) is unique identification information set for the range added by the range management control unit 4 during the range addition process, and a number increased by one is set, for example.
The start group number 15 (an example of group specifying information) is a start number of a data unit (group) in which the data processing unit 3 collects one or more rows. The data processing unit 3 combines one or more rows into a buffer size to form one group, and assigns serial numbers to the groups in the order of generation.
The number of groups 16 (an example of group identification information) is the number of groups belonging to the range having a number consecutive from the start group number 15.
The start group number 15 and the group number 16 are information for specifying groups belonging to the range indicated by the entry.
The confirmation state 17 (an example of addition availability information) indicates whether a group can be loaded in the range.
A load state 18 (an example of group presence / absence information) indicates whether the range is empty (no group) or loaded.
The confirmation time 19 (an example of the confirmation range time) is a time when the group cannot be loaded into the range (a time when “FIXED” is set in the confirmation state 17 of the entry).
The entry last write time 20 (an example of the last addition time) is the time when database creation, loading, range deletion, or backup restoration was last performed.
The minimum line number 21 (an example of arbitrary data specifying information) and the maximum line number 22 (an example of arbitrary data specifying information) are the minimum value and the maximum value of the line numbers included in the range, respectively.
The minimum time stamp 23 (an example of arbitrary data specifying information) and the maximum time stamp 24 (an example of arbitrary data specifying information) are the minimum value and the maximum of the time stamps (time when a line is added to the range) belonging to the range, respectively. Value.
The minimum line number 21, the maximum line number 22, the minimum time stamp 23, and the maximum time stamp 24 are information for specifying lines belonging to the range indicated by the entry. As long as the information can identify a line belonging to the range indicated by the entry, it may be one of the line number and the time stamp, or may not be the line number and the time stamp. Hereinafter, the minimum line number 21 and the maximum line number 22 are referred to as “range key # 1”, and the minimum time stamp 23 and the maximum time stamp 24 are referred to as “range key # 2”.

  Further, the confirmation state 17 will be described in detail. The confirmed state 17 includes a “current” (CURRENT) state and a “confirmed” (FIXED) state. The range in which the confirmed state is CURRENT is referred to as “current range (processing range)”, and the range in which the confirmed state is FIXED is referred to as “confirmed range (determined range)”. There is only one current range in the database, and all ranges other than the current range are defined ranges. The newly created database contains one current range and no fixed range. That is, when a new database is generated in the data management apparatus 1, the range management control unit 4 generates one entry for the current range in the range management table 5, and does not generate an entry for the determined range. When the data processing unit 3 inputs a range addition request from the terminal device 2, the range management control unit 4 generates a new current range entry in the range management table 5 and confirms the current range entry up to that point. The state 17 is changed from CURRENT to FIXED and set as a fixed range. The range management control unit 4 assigns a range ID obtained by adding 1 to the range ID of the current range so far to the entry of the new current range. The range management control unit 4 does not change the confirmed state 17 by an operation other than the addition of the range. Here, data can be loaded into the current range, and data is not loaded into the determined range (group addition). In other words, the user of the data management system 900 can load data to the current range by making a load request from the terminal device 2 to the data management device 1, but the data can be loaded to the determined range. It cannot be loaded.

  Further, the load state 18 will be described in detail. In the database of the data management apparatus 1, data backup, deletion, and restoration can be performed in units of ranges. Load states include “empty” (EMPTY) and “loaded” (LOADED). A range in which the load state is EMPTY is referred to as an “empty range”, and a range in which the load state is LOADED is referred to as a “loaded range”. When the data processing unit 3 inputs a load request, if the current range is an empty range, the range management control unit 4 changes the load state 18 of the entry from EMPTY to LOADED. When the data processing unit 3 inputs a deletion request for the loaded range, the range management control unit 4 changes the load state 18 of the entry from LOADED to EMPTY. Further, when the search request is input, the data processing unit 3 does not search for data in the empty range and does not refer to data in the empty range. When the data processing unit 3 inputs a backup restore request, the range management control unit 4 restores the load state 18 saved at the time of backup. When a new database is generated in the data management apparatus 1, the range management control unit 4 generates one empty range entry whose load state 18 is EMPTY.

FIG. 7 is a state transition diagram of the range showing changes in the confirmed state 17 and the load state 18 with respect to the range operation processing.
Range operation processing is processing executed by the data management apparatus 1 in response to a processing request input from the terminal device 2, and indicates loading, range deletion, range addition, range designation backup, and restoration. In the operation on the range, the range management control unit 4 indicates the range status (confirmed state 17, load state 18) of the entry in the range management table 5 as indicated by arrows in the figure, ST1, ST2, ST3, Set to any state of ST4.
For example, when performing a load or restore operation on the range of the state ST1 (determined state 17: CURRENT, load state 18: EMPTY), the range management control unit 4 performs the state ST2 (determined on the entry of the range. State 17: CURRENT, load state 18: LOADED). When the range addition operation is further executed, the range management control unit 4 changes the state of the entry in the range that was in the state ST2 to the state ST4 (determined state 17: FIXED, load state 18: LOADED). Note that the state of the range entry newly created by the range management control unit 4 by the addition of the range is the state ST1. In the state ST3, the final state 17 is FIXED and the load state 18 is EMPTY.

The data management device 1 does not perform operations in a range not shown in the figure.
That is, the data management apparatus 1 performs backup, restore, load, and range addition processing on the current range (state ST1) that is empty. At this time, the range management control unit 4 does not change the state of the entry of the range in the range management table 5 when restoring a backup or an empty range, and loads or restores a non-empty range. When the range is added, the state is changed to the state ST3.
In addition, the data management apparatus 1 performs each process of load, backup, range addition, and range deletion for the current non-empty range (state ST2). At this time, the range management control unit 4 does not change the state of the entry of the range in the range management table 5 at the time of loading or backup, and changes to the state ST4 when adding the range. When the range is deleted, the state is changed to the state ST1.
In addition, the data management device 1 performs backup and range deletion processing on a non-empty fixed range (state ST4). At this time, the range management control unit 4 does not change the state of the entry of the range in the range management table 5 at the time of backup, but changes it to the state ST3 at the time of deleting the range.
In addition, the data management device 1 performs backup and restore processing on an empty fixed range (state ST3). At this time, the range management control unit 4 does not change the state of the entry in the range of the range management table 5 at the time of backup or restore of the empty range, and at the time of restore of the non-empty range Changes to state ST4.

As shown in FIG. 7, the range management control unit 4 changes the load state 18 but does not change the finalized state 17 when restoring to the state at the time of backup. Note that the data management apparatus 1 executes the restore process for the range only when the load state 18 is EMPTY.
However, the data management apparatus 1 may execute other processes in each state. For example, overwriting of the loaded line is permitted, and the restore process may be executed even when the load state 18 is LOADED.

Next, the operation of the load process in the data management apparatus 1 will be described.
The data processing unit 3 performs arbitrary data processing on a row input as a load request from the terminal device 2 and creates a group that is a cluster of data obtained by collecting a plurality of rows.
FIG. 8 is an example of a group generated as a result of data processing.
In this example, the group 100 is configured by five rows 101a to 101e. There may be one row included in the group. Moreover, the data arrangement within the group is free. For example, when a row is composed of a plurality of fields, blocks grouped for each field may be further grouped into a group. Alternatively, the data in the group may be collected by performing data processing such as compression or encryption.

In addition, the data processing unit 3 takes out key information for each group when creating a group.
The key information is, for example, the minimum value and the maximum value of the key. Note that the key is set in advance as a database definition and stored in the storage device.
The key information for each group may be the same as or different from the field in which the key attribute is set in the row.
FIG. 9 shows an example of the group 100 in the case where one row has three fields of row number 102a, time stamp 102b, and color 102c.
In FIG. 9, when the fields defined as keys are the row number 102a and the time stamp 102b, the key information of the group 100 includes the row number 102a having the minimum value “1”, the maximum value “5”, and the time stamp 102b. The minimum value is “2005/10/13 12:01”, and the maximum value is “2005/10/20 11:54”.

  The range management control unit 4 inputs the group and its key information from the data processing unit 3, updates the range management table 5, and writes the group to the data storage area unit 7 via the data access control unit 6. The data access control unit 6 stores the group in the data storage area unit 7 via the range management control unit 4.

FIG. 10 is a flowchart showing the operation of the load process in the data management apparatus 1.
A load process executed by the data management device 1 when the data processing unit 3 inputs a load request from the terminal device 2 will be described below with reference to FIG.

First, the data processing unit 3 initializes variables (hereinafter referred to as key information variables) for specifying group key information (minimum value, maximum value). Here, as an example, the key information is two of the line number and the time stamp, and a sufficiently large value (for example, all decimal digits are 9) that cannot normally be taken is set as the minimum value, and the maximum value is set as the maximum value. Sets 0 or a null value (step S100).
Next, the data processing unit 3 inputs a row to be loaded from the terminal device 2 (step S101).
Next, the data processing unit 3 determines the end of line input (step S102).
As a result of the determination, if the line input is not completed, the data processing unit 3 compares the field value of each of the line number and time stamp of the input line with the value of the key information variable, and the value of the field of the input line is If the value of the key information variable is smaller than the minimum value of the key information variable, the minimum value of the key information variable is replaced with the value of the field. Replace with a value. It is assumed that the position of each field in the row is known in advance by referring to the definition information in the database (step S103).
Next, the data processing unit 3 adds the input row to the group. At this time, the data processing unit 3 additionally stores the input group in the buffer memory 951 (step S104).
Next, the data processing unit 3 determines whether or not a condition to be additionally stored in the update area 8 as one group (hereinafter referred to as a group condition) is satisfied. The group condition is set such that the number of rows added to the group satisfies the preset number of rows, or the integrated value of the length of the rows added to the group satisfies the preset group size. And The size of one group is preferably a size suitable for a buffer size close to the buffer size within the buffer size (step S105).
If the group condition is not satisfied, the data processing unit 3 moves the process to step S101.

  When the group condition is satisfied, the data processing unit 3 outputs a data management command for instructing the range management control unit 4 to write the group. The data management command includes a key information variable. The range management control unit 4 outputs a data access instruction for instructing the data access control unit 6 to write a group to the update area 8 of the data storage area unit 7. The data access control unit 6 is generated by the data processing unit 3. The group thus written is additionally written and saved from the buffer memory 951 to the update area 8 of the data storage area section 7. Further, the data access control unit 6 releases the area of the buffer memory 951 in which the group written in the update area 8 is stored (step S106).

  Next, the range management control unit 4 updates the range management table 5. At this time, the range management control unit 4 searches the range management table 5 for an entry whose finalized state 17 is CURRENT. Then, the table last write time 11 of the selected entry, the group specifying information (start group number 15, group number 16) and the row specifying information (minimum row number 21, maximum row number 22, minimum time stamp 23, maximum of the selected entry) Time stamp 24) is updated. When the load state 18 is EMPTY, the load state 18 is changed to LOADED according to the state transition diagram in the range of FIG. When the load state 18 is LOADED, the group number 16 is set to a value obtained by adding 1 to the current group number 16, and when the load state 18 is EMPTY, the start group number of the entry in the previous range is set. A value obtained by adding 15 and the number of groups 16 is set. Then, the current time is set as the entry last write time 20. When the minimum value of the group line number indicated by the key information variable is smaller than the minimum line number 21 of the range management table 5, the minimum line number 21 is replaced with the minimum value of the line number of the key information variable, and the key information variable indicates When the maximum value of the group line number is larger than the maximum line number 22 of the range management table 5, the maximum line number 22 is replaced with the maximum value of the line number of the key information variable. Similarly, the minimum value and maximum value of the time stamp of the group indicated by the key information variable are compared with the minimum time stamp 23 and the maximum time stamp 24 and replaced (step S107).

Next, the data processing unit 3 determines whether the end of line input (no new line is input) is detected in step S102 (step S107).
If the end of line input is detected as a result of the determination, the data processing unit 3 ends the load process.
If the end of line input is not detected, the data processing unit 3 moves the process to step 101.
If it is determined in step S102 that the line has ended, the data processing unit 3 determines whether there is an unwritten group (step S109).
The data processing unit 3 moves the process to step S106 when there is an unwritten group, and ends the loading process when there is no unwritten group.

As a result of the above load processing operation, the group is stored in the update area 8 of the data storage area section 7.
FIG. 11 shows an example in which the data management apparatus 1 stores three groups 100 a to 100 c in the update area 8.

Next, an outline of operation of range addition processing in the data management apparatus 1 will be described.
The data processing unit 3 receives a range addition request (an example of range addition request data) from the terminal device 2, outputs a data management command for instructing addition of a range, and calls the range management control unit 4. The range management control unit 4 outputs a data access command for moving the group in the update region 8 to the reference region 9 and calls the data access control unit 6. The data access control unit 6 moves the group in the update area 8 of the data storage area unit 7 to the reference area 9, and the range management control unit 4 updates the range management table 5.

FIG. 12 is a flowchart showing the operation of range addition processing in the data management apparatus 1.
The range addition process executed by the data management device 1 when the data processing unit 3 inputs a range addition request from the terminal device 2 will be described below with reference to FIG.

First, the data processing unit 3 outputs a data management command for instructing addition of a range and calls the range management control unit 4, and the range management control unit 4 searches the range management table 5 for an entry whose finalized state 17 is CURRENT. To select. Then, the range management control unit 4 acquires the start group number 15 and the group number 16 of the selected entry and outputs them to the data processing unit 3 (step S200).
Next, the data processing unit 3 initializes a counter N, which is a variable for counting the number of processed groups, to 0 (step S201).
Next, the data processing unit 3 determines whether the counter N is smaller than the number of groups 16 input from the range management control unit 4 (step S202).
When the counter N is smaller than 16, the data processing unit 3 moves the group having the group number obtained by adding the counter N to the start group number 15 input from the range management control unit 4 from the update area 8 to the reference area 9. The command is output to the range management control unit 4. The range management control unit 4 outputs to the data access control unit 6 a data access command instructing movement of the group corresponding to the input data management command. Then, the data access control unit 6 reads the group from the update area 8 of the data storage area unit 7 and writes it to the buffer memory 951 (step S203).
Next, the data processing unit 3 outputs a data management command for writing the group stored in the buffer memory 951 to the reference area 9 of the data storage area unit 7 to the range management control unit 4. The range management control unit 4 outputs a data access command corresponding to the data management command to the data access control unit 6, and the data access control unit 6 writes the group stored in the buffer memory 951 into the reference area 9. The area of the buffer memory 951 in which the written group is stored is released (step S204).
Next, the data processing unit 3 adds 1 to the counter N (step S205).
The data management apparatus 1 repeats the above processing until the counter N reaches 16 or more groups.
When it is determined in step S202 that the counter N is 16 or more, the data processing unit 3 outputs a data management command for instructing deletion of the group in the update area 8 to the range management control unit 4. Then, the range management control unit 4 outputs a data access command for instructing deletion of the group in the update area 8 to the data access control unit 6, and the data access control unit 6 deletes the group stored in the update area 8. (Step S206).

As described above, the group in the update area 8 in which a row can be added is moved to the reference area 9 in which only the reference can be made.
FIG. 13 shows an image in which the three groups 100 a to 100 c in the update area 8 are moved to the reference area 9.

  Finally, the data processing unit 3 outputs a data management command requesting addition of an entry to the range management control unit 4, and the range management control unit 4 updates the range management table 5. At this time, the range management control unit 4 changes the confirmed state 17 from CURRENT to FIXED according to the state transition diagram of FIG. 7 for the entry searched in step S200, and sets the current time as the confirmed time 19. Further, the range management control unit 4 adds a new range entry, sets a value obtained by adding 1 to the previous range ID 14 to the range ID 14, and starts the previous range in the start group number 15. The value obtained by adding the group number 15 and the group number 16 is set, the group number 16 is set to 0, the CURRENT state is set to CURRENT, the load state 18 is set to EMPTY, and the entry last write time 20 Set the time and set a null value for the other items in the added entry. Further, the range management control unit 4 sets the current time as the table last write time 11, and sets a value obtained by adding 1 to the range number 12 (step S207).

  FIG. 14 is an example of the range management table 5 immediately after the range is added to the database in the state of the range management table 5 shown in FIG.

Next, an outline of the operation of range deletion processing in the data management apparatus 1 will be described.
The data management apparatus 1 can release the target range area from the data storage area unit 7 by the range deletion process.
FIG. 15 is a conceptual diagram of range deletion, and shows a case where the range 30b is deleted.
First, the data processing unit 3 receives a range deletion request (an example of range deletion request data) by designating a range ID from the terminal device 2, and outputs a data management command for instructing deletion of the range corresponding to the range ID. The management control unit 4 is called. The range management control unit 4 searches for and selects an entry whose range ID 14 matches the range ID indicated by the data management request, and sets the group identification information (start group number 15 and group number 16) of the selected entry in the range management table 5. Get from. Then, a data access command for instructing deletion of the group belonging to the range indicated by the selected entry is output and the data access control unit 6 is called, and the data access control unit 6 selects a group belonging to the range corresponding to the designated range ID. Delete from the data storage area section 7. In addition, the range management control unit 4 updates the range management table 5.

FIG. 16 is a flowchart showing an operation of range deletion processing in the data management apparatus 1.
A range deletion process executed by the data management apparatus 1 when the data processing unit 3 inputs a range deletion request by specifying a range ID from the terminal device 2 will be described below with reference to FIG.

First, the data processing unit 3 outputs a data management command for instructing deletion of the range corresponding to the range ID and calls the range management control unit 4, and the range management control unit 4 sets the range ID 14 to the range ID indicated by the data management command. A matching entry is searched and selected in the range management table 5. Then, the range management control unit 4 acquires the start group number 15 and the group number 16 of the selected entry and outputs them to the data processing unit 3 (step S300).
Next, the data processing unit 3 initializes a counter N, which is a variable for counting the number of processed groups, to 0 (step S301).
Next, the data processing unit 3 determines whether the counter N is smaller than the number of groups 16 input from the range management control unit 4 (step S302).
When the counter N is smaller than the number of groups 16, the data processing unit 3 performs range management of a data management command for instructing deletion of the group corresponding to the group number obtained by adding the counter N to the start group number 15 input from the range management control unit 4 Output to the control unit 4. The range management control unit 4 outputs a data access command for instructing deletion of the group corresponding to the input data management command to the data access control unit 6. Then, the data access control unit 6 releases the area where the group is stored in the data storage area unit 7 (step S304).
Next, the data processing unit 3 adds 1 to the counter N (step S305).
The data management apparatus 1 repeats the above processing until the counter N reaches 16 or more groups.
When it is determined in step S302 that the counter N is 16 or more, the data processing unit 3 outputs a data management command for instructing deletion of an entry to the range management control unit 4, and the range management control unit 4 performs step S300. For the retrieved entry, the load state 18 is changed from LOADED to EMPTY according to the state transition diagram of FIG. However, when the load state 18 is already EMPTY, the load state 18 is not changed. Further, the range management control unit 4 sets the current time as the table final write time 11 and the entry final write time 20 (step S306).

  FIG. 17 is an example of the range management table 5 immediately after the range whose range ID 14 is “2” is deleted from the database in the state of the range management table 5 shown in FIG.

Next, an outline of the search processing operation in the data management apparatus 1 will be described.
The data processing unit 3 receives a search request from the terminal device 2, analyzes the search request, requests the range management control unit 4, and acquires a group to be searched. Then, the data processing unit 3 executes a matching process for each row included in each acquired group and outputs the result to the terminal device 2. The range management control unit 4 reads out the entry of the search target range by referring to the range management table 5 with the key information of the range received as the analysis result of the search request from the data processing unit 3, and searches the search target range with the group specifying information of the entry. Identify the group. Then, the range management control unit 4 requests the data access control unit 6 to read all groups belonging to the search target range from the data storage area unit 7.

FIG. 18 is a flowchart showing the search processing operation in the data management apparatus 1.
A search process executed by the data management device 1 when the data processing unit 3 inputs a search request (an example of search request data) indicating a search condition from the terminal device 2 will be described below with reference to FIG.

First, the data processing unit 3 inputs a search query statement indicating a search condition from the terminal device 2 as a search request (step S400).
Next, the data processing unit 3 extracts a condition that becomes a range key from the search condition indicated by the search query. It is assumed that the range key information is set in advance when the database is defined (step S401).

In the example of the range management table 5 in FIG. 6, two of a line number and a time stamp are defined as range keys.
FIG. 19 shows an example of a search query sentence based on this range of keys.
The search query shown in FIG. 19 has a line number larger than “3000”, a time stamp of “2004/04/16 00:00” or more and “2004/06/15 23:59” or less, and color Means that “RED” is used as a search condition. Here, the information extracted as the range key condition is that the line number is larger than “3000” and the time stamp is “2004/04/16 00:00” or more and “2004/06/15 23:59”. The following information.

Next, the data processing unit 3 determines whether or not the search condition indicated by the search query statement has a condition that becomes a range key (step S402).
When there is a condition that becomes a range key, the data processing unit 3 outputs to the range management control unit 4 a data management command for requesting group specifying information in the range corresponding to the range key. The range management control unit 4 searches the range management table 5, the load state 18 is LOADED, and line identification information (minimum line number 21, maximum line number 22, minimum time stamp 23, maximum time stamp 24) is specified in the range key. ) Is selected, and group identification information (start group number 15, group number 16) of the selected entry is output to the data processing unit 3. The data processing unit 3 inputs the start group number 15 and the group number 16 output by the range management control unit 4 as information on the search target group (step S403).
When the conditions of FIGS. 6 and 19 are assumed, a range whose range ID 14 is 4, 5, and 6 is a search target range, and a group belonging to the search target range is a search target group.
If there is no condition that becomes a range key, the data processing unit 3 outputs a data management command for requesting group specifying information of an entry whose load state 18 is LOADED to the range management control unit 4. The range management control unit 4 searches the range management table 5 and selects an entry in a range where the load state 18 is LOADED. The group specifying information (start group number 15 and group number 16) of the selected entry is used as the data processing unit 3. Output to. The data processing unit 3 inputs the start group number 15 and the group number 16 output by the range management control unit 4 as information on the search target group (step S404).

  Next, the data processing unit 3 outputs a data management command for instructing reading of one group included in the search target group to the range management control unit 4. The range management control unit 4 outputs to the data access control unit 6 a data access command for instructing reading of the group corresponding to the data management command. The data access control unit 6 reads the group to be searched from the data storage area unit 7 and stores it in the buffer memory 951 (step S405).

Next, the data processing unit 3 determines whether or not reading of all groups included in the search target group has been completed (step S406).
If the reading is finished, the data processing unit 3 finishes the search process.
If the reading is not finished, the data processing unit 3 performs a row-by-row collation process on the group stored in the buffer memory 951, and outputs information on the row corresponding to the search condition to the terminal device 2 as a search result. In the case of FIG. 19 as an example, the data processing unit 3 searches each row included in the group stored in the buffer memory 951 and performs a matching process of selecting a row whose “color” field is “RED”. Further, after outputting the search result to the terminal device 2, the data processing unit 3 releases the area of the buffer memory 951 in which the collated group is stored (step S407).

Next, an outline of operation of backup processing in the data management device 1 will be described.
In the data management apparatus 1, as a backup type, in addition to a full backup, a differential backup, and an incremental backup, a backup by specifying a range is realized. A full backup is a backup of all data, a differential backup is a backup of data that has been added or changed after a full backup, and an incremental backup is a backup of data that has been added or changed after a backup other than a full backup (differential backup or incremental backup) It is a backup. A range-designated backup is a backup in a designated range.
The data processing unit 3 receives a backup request (an example of backup request data) from the terminal device 2, acquires the database definition information and the data stored in the data storage area unit 7 via the range management control unit 4, and acquires the data An archive file 300 in which the collected data is collected is generated and stored in the storage device. The range management control unit 4 acquires information in the range management table 5 and requests the data access control unit 6 for a group to be backed up in accordance with a backup command from the data processing unit 3.

FIG. 20 is a diagram showing a data structure of the archive file 300 generated by the data management apparatus 1 in the backup process.
Reference numeral 300 denotes an archive file.
301 is header information (archive header information) of the archive file 300, such as the type of backup, the generation date of the archive file 300, and range information (start range ID, number of ranges).
Reference numeral 302 denotes database definition information, which includes database field information and key information.
Reference numeral 303 denotes information of the range management table 5.
Reference numeral 304 denotes information (group data information) related to a group included in the archive file 300 generated by backing up from the data storage area unit 7.

FIG. 20 shows an example when the number of backup target groups is four.
The group data information 304 includes group data header information 313 and groups 312a to 312d.
The group data header information 313 includes a start group number 305, a group number 306, an update area group number 307, and individual group information 308a to 308d.
Furthermore, the individual group information 308a to 308d includes an offset 309 indicating the position of the group in the archive file 300, a group size 310, and a flag 311 indicating whether the group is an empty group.
The offset 309 (OFFSET # 1) of the individual group information 308a indicates the position of the group 312a (group # 1), the offset 309 (OFFSET # 2) of the individual group information 308b indicates the position of the group 312b (group # 2), The offset 309 (OFFSET # 3) of the individual group information 308c indicates the position of the group 312c (group # 3), and the offset 309 (OFFSET # 4) of the individual group information 308d indicates the position of the group 312d (group # 4).

FIG. 21 is a table (backup start position table) for managing the backup start position used by the data management apparatus 1 for differential backup or incremental backup.
Reference numeral 400 denotes the structure of the backup start position table.
401 is a group number (differential backup start group number) indicating the start position of the backup target in the differential backup.
Reference numeral 402 denotes a group number (incremental backup start group number) indicating the start position of the backup target in the incremental backup.
In the differential backup start group number 401, the data processing unit 3 sets a value obtained by adding 1 to the last group number immediately after the full backup.
In the incremental backup start group number 402, the data processing unit 3 sets a value obtained by adding 1 to the last group number immediately after the differential backup and the incremental backup.

FIG. 22 is a flowchart showing the operation of the backup process in the data management apparatus 1.
The backup process executed by the data management device 1 when the data processing unit 3 inputs a backup request from the terminal device 2 will be described below with reference to FIG.

First, the data processing unit 3 receives a backup request in which a backup type is designated from the terminal device 2. The types of backup include range designation backup, full backup, differential backup, and incremental backup (step S500).
Next, the data processing unit 3 outputs a data management command for requesting information to be set in the archive file 300 to the range management control unit 4, and the range management control unit 4 uses the information acquired from the range management table 5 as the data processing unit 3. Output to. At this time, the range management control unit 4 includes a start group number 305, a group number 306, an update area group number 307 set in the group data header information 313 of the archive file 300, information set in the range management table information 303, Information about the start range ID and the number of ranges set in the archive header information 301 is acquired from the range management table 5 (step S501).
Next, the data processing unit 3 creates archive header information 301, database definition information 302, range management table information 303, and group data header information 313, and writes them in the archive file 300 in order. At this time, the data processing unit 3 generates archive header information 301 based on the backup request input in step S500 and the information in the range management table 5 acquired from the range management control unit 4 in step S501, and is defined when the database is created. The database definition information 302 is generated based on the information thus obtained, and the range management table information 303 is generated based on the information of the range management table 5 acquired from the range management control unit 4 in step S501. Further, the data processing unit 3 generates group data header information 313 based on the information of the range management table 5 acquired from the range management control unit 4 in step S501. The data processing unit 3 sets predetermined initial values in the individual group information 308 a to 308 d in the group data header information 313. The range management table information 303 includes, for example, a load state 18 (an example of group presence / absence information) (step 502).

Next, the data processing unit 3 initializes a counter N, which is a variable for counting the number of processed groups, to 0 (step S503).
Next, the data processing unit 3 determines whether or not the counter N is smaller than the group number 306 set in the group data header information 313 of the archive file 300 (step S504).
When the counter N is smaller than the number of groups 306, the data processing unit 3 is a variable indicating the number of the backup target group by adding the counter N to the start group number 305 set in the group data header information 313 of the archive file 300. The group number GN is set (step S505).
Next, the data processing unit 3 determines whether the group indicated by the group number GN is not an empty group based on the load state 18 included in the range management table information 303. At this time, when the load state 18 is EMPTY, the data processing unit 3 determines that the group is empty (step S506).
If it is determined that the group with the group number GN is empty, the data processing unit 3 moves the process to step S509.
If it is determined that the group with the group number GN is not empty, the data processing unit 3 outputs a data management command requesting a group corresponding to the group number GN to the range management control unit 4, and the range management control unit 4 A data access command requesting a group corresponding to is output to the data access control unit 6. The data access control unit 6 reads the group with the group number GN from the data storage area unit 7 and stores it in the buffer memory 951 (step S507).
Next, the data processing unit 3 additionally writes out a group corresponding to the group number GN stored in the buffer memory 951 to the archive file 300. In addition, the data processing unit 3 releases the area of the buffer memory 951 in which the group corresponding to the group number GN is stored (step S508).
In addition, the data processing unit 3 updates the group data header information 313 output to the archive file 300 in step S502 for the group corresponding to the group number GN. Specifically, the offset 309 in the archive file 300 as the individual group information, the group size 310, and a flag 311 indicating whether or not the group is an empty group are set (step S509).
Next, the data processing unit 3 adds 1 to the counter N (step S510).
The data management apparatus 1 repeats the above processing until the counter N reaches the number of groups 306 or more.
If it is determined in step S504 that the counter N is greater than or equal to the number of groups 306, the data processing unit 3 updates the backup start position table 400 and ends the backup process. When updating the backup start position table 400, in the case of a full backup, the data processing unit 3 starts differential backup with a value (group number GN + 1) obtained by adding 1 to the group number of the group last added to the archive file 300. Set to group number 401. In addition, in the case of differential backup or incremental backup, the data processing unit 3 sets a value obtained by adding 1 to the group number of the group last added to the archive file 300 (group number GN + 1) as the incremental backup start group number 402. (Step S511).

FIG. 23 is a flowchart illustrating the details of the operation in step S501 for each type of backup.
First, the data processing unit 3 determines the type of backup indicated by the backup request. The range designation backup request includes a range ID indicating the range to be backed up (step S600).
In the case of range-designated backup, the data processing unit 3 determines the start group number 15 and the group for the entry whose range ID 14 selected by the range management control unit 4 by searching the range management table 5 matches the range ID included in the backup request. Equation 16 is obtained. In step S502, the data processing unit 3 sets the start group number 15 and the group number 16 acquired from the range management table 5 to the start group number 305 and the group number 306 of the group data header information 313 of the archive file 300. (Step S601).
In the case of full backup, the data processing unit 3 calculates the start group number and the number of groups for the entire range based on the information in the range management table 5. The start group number calculated here is the minimum start group number 15 in all entries of the range management table 5, and the number of groups is the total value of the group numbers 16 of all entries. In step S <b> 502, the data processing unit 3 sets the calculated start group number and group number to the start group number 305 and the group number 306 of the group data header information 313 of the archive file 300. The data processing unit 3 acquires information of the range management table 5 via the range management control unit 4 (step S602).
In the case of differential backup, the data processing unit 3 refers to the backup start position table 400 to acquire the differential backup start group number 401, and determines the number of groups after the differential backup start group number 401 based on the information in the range management table 5. calculate. The number of groups calculated here is obtained by subtracting the differential backup start group number 401 from the value obtained by adding the group number 16 of the entry having the maximum start group number 15 to the maximum start group number 15 in all entries of the range management table 5. It is a value obtained by adding 1 to the value. In step S502, the data processing unit 3 sets the differential backup start group number 401 to the start group number 305 of the group data header information 313 for the archive file 300, and sets the calculated number of groups to the group of the group data header information 313. It sets to number 306. The data processing unit 3 acquires information of the range management table 5 via the range management control unit 4 (step S603).
In the case of incremental backup, the data processing unit 3 refers to the backup start position table 400 to acquire the incremental backup start group number 402 and calculates the number of groups after the incremental backup start group number 402 based on the range management table 5. . The number of groups calculated here is obtained by subtracting the incremental backup start group number 402 from the value obtained by adding the number of groups 16 of the entry having the maximum start group number 15 to the maximum start group number 15 in all entries of the range management table 5. It is a value obtained by adding 1 to the value. In step S502, the data processing unit 3 sets the incremental backup start group number 402 to the start group number 305 of the group data header information 313 for the archive file 300, and sets the calculated number of groups to the group of the group data header information 313. It sets to number 306. The data processing unit 3 acquires the information of the range management table 5 through the range management control unit 4 (step S604).

  In the processing of step S601 to step S604, the data processing unit 3 starts or calculates the start group number and the number of groups that are acquired or calculated according to the type of backup and set in the group data header information 313 of the archive file 300. In each group from the group number, the number of groups belonging to the range indicated by the entry whose determination state 17 of the range management table 5 is CURRENT is calculated, and the calculated group number is group data header information of the archive file 300 in S502. The number of groups in the update area 307 set to 313 is assumed. Further, the data processing unit 3 calculates the range ID and the range number of each range to which the group corresponding to the start group number and the group number acquired or calculated according to the type of backup belongs based on the range management table. In step S502, the data processing unit 3 sets the calculated range ID and the number of ranges as the start range ID and the number of ranges of the archive header information 301 of the archive file 300. The data processing unit 3 acquires information of the range management table 5 via the range management control unit 4 (step S605).

Next, an outline of restore processing operation in the data management apparatus 1 will be described.
In response to the restore request from the terminal device 2, the data management apparatus 1 executes the restore process via the range management control unit 4 and the data access control unit 6 with the archive file 300 generated during the backup process as an input. In response to the restore request input by the data processing unit 3, the range management control unit 4 updates the range management table 5, and the data access control unit 6 writes the group to be restored to the data storage area unit 7.

FIG. 24 is a flowchart showing the restore processing operation in the data management apparatus 1.
A range deletion process executed by the data management apparatus 1 when the data processing unit 3 inputs a restore request from the terminal apparatus 2 will be described below with reference to FIG.

First, the data processing unit 3 receives a restore request from the terminal device 2 (step S700).
Next, the data processing unit 3 acquires the archive file 300 from the storage device of the data management apparatus 1 and checks whether the restore condition is satisfied according to the backup type included in the archive header information 301. For example, when the archive file 300 generated by the range-designated backup is restored, the load condition 18 of the entry indicating the range to be restored is EMPTY according to the state transition diagram of FIG. For example, the database definition information 302 and the range management table information 303 are consistent with the database definition information and the range management table information 303 of the database that is currently being restored. When the archive file 300 is restored in the overwrite mode, the load state 18 of the entry indicating the range to be restored need not be EMPTY. The data processing unit 3 acquires information of the range management table 5 via the range management control unit 4 (step S701).
If it is determined that the restore condition is not satisfied, the data processing unit 3 ends the restore process.
When it is determined that the restoration condition is satisfied, the data processing unit 3 refers to the archive file 300 and calculates the value obtained by subtracting the update area group number 307 from the group number 306 as the variable, the reference area group number RN. (Step S702).
Next, the data processing unit 3 initializes a counter N that is a variable to 0 in order to count the number of processed groups (step S703).
Next, the data processing unit 3 determines whether the counter N is smaller than the group number 306 (step S704).
When the counter N is smaller than the group number 306, the data processing unit 3 sets a value obtained by adding the counter N to the start group number 305 to the group number GN that is a variable (step S705).
Next, the data processing unit 3 refers to the archive file 300 and determines whether the individual group information flag 311 corresponding to the group number GN indicates empty (step S706).
If it is determined that the flag 311 does not indicate empty, the data processing unit 3 reads the group corresponding to the group number GN from the archive file 300 and stores it in the buffer memory 951 (step S707).
If it is determined that the flag 311 indicates empty, the data processing unit 3 creates a virtual empty group having the size 310 of the individual group information as setting information (step S708).
Next, the data processing unit 3 determines whether or not the counter N is smaller than the reference area group number RN (step S709).
When the counter N is smaller than the reference area group number RN, the data processing unit 3 outputs a data management command for writing the restoration target group to the reference area 9 of the data storage area unit 7 to the range management control unit 4. The management control unit 4 outputs a data access command corresponding to the data management command to the data access control unit 6, and the data access control unit 6 writes the group stored in the buffer memory 951 into the reference area 9. Further, the data access control unit 6 releases the area of the buffer memory 951 in which the group written in the reference area 9 is stored (step S710).
If the counter N is equal to or greater than the number RN in the reference area, the data processing unit 3 outputs to the range management control unit 4 a data management instruction for writing the restoration target group to the update area 8 of the data storage area unit 7, The range management control unit 4 outputs a data access command as a data management command to the data access control unit 6, and the data access control unit 6 writes the group stored in the buffer memory 951 into the update area 8. In addition, the data access control unit 6 releases the area of the buffer memory 951 in which the group written in the update area 8 is stored (step S711).
Next, the data processing unit 3 adds 1 to the counter N and shifts the processing to step S704.
If it is determined in step S704 that the counter N is greater than or equal to the number of groups 306, the data processing unit 3 outputs to the range management control unit 4 a data management command for instructing restoration of the range management table 5, and the range management control unit 4 Updates the range management table 5, and the data management apparatus 1 ends the restore process (step S713).

FIG. 25 is a flowchart showing the operation of updating the range management table 5 in step S713.
First, the data processing unit 3 refers to the backup type of the archive header information 301 of the archive file 300, and determines whether or not a range-designated backup is restored (step S800).
In the case of restoring a range-designated backup, the data processing unit 3 outputs a data management command for instructing restoration of one range of the range management table 5 to the range management control unit 4. The range management control unit 4 changes the load state 18 of the entry in the range to be restored in the range management table 5 to LOADED according to the state transition diagram of FIG. 7 (step S801).
If it is not the restoration of the range designation backup, the data processing unit 3 outputs a data management command for instructing the restoration of the entire range of the range management table 5 to the range management control unit 4, and the range management control unit 4 performs the range management of the archive file 300. The information in the range management table 5 is replaced with the table information 303 (step S802).

The buffer memory in the embodiment may be a cache memory that is faster than the buffer memory.
In the data management apparatus 1 represented by a database, it is desired to increase the speed of data access. Therefore, the data management device 1 in the embodiment improves the data access efficiency by inputting / outputting individual data (rows) between the CPU 911 and the magnetic disk device 920 via the buffer memory (or cache memory). To speed up data access. In addition, data is managed in units of data (groups) in which the rows are grouped by buffer size (or cache size), and the rows are accessed in units of groups. Therefore, the rows corresponding to the buffer size (or cache size) are inserted. It is easy to output, and the data access efficiency is further improved. In addition, the data management device 1 stores data (range) in which groups are arbitrarily arranged in a continuous area of a storage area (reference area 9 or update area 8), so that a magnetic disk device can be used when accessing a continuous group. 920 shortens the data access time for moving the header and magnetic disk.
Therefore, the data management device 1 in the embodiment is particularly useful as a write-once database, for example, data generated in an arbitrary time unit by setting a time unit such as hour, day, week, or month as a range. Can be processed at high speed.

As described above, the first embodiment employs a method in which data is continuously arranged and stored in the data storage area unit 7 in units of groups in which a plurality of rows are grouped. Since the input / output unit is a group unit and the groups to be accessed are continuously arranged, high-speed data access is possible.
Further, it is possible to have key information (minimum value, maximum value) for each range in the range management table 5, and use this key information in the search to narrow down the data in the access target range before reading the data. And efficient data access becomes possible.
In addition, the range management table 5 has a load state 18, and an empty range can be excluded from the search target, thereby enabling efficient data access.
In addition, in the data load, it is only necessary to continuously add data in increments of groups to the update area 8 to which data addition is permitted, and since there is no index, high-speed loading is possible.
Further, the range can be added at an arbitrary time by the user instead of a predetermined time unit.
Further, since the range is managed as a continuous area arranged in the data storage area unit 7 by the range management table 5, batch processing is possible and it does not take time to delete the range.
In addition, in the backup, in addition to the realization of the range-designated backup, the read range can be limited by skipping the backup process for the empty range with reference to the load state 18 of the range management table 5, so that the backup can be efficiently performed. Is possible.
Further, in the restoration, in addition to the realization of the range-specified restoration, it is possible to faithfully restore the range deleted from the range management table 5 included in the archive file 300 as being deleted.

In the first embodiment, the following data management apparatus 1 has been described.
As the database data storage area, there is an update area that allows data addition and a reference area that allows only data reference. The data in the update area is added to the reference area, and the data added to the reference area is from the update area. A data management apparatus 1 including data storage means (data storage area unit 7) to be deleted.

  Data processing means (data processing unit 3) for performing predetermined data processing on a plurality of rows (records), and managing a range on the data storage area in units of a group of a plurality of rows as a result of the data processing A range management control means (range management control unit 4) that includes a range management table and manages and controls the ranges that are sequentially added to the data storage area, and access to the group on the data storage area that stores the group A data management device 1 comprising data access control means (data access control unit 6) for controlling the data.

  The range management control means is configured to select either the current state of the range (CURRENT) in which the addition of the group is permitted or the state of the range (FIXED) in which the addition of the group is not permitted (FIXED). The data management apparatus 1 is characterized in that the confirmed state 17) is held in the range management table and the range is managed.

  The range management control means moves the group stored in the update area to the reference area when changing the state of the range from the current state of the range to the state of the determined range. A data management device 1.

  When the range management control unit changes the state of the range from the state of the current range to the state of the determined range, the plurality of groups stored in the update area are set to a predetermined condition (for example, a buffer The data management apparatus 1 is merged according to size) and moved to the reference area.

  The range management control means sets the range status to either the status of the loaded range indicating that the group has been added or the status of an empty range to which the group has not been added. The data management device according to claim 1, wherein the state of the current range is set to the state of the loaded range when the group is added.

  The range management control means is configured to select one of the range state (LOADED) indicating that the group has been added and an empty range state (EMPTY) to which the group has not been added. The data management apparatus 1 is characterized in that a state (load state 18) is held in the range management table, and the state of the empty range is set when the range is deleted.

  The range management control means includes the range state of the loaded range indicating that the group has been added and the range state of the empty range to which the group has not been added, and the range state. Predetermined key range information (minimum row number 21, maximum row number 22, minimum time stamp 23, maximum time stamp 24) related to the included rows is held in the range management table, and the data processing means in the search is the empty The data management apparatus 1 is characterized in that a search process is performed by removing the group in the range in which the range or the search condition is false in comparison with the key range information from the search target.

  The range management control means sets the range status to either the status of the loaded range indicating that the group has been added or the status of an empty range to which the group has not been added. And the data processing means performs a backup designating the range.

  The data management apparatus 1 is characterized in that in the restoration of a backup in which the range is specified, the restoration is executed only when the information in the range is the same as that at the time of backup.

  The range management control means sets the range status to either the status of the loaded range indicating that the group has been added or the status of an empty range to which the group has not been added. And the data processing means generates an archive file that holds the information in the empty range during backup.

  In backup restoration, the data processing means restores the information in the empty range.

  The range management table retains time information (determined time 19) when changing from the current state of the range to the state of the fixed range, and executes the data processing means and the range management control means. A data management apparatus 1 characterized by the following.

  The range management table holds information on the last addition time (entry last write time 20) when the group was added to the range, and executes the data processing unit and the range management control unit. Data management device 1 to perform.

  The range management table holds information on the minimum row number and the maximum row number of rows included in the range, and performs a search by the data processing means and the range management control means. .

  The range management table holds information on the minimum time stamp and the maximum time stamp of the rows included in the range, and performs a search by the data processing means and the range management control means. .

Embodiment 2. FIG.
The data management apparatus 1 according to the first embodiment adds a range by moving a group in the update area 8 of the data storage area unit 7 to the reference area 9.
On the other hand, in the second embodiment, a data management apparatus 1 will be described in which a new group generated by merging a group in the update area 8 is moved to the reference area 9.

FIG. 26 is a conceptual diagram illustrating group merging processing.
FIG. 26 shows that the groups 100a to 100c become the new group 1000a and the groups 100d to 100g become the new group 1000b by the group merge processing executed by the data processing unit 3.
Assume that a group merge condition is set in advance in the storage device of the data management apparatus 1 and can be arbitrarily set for the group to be merged. In the following description, a new group after merging the group is referred to as a merge group.

  The configuration diagram of the data management system 900 according to the second embodiment and the diagram showing an example of the range management table 5 are the same as those in the first embodiment (FIGS. 3 and 6). However, the data management device 1 according to the second embodiment includes a merge buffer memory having the same buffer size as the buffer memory 951.

FIG. 27 is a flowchart illustrating an operation in the range addition process according to the second embodiment.
The operation of range addition processing in the data management apparatus 1 according to the second embodiment will be described below with reference to FIG.

When the data processing unit 3 inputs a range addition request from the terminal device 2, the data processing unit 3 outputs a data management command for requesting group identification information of the current range to the range management control unit 4. The range management control unit 4 searches the range management table 5 for and selects an entry in the current range whose finalized state 17 is CURRENT, obtains the start group number 15 and the group number 16 of the selected entry, and performs data processing. It outputs to the part 3 (step S900).
Next, the data processing unit 3 initializes a counter N that is a variable for counting the number of processed groups to 0 (step S901).
Next, the data processing unit 3 initializes a counter M, which is a variable for counting the number of merged groups, to 0 (step S902).
Next, the data processing unit 3 determines whether or not the counter N is smaller than the number of groups 16 input from the range management control unit 4 (step S903).
When the counter N is smaller than 16, the data processing unit 3 sends a data management command for requesting a group corresponding to the group number obtained by adding the counter N to the start group number 15 input from the range management control unit 4 to the range management control unit. 4 is output. The range management control unit 4 outputs a data access command corresponding to the data management command to the data access control unit 6. The data access controller 6 reads the group corresponding to the data access command from the update area 8 of the data storage area 7 and stores it in the buffer memory 951 (step S904).
Next, when there is a merge group that has not been written in the reference area 9, the data processing unit 3 merges the group stored in the buffer memory 951 into the merge group to form a new merge group. At this time, the data processing unit 3 stores the merge group in the merge buffer memory. In addition, the data processing unit 3 releases the area of the buffer memory 951 in which the merged group is stored (step S905).
Next, the data processing unit 3 adds 1 to the counter N (step S906).
Next, the data processing unit 3 determines whether or not the merge group satisfies the merge completion condition. As the merge completion condition, an arbitrary condition such as the maximum number of groups to be merged or the maximum size of the merge group can be set in advance. It is preferable that the size of the merge group matches the buffer memory 951 (step S907).
If it is determined that the merge completion condition is not satisfied, the data processing unit 3 moves the process to step S903.
If it is determined that the merge completion condition is satisfied, the data processing unit 3 outputs a data management command for writing the merge group to the reference area 9 of the data storage area unit 7 to the range management control unit 4. The range management control unit 4 outputs a data access command corresponding to the data management command to the data access control unit 6, and the data access control unit 6 writes the merge group stored in the merge buffer memory in the reference area 9. In addition, the data access control unit 6 releases the merge buffer memory area in which the group written in the reference area 9 is stored (step S908).
Next, the data processing unit 3 adds 1 to the counter M and shifts the processing to step S903 (step 909).
The data management apparatus 1 repeats the above processing until the counter N reaches 16 or more groups.
If it is determined in step S903 that the counter N is equal to or greater than 16 groups, the data processing unit 3 determines whether there is any unwritten merge group (step S910).
If it is determined that there is an unwritten merge group, the data processing unit 3 writes the merge group to the reference area 9 in the same manner as in steps S908 and 909 (step S911), and adds 1 to the counter M (step S912).
If it is determined that there is no unwritten merge group, the data processing unit 3 moves the process to step S913.
Next, the data processing unit 3 outputs a data management command for instructing deletion of the group in the update area 8 to the range management control unit 4, and the range management control unit 4 accesses the data access command corresponding to the data management command for data access. The data access control unit 6 deletes the group stored in the update area 8 (step S913).
As described above, the groups in the update area 8 are merged and moved to the reference area 9.

  Finally, the data processing unit 3 outputs to the range management control unit 4 a data management command for instructing the update of the range management table 5 after merging when the number of merge groups is the counter M. The range management control unit 4 The management table 5 is updated. At this time, the range management control unit 4 searches for an entry whose finalized state 17 is CURRENT and selects it from the range management table 5. The range management control unit 4 sets the counter M, which is the number of merge groups, in the group number 16 of the selected entry, changes the confirmation state 17 from CURRENT to FIXED, and sets the current time as the confirmation time 19. Also, the range management control unit 4 adds a new range entry, sets a value obtained by adding 1 to the range ID of the previous entry to the range ID 14, and sets the value of the previous entry to the start group number 15. The value obtained by adding the starting group number 15 and the group number 16 is set, the group number 16 is set to 0, the CURRENT state is set to CURRENT, the load state 18 is set to EMPTY, and the entry last write time 20 is Set the current time and set null values for the other items in the entry. Further, the range management control unit 4 sets the current time as the table last write time 11, and sets a value obtained by adding 1 to the range number 12 (step S914).

  The group merging process is performed not only when the range addition process is executed, but also when the group in the update area 8 exceeds a predetermined size (for example, buffer size) or the number of cases during the process of adding (loading) a row. Sometimes it may be done.

The second embodiment is different from the first embodiment in that a group in the update area 8 is merged to generate a new group when a group in the update area 8 is moved to the reference area 9 when a range is added. Is different.
Thus, even when the row loading frequency is low and the size of the group added to the update area 8 is reduced and the access efficiency is lowered, the group size appropriate for improving the input / output efficiency by the merge processing ( For example, the buffer size can be maintained, and the access efficiency can be improved.

The following data management apparatus 1 has been described in the second embodiment.
The range management control means (range management control unit 4) changes the range state (fixed state 17) from the current range (CURRENT) state to the fixed range state (FIXED) and adds a new range The data management apparatus 1 is characterized in that the current range is set as the current range and there is always only one current range.

FIG. 2 is a diagram illustrating an appearance of a data management system 900 according to the first embodiment. 1 is a hardware configuration diagram of a data management system 900 according to Embodiment 1. FIG. 1 is a configuration diagram of a data management system 900 according to Embodiment 1. FIG. FIG. 3 is a conceptual diagram of “range” realized by the data management apparatus 1 according to the first embodiment. The conceptual diagram at the time of adding a range with respect to the database 30 shown in FIG. An example showing range management information held in the range management table 5 in the first embodiment. FIG. 6 is a state transition diagram of a range shown with respect to a change in a confirmed state 17 and a load state 18 with respect to a range operation process in the first embodiment. 4 is an example of a group generated as a result of data processing in the first embodiment. An example of the group 100 when one row in the first embodiment has three fields of a row number 102a, a time stamp 102b, and a color 102c. 5 is a flowchart showing an operation of a load process in the data management apparatus 1 according to the first embodiment. An example in which the data management apparatus 1 according to the first embodiment stores three groups 100 a to 100 c in the update area 8. 5 is a flowchart showing an operation of range addition processing in the data management apparatus 1 according to the first embodiment. The image which moved the three groups 100a-100c in the update area 8 in Embodiment 1 to the reference area 9. FIG. The example of the range management table 5 immediately after adding a range with respect to the database in the state of the range management table 5 shown in FIG. The conceptual diagram at the time of deleting the range 30b in Embodiment 1. FIG. 5 is a flowchart showing an operation of range deletion processing in the data management apparatus 1 according to the first embodiment. The example of the range management table 5 immediately after deleting the range whose range ID 14 is "2" with respect to the database in the state of the range management table 5 shown in FIG. 5 is a flowchart showing the search processing operation in the data management apparatus 1 according to the first embodiment. 4 is an example of a search query statement in the first embodiment. FIG. 3 is a diagram showing a data structure of an archive file 300 generated by the data management apparatus 1 according to Embodiment 1 in a backup process. FIG. 3 is a diagram showing a table for managing a backup start position used by the data management apparatus 1 in the first embodiment for differential backup or incremental backup. 5 is a flowchart showing the operation of backup processing in the data management apparatus 1 according to the first embodiment. The flowchart which demonstrated the detail of operation | movement divided into the kind of backup about step S501. 5 is a flowchart showing an operation of restore processing in the data management apparatus 1 according to the first embodiment. The flowchart which showed the operation | movement of the update of the range management table 5 in step S713. FIG. 11 is a conceptual diagram illustrating group merging processing according to the second embodiment. 9 is a flowchart showing an operation in range addition processing according to the second embodiment.

Explanation of symbols

  1 data management device, 2 terminal device, 3 data processing unit, 4 range management control unit, 5 range management table, 6 data access control unit, 7 data storage area unit, 8 update area, 9 reference area, 10 generation date, 11 Table last write time, 12 Range number, 13 Range key number, 14 Range ID, 15 Start group number, 16 Group number, 17 Final state, 18 Load state, 19 Final time, 20 Final write time, 21 Minimum row number, 22 Maximum line number, 23 Minimum time stamp, 24 Maximum time stamp, 30 Database, 30a, 30b, 30c, 30d Range, 100 group, 101 line, 102a Line number, 102b Time stamp, 102c Color, 200 Search query, 300 Archive file 301 Archive Header information, 302 Database definition information, 303 Range management table information, 304 Group data information, 305 Start group number, 306 Number of groups, 307 Number of groups in the update area, 308 Individual group information, 309 Offset, 310 size, 311 flag, 312 Group, 400 Backup start position table, 401 Differential backup start group number, 402 Incremental backup start group number, 900 Data management system, 901 Display device, 902 K / B, 903 Mouse, 904 FDD, 905 CDD, 906 Printer device, 907 Scanner device, 910 system unit, 911 CPU, 912 bus, 913 ROM, 914 RAM, 915 communication board, 920 magnetic disk device, 9 1 OS, 922 Window system, 923 Program group, 924 File group, 931 telephone, 932 FAX machine, 940 Internet, 941 web server, 942 LAN, 951 buffer memory, 1000 a group.

Claims (16)

A buffer memory in which a plurality of arbitrary data is stored as a group ;
A first storage device having a plurality of ranges as storage area ranges for storing groups;
For each range of the storage area of the first storage device, group specifying information for specifying a plurality of groups stored in the range, and an option for specifying a plurality of arbitrary data included in the group stored in the range A second storage device that stores a range entry including data specifying information and group presence / absence information that specifies a range in which no group is stored as an empty range ;
An input device to which search request data including a search condition for a value of arbitrary data and a search condition for the arbitrary data specifying information is input;
A range management control unit that searches and selects a range entry that matches a search condition of arbitrary data specifying information included in the search request data and indicates that the group presence / absence information is not an empty range;
An input / output control unit that acquires the group indicated by the group specifying information of the range entry selected by the range management control unit from the first storage device, and stores the acquired group in the buffer memory;
A data processing unit that searches for and outputs arbitrary data that meets a search condition for a value of the arbitrary data included in the search request data from the group stored in the buffer memory. Management device. A buffer memory in which a plurality of arbitrary data is stored as a group ;
A first storage device having a plurality of ranges as storage area ranges for storing groups;
For each range of the storage area of the first storage device, group specifying information for specifying a plurality of groups stored in the range, and an option for specifying a plurality of arbitrary data included in the group stored in the range A second storage device for storing a range entry including data specifying information, an adding range information for specifying a processing range in which a group is being added, and a confirmed range in which a group has been added ;
An input device to which arbitrary data is input;
A data processing unit that generates a new group in the buffer memory including arbitrary data input to the input device;
When the data processing unit generates a new group, it searches for and selects a range entry in the processing range based on the addability information, and group identification information of the range entry in the processing range is stored in the processing range. Update the information to identify the group and the new group, and include the arbitrary data specifying information of the range entry of the processing range in the arbitrary data included in the group stored in the processing range and the new group A range management control unit that updates information to identify arbitrary data to be
An input / output control unit that acquires the new group from the buffer memory and stores the acquired new group in a storage area that is a storage area in the processing range and that is continuous with the group that is stored in the processing range; <br/> A data management device characterized by comprising: The second storage device includes group presence / absence information that specifies a range in which no group is stored as an empty range , and stores the range entry information in the range entry,
When the data processing unit generates a new group, the range management control unit searches for and selects a range entry in the processing range based on the addability information, and the range entry in the processing range is not an empty range. 3. The data management apparatus according to claim 2, wherein group presence / absence information is set. The second storage device stores the last addition time when the group was last added to the range, in the range entry,
The scope management control unit according to any one of claims 2 to 3, characterized in that to set the final additional time ranging entry process in the range of time during which the data processing unit has generated a new group Data management device. A buffer memory in which a plurality of arbitrary data is stored as a group ;
An update area having one in-process range as a storage area range for adding and storing a group, and a storage area range for storing the group and not storing the additional group A first storage device having a plurality of reference areas as a fixed range;
Second storage for storing a range entry including additional availability information for specifying whether the range is a processing range or a fixed range for each range of the storage area of the first storage device Equipment,
An input device to which range addition request data for requesting addition of a range is input;
When range addition request data is input to the input device, the range entry in the processing range is searched and selected based on the addition availability information, the addition availability information of the selected range entry is changed to a fixed range, and processing is performed. A range management control unit that newly adds a range entry in which addition permission information indicating a middle range is set and stores the range entry in the second storage device;
When the range addition request data is input to the input device, the group stored in the in-process range of the update area is stored in the buffer memory, and the group stored in the buffer memory is stored in the fixed range of the reference area. A data management apparatus comprising: an input / output control unit that stores in a fixed range in which no group is stored, and deletes the group stored in the update area . The data management device further includes:
When a range addition request data is input to the input device, a data processing unit that merges a plurality of groups stored in the update area to generate one group,
6. The data management apparatus according to claim 5, wherein the input / output control unit additionally stores the group generated by the data processing unit in the reference area. The second storage device stores the range entry including the fixed range time when the range becomes the fixed range,
The scope management control unit, either one of claims 5 to claim 6, characterized in that the setting confirmation range time range entry process in the range of time when the ranging addition request data to the input device is input The data management device described in 1. A buffer memory in which a plurality of arbitrary data is stored as a group ;
A first storage device having a plurality of ranges as storage area ranges for storing groups;
For each range of the storage area of the first storage device, a range entry including range identification information for identifying the range and group presence / absence information for specifying a range in which no group is stored as an empty range is stored. Two storage devices;
An input device to which range deletion request data including identification information for identifying a range to be deleted is input;
Range for searching for and selecting a range entry including range identification information that matches the identification information included in the range deletion request data input to the input device, and setting group presence / absence information indicating an empty range in the selected range entry data management apparatus comprising the <br/> a management control unit. A buffer memory for storing a plurality of arbitrary data as a group ;
A first storage device having a plurality of ranges as storage area ranges for storing groups;
For each range of the storage area of the first storage device, a range entry including group specifying information for specifying a plurality of groups stored in the range and range identification information for identifying the range is stored for each range. A second storage device,
A third storage device;
An input device to which backup request data including identification information for identifying a range to be backed up is input;
A range management control unit that searches and selects a range entry including range identification information that matches the identification information included in the backup request data input to the input device;
An input / output control unit that acquires the group indicated by the group specifying information of the range entry selected by the range management control unit from the first storage device, and stores the acquired group in the buffer memory;
A data management apparatus comprising: a data processing unit that generates a backup file including a group stored in the buffer memory and stores the generated backup file in the third storage device. The second storage device includes group presence / absence information that specifies a range in which no group is stored as an empty range , and stores the range entry information in the range entry,
10. The data management apparatus according to claim 9, wherein the data processing unit generates the backup file including group presence / absence information of a range entry selected by the range management control unit. The range management control unit includes group presence / absence information included in a backup file when restore request data for restoring the backup file stored in the third storage device to the first storage device is input to the input device. To the range entry,
11. The data management apparatus according to claim 10 , wherein the input / output control unit stores a group included in a backup file in the first storage device when restore request data is input to the input device. The data processing unit generates a backup file including identification information for identifying a backed up range,
The range management control unit searches and selects a range entry including range identification information that matches the identification information included in the backup file when restore request data requesting restoration of the backup file is input to the input device. And
Entering-output control unit performs the restoration processing for storing a group to group presence information in the range entry the range management control unit has selected are included in the backup file if shows the empty range in the first storage device, the data management system of claim 11, wherein the group presence information ranging entry the range management controller selects does not perform the restore process if not to indicate an empty range. Said second storage device, storing a range entries contain any data specifying information for specifying the minimum number and maximum number among the numbers identifying each of the plurality of arbitrary data stored in the range The data management apparatus according to any one of claims 1 to 12, wherein the data management apparatus is characterized in that: The second storage device stores a range entries contain any data specifying information for specifying the minimum time stamp and maximum time stamp among the time stamps included in the plurality of arbitrary data respectively stored in the range data management apparatus according to any one of claims 1 to 13, characterized in that. A buffer memory in which a plurality of arbitrary data is stored as a group, a first storage device having a plurality of ranges as a storage region range for storing a group, and each storage area range of the first storage device The group specification information for specifying a plurality of groups stored in the range, the arbitrary data specification information for specifying a plurality of arbitrary data included in the group stored in the range, and a range in which no groups are stored are defined as empty ranges. A second storage device that stores a range entry including group identification information to be specified; an input device to which search request data including a search condition for a value of arbitrary data and a search condition for the arbitrary data specification information is input; A data management method for a data management device comprising a range management control unit, an input / output control unit, and a data processing unit,
The range management control unit uses a CPU (Central Processing Unit) to identify a range entry that satisfies the search condition of the arbitrary data specifying information included in the search request data and indicates that the group presence / absence information is not an empty range. Search and select
The input / output control unit acquires the group indicated by the group specifying information of the range entry selected by the range management control unit from the first storage device, stores the acquired group in the buffer memory,
The data processing unit uses a CPU to search for and output arbitrary data that meets a search condition for a value of arbitrary data included in the search request data from the group stored in the buffer memory. A data management method for the data management apparatus . A data management program for causing a computer to execute the data management method according to claim 15 .

Images