JP2001202276A - Huge data managing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently retrieve and update huge data from a comparatively small-scale information processing system. SOLUTION: In the huge data managing system, a data managing system 1a is provided with a page managing mechanism 10a, a data managing system connecting mechanism 20a and a data storing mechanism 30a, and an information processing system 1b is provided with a page data access mechanism 10b and a data managing system connecting mechanism 20a.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for managing long data such as multimedia data and document data, and an information processing system for accessing the long data.

[0002]

2. Description of the Related Art In a conventional data management system, even when it is desired to search only a part of long data, it is necessary to read all the data at once from the beginning. , It was difficult to search for long data. Further, even when only a part of the read long data is changed, it is necessary to write all the data at once from the beginning, and the updating efficiency is low.

Japanese Patent Application Laid-Open No. H11-120067, entitled "Database Processing Method for Preliminarily Managing Partial Information of Data," and Japanese Patent Application Laid-Open No. H09-305449, "Database Management System" And the like.

[0004]

By dividing and managing long data into certain pages and enabling search / update on a page-by-page basis, even in a relatively small-scale information processing system, the information processing system can be used. It is possible to sequentially search and update long data that is larger than the allowable range in page units, and it is possible to improve access efficiency to long data by searching and updating only a necessary portion of the long data.

[0005]

SUMMARY OF THE INVENTION The present invention relates to a data management system for managing long data, a data storage mechanism for storing the long data, a page management mechanism for managing the long data in page units, and a data management system. This can be achieved by providing a page data access mechanism for controlling search / update of long data in units of pages in an information processing system for accessing long data, and a data management system connection mechanism in both systems.

[0006]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, an embodiment of a long data management system according to the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a long data management system according to one embodiment of the present invention. FIG.
9 is a flowchart illustrating a search process in the information processing system. FIG. 3 is a flowchart illustrating an update process in the information processing system. FIG. 4 is a flowchart illustrating data access processing in the data management system.

In FIG. 1, 1a is a data management system, 1b is an information processing system, 2a is a data storage device,
10a is a page management mechanism, 10b is a page data access mechanism, 20a and 20b are data management system connection mechanisms, and 30a is a data storage mechanism.

In one embodiment of the long data management system shown in FIG. 1, the data management system 1a and the information processing device 1b are provided as one and separate systems for simplification of the drawing. May be constituted by a plurality, or the data management system 1a and the information processing apparatus 1b may exist in the same system. In the following description, it is assumed that the data management system 1a and the information processing device 1b are connected by a general communication line by the data management system connection mechanisms 20a and 20b.

The information processing system 1b requests the page data access mechanism 10b to acquire long data.
The page data access mechanism 10b first inquires of the page management mechanism 10a about the configuration information (the number of pages and the data length per page) of the corresponding long data. The page management mechanism 10a creates and returns configuration information of the corresponding long data from the management information for each long data. Thereafter, the page data access mechanism 10b sets the acquired page number and the number of acquired pages, and issues a long data acquisition request to the page management mechanism 10a. The number of acquired pages is set to a number of pages within a range allowable by the information processing apparatus 1b. The acquisition page number can be set to any value at the beginning, middle, or end. The page management mechanism 10a of the data management system 1a determines the physical position and ordering of the corresponding page on the data storage device 2a from the management information for each long data, and starts with the obtained page number and starts with the length of the number of obtained pages. Data is retrieved from the data storage device 2a through the data storage mechanism 30a and returned to the page data access mechanism 10b. At this time, the page management mechanism 10a prefetches and stores the last page and several pages before and after the last page among the returned pages, and prepares for the next long data acquisition request. The reason for storing the previous and next pages is that the pages constituting long data may move not only in the forward direction but also in the reverse direction, and it is possible to improve the search efficiency by pre-reading the pages. Become.

In the information processing apparatus 1b, the page data access mechanism 10b sets a checksum for each page of long data held after the search. The page data access mechanism 10b compares the checksum with each page of long data held at the time of the next search or at the end of connection with the data management system 1a. As a result of the comparison, if there is a page that has been updated, the page data access mechanism 10b sends a long data update request to the page management mechanism 10a with the updated page number and update data attached. The page management mechanism 10a that has received the long data update request determines the physical position of the corresponding page on the data storage device 2a from the management information for each long data, and updates the page with the update data. The update data may be larger than the data length per page. In this case, the page management mechanism 10a first writes the update data of the data length per page to the page to be updated. Next, a new page is allocated on the data storage device 2a, the remaining update data is written, and the new page is associated between the page to be updated and the page immediately after it, and the management information for each long data is stored. To update.

Next, an embodiment of a long data search process in the information processing system 1b will be described with reference to the flow shown in FIG.

(1) Page data access mechanism 10b
Acquires the configuration information (the number of pages and the data length per page) of the corresponding long data from the page management mechanism 10a (step 201).

(2) In the case of the first search processing (step 2
02), the acquired page number is set to 0 (step 20)
4).

(3) In the case of the second or subsequent search processing (step 202), the acquired page number is set to n. This n
Is (previous acquired page number + acquired page number) for forward search, and (previous acquired page number-acquired page number) for reverse search (step 20).
3).

(4) The number of acquired pages is determined by the information processing apparatus 1b.
Is set to a number of pages within an allowable range (step 20).
5).

(5) A long data acquisition request is made to the page management mechanism 10a (step 206).

(6) Receive large data from the page management mechanism 10a (step 207).

Next, an embodiment of the long data update processing in the information processing system 1b will be described with reference to the flow shown in FIG. The page data access mechanism 10b performs the following steps for each page of the long data held at the time of the next search or at the end of the connection with the data management system 1a.

(1) Page data access mechanism 10b
Compares each page held with the checksum of each page (step 301).

(2) As a result of the comparison, if there is an update (step 302), an updated page number is set (step 303), and update data is set (step 30).
4).

(3) A long data update request is issued to the page management mechanism 10a (step 305).

Next, an embodiment of the long data access processing in the data management system 1a will be described with reference to the flow shown in FIG.

(1) The page management mechanism 10a receives a request from the page data access mechanism 10b (step 401).

(2) When the received request type is a configuration information acquisition request for long data (step 402), the management information for each long data is used to determine the number of pages of the corresponding long data and the data length per page. Created as
It is returned to the page data access mechanism 10b (step 403).

(3) If the received request type is a search request (step 402), the physical position and ordering of the corresponding page on the data storage device 2a are determined from the management information for each long data, and The data of the corresponding page of the long data is taken out (step 404) and returned to the page data access mechanism 10b (step 40).
5). Thereafter, prefetching of several pages before and after is performed (step 406) to prepare for the next search request.

(4) If the received request type is an update request (step 402), the process is distributed according to the update type (step 407).

(5) When the update type is added (step 4
07), a new page is allocated on the data storage device 2a (step 408), update data is written (step 409), the association with the previous and next pages is changed, and the management information for each long data is updated (step 410). .

(6) When the update type is changed (step 4)
07), from the management information for each large data, the physical position of the corresponding page on the data storage device 2a is determined,
Write the updated data to the corresponding page (step 41)
1). At this time, if the update data is larger than the data length per page (step 412), the data storage device 2a
A new page is allocated thereon (step 413), the remaining data of the update data is written (step 414), the association with the previous and next pages is changed, and the management information for each long data is updated (step 415).

(7) When the update type is delete (step 4)
07), from the management information for each large data, the physical position of the corresponding page on the data storage device 2a is determined,
The corresponding page is deleted from the data storage device 2a (step 416), the association with the previous and next pages is changed, and the management information for each long data is updated (step 41).
7).

[0031]

As described above, according to the present invention, the data management system 1a includes the page management mechanism 10a, the data management system connection mechanism 20a, and the data storage mechanism 30.
By providing the page data access mechanism 10b and the data management system connection mechanism 20a in the information processing system 1b, it is possible to efficiently search and update long data.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a long data management system according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a search process in the information processing system.

FIG. 3 is a flowchart illustrating an update process in the information processing system.

FIG. 4 is a flowchart illustrating a data access process in the data management system.

[Explanation of symbols]

1a: data management system, 1b: information processing system,
2a: data storage device, 10a: page management mechanism, 10
b: page data access mechanism, 20a, 20b: data management system connection mechanism, 30a: data storage mechanism.

Claims (3)

[Claims] 1. Dividing long data such as multimedia data and document data into a certain fixed page unit, and managing physical position and ordering information of each page on a data storage device, thereby enabling a Can be searched and updated, and the top, middle,
A long data management system that can be accessed from any page at the end and can be moved forward and backward from the last accessed page. 2. The system is connected to the data management system according to claim 1 and sequentially retrieves long and long data in a page unit managed by the data management system according to claim 1 for a number of pages within an allowable range of the system in a forward direction and a backward direction. Information processing system that can do. 3. The information processing system according to claim 2, wherein
A checksum is provided for each page of long data held after the search, and the checksum is compared with each held page at the next search or at the end of connection with the data management system of claim 1. An information processing system capable of detecting a page that needs to be updated and sequentially updating the page.