JP3929269B2 - Data processing method and apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention manages a large file management system that needs to store a large amount of data and particularly a multimedia database, a map information database, and a knowledge database that handle character text, graphics, and image data. De Data processing method And processing equipment It is about.
[0002]
[Prior art]
In a system that handles long data (hereinafter simply referred to as “data”) such as character text, graphics, and image data, the logical record length of the data increases, and an external storage device that stores the data and a central processing unit that processes the data Since the amount of data transferred between the input / output buffers set in the main storage device is enormous, the access performance cannot be obtained due to the limit of the data transfer speed. In addition, by repeatedly updating, adding, and deleting a huge number of records, invalid areas are scattered in the file, and the file space efficiency deteriorates.
For this, for example, as shown in JP-A-1-106217, instead of creating independent files in a plurality of secondary storage devices, divided files that constitute one file as a whole The file information related to the other divided files is stored in one of the plurality of divided files, and each secondary storage indicated by the file information is read by reading the above file information at the time of data storage / reading request. There is a data processing device that stores data in parallel in each divided file in the device and reads data in parallel from each divided file. This technique has an advantage that the data access speed can be increased by parallel processing of data, and the user does not need to be aware of the location management of the divided files.
Separately, as shown in Japanese Patent Publication No. 2-52305, in an image file device that stores a document such as a drawing as an image file, the input document is divided into a predetermined number of image units. There is an image data updating method in which, when a document is changed, only the image unit whose contents have been changed is stored again. This technology does not newly store the entire changed image, so the image storage space of the image file can be used effectively, and the difference between the uncorrected document and the corrected document is automatically recognized. There is an advantage that it is not necessary to instruct which part of the corrected document has been changed, and the load on the operator can be reduced.
[0003]
[Problems to be solved by the invention]
In the above prior art, since all the data processing of each divided file is always performed in parallel, data operation for processing only a part of the data in the divided file, management of necessary input / output buffers, data storage in the divided file The system and variable length data format are not considered, and there are the following problems.
In other words, in the method of storing data divided into a plurality of divided files in anticipation of parallel data transfer, a large number of input / output buffers are prepared in the main storage device in order to obtain the effect of parallel data transfer. There must be. Since multimedia data such as character text, graphics, and images has a record length of several hundred KB to several hundred MB, the main storage capacity required for input / output of one record increases. For this reason, if all record operation requests are made in units of records, when multiple record operation requests are multiplexed, the main storage capacity is limited, and a time loss due to waiting for I / O buffer free space occurs. Throughput decreases.
In the record search instruction, when the search condition determination process is performed by reading data into the input / output buffer in the main storage device, the search condition determination is performed only on a part of the data items. If all the data constituting a record is read, useless data unrelated to the search condition occurs and the input / output buffer is unnecessarily occupied in the case where the condition hit does not occur.
Further, when data is updated in units of records, even if data is divided and stored and input / output in parallel, the portions that have not been updated become useless input / output, and the input / output buffer is also occupied uselessly. Further, in the image file device, in order to recognize and re-store only the image unit that has actually changed among the divided and stored images, it is necessary to input the corrected manuscript and the data before correction and to compare the differences. Thus, useless input of an image unit whose data has not been updated and useless occupation of an input / output buffer occur similarly.
[0004]
On the other hand, in the divided storage of records into a plurality of files, the following problems occur unless the lengths of the divided sub-records are the same.
That is, if the length of each divided sub-record is changed, the capacity of each divided file changes, and the storage address such as the relative byte address in the file or the physical address where each sub-record is stored is also different for each divided file. It will change to. For this reason, random access to any record is possible if each subrecord realization value has the storage address of the corresponding next subrecord realization value and does not associate each subrecord that constitutes one record. become unable. On the other hand, when such association between sub-records is performed, the storage address of the sub-record is not known unless the previous sub-record having the storage address of the sub-record is input except for the sub-record including the item that is the key for storing the record. , It becomes impossible to input each sub-record in parallel.
Further, when the record has a variable length format, there are the following problems.
In other words, when divided storage is performed in a plurality of files with the maximum record length, the record realization value having a short record length may or may not be stored in the rear divided file. For this reason, a sub-record realization value having all invalid areas or many invalid areas is generated, and the space efficiency of the rear divided file is deteriorated.
The present invention has been made in view of the above circumstances, and an object of the present invention is to solve the above-described problems in the prior art, and to process a large number of long data using a limited input / output buffer area. It is an object of the present invention to provide a data processing method and apparatus which can be performed at high speed and efficiently with the minimum necessary input / output processing.
Other objects of the present invention will become apparent from the following description.
[0005]
[Means for Solving the Problems]
An object of the present invention is to provide a record including a memory for storing management information and a plurality of storage devices, and including a fixed portion of a logical record and a variable portion of the logical record. Min Split The case In the data processing method in the data processing apparatus, the fixed portion of the logical record is Divided into at least one fixed-length sub-record, A plurality of fixed lengths provided in the storage device Case in physical continuous area The variable part of the logical record, As a variable length sub-record, Variable length provided in the storage device Physical continuous area Storage information for specifying the storage location of the fixed-length sub-record in the storage device. Said In memory Remembered Record management table Stored in The search request is analyzed in response to the input of the search request, and the record in the search request is analyzed. Co From the record specified in the record operation command and the record management table ,in front The storage device storing the specified sub-record of the specified record is specified and stored in the memory. Remembered The storage information of the sub-record is acquired with reference to a record management table, the specific sub-record is acquired from the specified storage device, and search processing is performed on the acquired sub-record based on the search request A data processing method, a memory for storing management information, and a plurality of storage devices, and a record including a fixed portion of a logical record and a variable portion of the logical record is divided into the plurality of storage devices In the data processing device for storing, the fixed part of the logical record is Divide it into at least one fixed-length sub-record and Multiple fixed lengths provided in the storage device Case in physical continuous area Means for delivering the variable part of the logical record, As a variable length sub-record, Variable length provided in the storage device Physical continuous area And storage information for specifying the storage location of the fixed-length sub-record in the storage device. Said memory Remembered In record management table Store Analyzing the search request in accordance with the input of the search request, from the record specified in the record operation instruction in the search request and the record management table ,in front The storage device storing the specified sub-record of the specified record is specified and stored in the memory. Remembered Means for acquiring storage information of the sub-record with reference to a record management table, acquiring the specific sub-record from the specified storage device, and performing a search process on the acquired sub-record based on the search request This is achieved by a data processing device characterized in that.
[0006]
In the data processing method according to the present invention, record operations can be realized by inputting / outputting all sub-records in parallel in the operation of records divided and stored in sub-record units. On the other hand, if the number of required subrecords is limited to the minimum according to the requested data operation, the amount of input / output buffers required for data operation and storage The number of inputs and outputs can be reduced.
Further, the file definition information registration / reference means of the data processing apparatus according to the present invention has file definition information indicating which file corresponds to each sub-record into which the record is divided in the registration list, and extends the record operation command. Can be referred to arbitrarily. This enables both parallel input / output of all subrecords and individual input / output of each subrecord at the time of executing the record operation instruction. In addition, it is possible to arbitrarily select in which file each subrecord is stored. For example, a semiconductor memory device capable of high-speed input / output is used according to the access characteristics of each subrecord. By selecting an external storage device, the performance of the entire system can be improved.
[0007]
Further, when the operation unit of the record operation instruction to be provided is an item unit or a sub-record unit, the sub-record selection means automatically selects the minimum sub-record necessary for the record operation based on the file definition information. This makes it possible to reduce the number of files that are actually input / output and the amount of necessary input / output buffers, compared to the case where all subrecords are always operated in record units.
Further, the sub-record unit input / output buffer management means can secure / release an input / output buffer in sub-record length units from a limited common buffer pool area. As a result, a plurality of record operation commands that can be processed simultaneously with the same input / output buffer amount can be increased as compared with the case where all sub-records are always operated in units of records. In addition, since the input data remaining in the input / output buffer is also managed in units of sub-records, the probability that a frequently accessed sub-record will remain in the input / output buffer increases, and the desired data is read hit in the input / output buffer. The probability of doing will also increase.
Furthermore, the sub-record unit parallel input / output means can perform independent input / output for each sub-record unit with the external storage device using the sub-record unit input / output buffer. This enables parallel input / output of all subrecords or multiple subrecords in a record, as well as individual input / output of specific subrecords, and provides a means to perform the necessary minimum input / output according to record operation instructions at high speed. it can.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a configuration diagram of a file management system according to an embodiment of the present invention. When the record operation instruction 1 and the data 2 are given to the file management system 10, various data processing such as data retrieval, update, deletion, and storage with respect to the external storage device 20 is performed. When the file definition parameter 3 is given to the file management system 10, the file definition information is registered in the file definition information register 30.
The file management system 10 includes a record operation instruction execution processing unit 12 that controls analysis and execution of the record operation instruction 1, and a record search processing unit 13 that performs various record operations in response to a call from the record operation instruction execution processing unit 12. , Record update processing unit 14, record deletion processing unit 15, record storage processing unit 16, sub-record selection processing unit 17 for selecting sub-records actually required for various record operations, and input / output of sub-records A sub-record unit input / output buffer management processing unit 18 that manages the input / output buffer, a sub-record unit parallel input / output processing unit 19 that inputs / outputs data in sub-record units between the external storage device 20, and a file definition parameter 3 is parsed and edited and registered in the file definition information register 30, and a record operation instruction execution processing unit And a file definition information registration / reference processing unit 11 can refer the file definition information required in accordance with the call from 2.
[0009]
The record search processing unit 13 includes a sub-record search processing 13a that is commonly called by the record update processing unit 14 and the record deletion processing unit 15. The sub-record unit input / output buffer management processing unit 18 performs four processes of search processing, return processing, reservation processing, and release processing, and the sub-record unit parallel input / output processing unit 19 performs two processes of input processing and output processing. The file definition information registration / reference processing unit 11 is divided into two processes, a reference process and a registration process.
FIG. 2 is a diagram illustrating an example of a data storage method in the file management system according to the present embodiment. The logical record data 50 is divided into sub-records 51, 52, 53, 54, and physical continuous areas 71, 72, 73, 74 in the external storage devices 21, 22, 23, 24 capable of parallel input / output. Are stored respectively. The physical continuous area 71 is composed of physical records 80 and 81 which are units of physical input / output. Similarly, the physical continuous areas 72, 73, and 74 are composed of physical records 82 and 83, physical records 84 and 85, and physical records 86, 87, and 88, respectively.
Here, the physical continuous area is, for example, an area which hits one cylinder of the magnetic disk device and can perform continuous input / output of physical records in the area mechanically at high speed. The subrecords 51, 52, and 53 are fixed portions of the logical record, and the physical continuous areas 71, 72, and 73 have the same physical length and are stored at the same physical address.
[0010]
The sub-record 54 is a variable part of the logical record, and the physical continuous area 74 and the physical continuous areas 71, 72, 73 are not necessarily the same physical length and the same physical address. Therefore, the pointer by the storage address of the physical record 86 of the physical continuous area 74 storing the variable sub-record 54 in the physical record 85 of the physical continuous area 73 storing the final sub-record 53 of the fixed part. 75, and a correspondence between sub-records is attached.
Similarly, the logical record data 60 is divided into sub-records 61, 62, 63, and 64, and physical continuous areas 91, 92, and 92 in the external storage devices 21, 22, 23, and 24 capable of parallel input / output. The fixed part and the variable part are associated with each other by the pointer 95 based on the storage address.
FIG. 3 is a diagram illustrating an example of a data format of the record management table in the file management system according to the present embodiment. The record management table 45 includes a header part 46 having information related to records and an entry part 47 having information related to sub-records. The entry unit 47 has an array format composed of entries for the number of sub-records constituting the record.
[0011]
FIG. 4 is a flowchart illustrating a processing procedure of the file definition information registration / reference processing unit 11 according to the present embodiment.
FIG. 5 is a flowchart illustrating a processing procedure of the record operation instruction execution processing unit 12 according to the present embodiment.
FIG. 6 is a flowchart illustrating a processing procedure of the record search processing unit 13 according to the present embodiment.
FIG. 7 is a flowchart showing the processing procedure of the sub-record search processing in the record search processing unit 13 of the present embodiment.
FIG. 8 is a flowchart illustrating the processing procedure of the sub-record selection processing unit 17 of the present embodiment.
9 to 11 are flowcharts showing the processing procedure of the record update processing unit 14 of the present embodiment.
FIG. 12 is a flowchart illustrating a processing procedure of the record deletion processing unit 15 according to the present embodiment.
13 and 14 are flowcharts illustrating the processing procedure of the record storage processing unit 16 of the present embodiment.
[0012]
FIG. 15 is a flowchart illustrating the search processing procedure of the sub-record unit input / output buffer management processing unit 18 according to this embodiment.
FIGS. 16 and 17 are flowcharts showing the procedure of the return process, the securing process, and the releasing process of the sub-record unit input / output buffer management processing unit 18 of this embodiment.
FIG. 18 is a flowchart illustrating the input processing procedure of the sub-record unit parallel input / output processing unit 19 according to the present embodiment.
FIG. 19 is a flowchart showing the procedure of the output process of the sub-record unit parallel input / output processing unit 19 of this embodiment.
Next, the operation of each processing unit of the embodiment shown in FIG. 1 will be described based on the flowcharts shown in FIGS.
First, the operation of the file definition information registration / reference processing unit 11 will be described with reference to FIG. When registering file definition information (step 100), record name, fixed length / variable length type, record length / maximum record length, subrecord name, subrecord length, subrecord storage specified as file definition parameter 3 File definition information such as a file name is input (step 102), edited and output to the file definition information register 30 (step 104).
[0013]
When referring to the file definition information (step 106), the definition information of the record name designated from the file definition information register 30 is input and returned to the request source (step 108). In other cases, error processing is performed (step 110) and an error is reported to the request source.
Next, the operation of the record operation instruction execution processing unit 12 will be described with reference to FIG. When the record operation instruction 1 is input, the record operation instruction execution processing unit 12 accepts and analyzes the record operation instruction (step 200), and if there is an error, processes the error (steps 202 and 230) Report the error to the requester.
If there is no error, it is checked whether the record management table 45 of the specified record has been created in the memory (step 204). If the record management table is not found in the memory (step 206), file definition information registration / reference is made. The processing unit 11 is called with a reference request (step 208), file definition information is input, the record management table 45 of the record is created in the memory, and the control flags of all subrecord entries are initialized (step 210). ).
[0014]
If the record management table 45 is found in the memory, only the control flags of all sub-record entries in the record management table 45 are initialized (steps 206 and 212).
If the specified record operation command is a search command, the record search processing unit 13 is called (steps 214 and 216). If the record command is an update command, the record update processing unit 14 is called (steps 218 and 220). In this case, the record deletion processing unit 15 is called (steps 222 and 224), and in the case of a storage instruction, the record storage processing unit 16 is called (steps 226 and 228).
In other cases, error processing is performed (step 230) and an error is reported to the request source.
Next, the operation of the record search processing unit 13 will be described with reference to FIG. The record search processing unit 13 determines whether or not the specified record operation command has a search condition (step 300). If there is no search condition, the sub-record selection processing unit 17 is called without specifying the search condition (step 302). If there is a search condition, the sub-record selection processing unit 17 is called with the search condition specified (step 304) to determine a sub-record that needs to be searched.
[0015]
The storage address of the search target record is obtained from the key item of the search condition, the relation with the record searched immediately before or the storage position of the first record (step 306). Next, the sub record search process 13a is called (step 308) to search for a sub record.
Here, as a method of obtaining the storage address of the search target record by the key item of the search condition, an index is added to the key item to perform a binary search, or the correspondence between the key item and the record storage address is determined by a hash function. There are methods to calculate.
If there is a search condition in the sub-record search process by the above-described sub-record search process 13a (step 310), it is determined whether or not the searched sub-record satisfies all the search conditions (step 312). If it cannot be determined that there is no record satisfying (step 318), the sub-record unit I / O buffer management processing unit 18 is called with a return request (step 320), and after the buffer is returned once, the processing from step 306 is executed. repeat.
[0016]
If it can be determined that there is no record that satisfies the search condition due to the single condition of the key item (step 318), error processing is performed (step 322), and the sub-record unit I / O buffer management processing unit 18 is called with a return request (step 326), the buffer is returned, and an error is reported to the request source.
If the searched sub-records satisfy all the search conditions (step 312), the sub-record selection processing unit 17 is called without specifying the search conditions (step 314), and the sub-records that are not necessary for determining the search conditions are also selected. Then, the sub-record search process 13a is called (step 316) to perform a search.
If there is no search condition in step 310, and if a record that satisfies the search condition is found in step 312 or later, the searched data is transferred to the area from which the record search command is issued (step 324), and the sub-record unit input / output buffer The management processing unit 18 is called with a return request (step 326), the buffer is returned, and the process ends.
Next, the operation of the sub-record search process 13a will be described with reference to FIG.
[0017]
Here, first, it is positioned at the first sub-record entry of the above-mentioned record management table 45 (step 330), and it is determined whether or not the operation target flag is set (step 332). When the operation target flag is set, in order to search for the sub-record, the storage address of the search target record and the input request flag are set (step 334), and the sub-record unit input / output buffer management processing unit 18 is requested to search. (Step 336) to determine whether the sub-record has already been input on the buffer. If the operation target flag is not set, nothing is done.
The above-described processing from step 332 to step 336 is repeated for all sub-record entries in the fixed part of the record management table 45 (steps 338 and 340). Next, in order to read all necessary sub-records of the fixed part, the sub-record unit parallel input / output processing part 19 is called with an input request (step 342). Then, it is determined whether or not there is a variable part in the definition of the record (step 344). If there is no variable part, the process is terminated.
[0018]
On the other hand, if there is a variable part, it is positioned at the first sub-record entry of the variable part of the record management table 45 (step 346), and it is determined whether the operation target flag is set (step 348). If the operation target flag is not set, the process is terminated. If the operation target flag is set, it is determined whether the storage address of the variable part is in the last subrecord of the fixed part (step 350).
If there is no storage address for the variable part, the process ends. If there is a storage address for the variable part, the storage address is set as the storage address of the sub record entry of the variable part, and the input request flag is set (step 352). ). Similarly to the fixed part, the sub-record unit I / O buffer management processing unit 18 is called with a search request for the variable part sub-record (step 354) to determine whether the sub-record has already been input on the buffer. Then, the sub-record unit parallel input / output processing unit 19 is called with an input request (step 356), the sub-record is read, and the process is terminated.
Next, the operation of the sub-record selection processing unit 17 will be described with reference to FIG.
[0019]
Here, it is positioned at the first subrecord entry of the record management table 45 (step 400), and the following processing is repeated for all subrecord entries of the record management table 45 (steps 424 and 426). If the request is not a search condition specification (step 402), it is determined whether the operation unit of the record operation instruction is an item unit or a sub-record unit (step 406). An operation target flag is set in the sub record entry (step 414).
If the operation unit of the record operation instruction is an item unit or a sub-record unit, it is determined whether the sub-record is a sub-record including an operation unit item or a sub-record of the operation unit (step 408). If it is a sub-record that contains or an operation unit sub-record, an operation target flag is set in the sub-record entry (step 414).
[0020]
If neither the sub-record containing the operation unit item nor the sub-record of the operation unit is determined, it is determined whether or not the sub-record is the final sub-record of the fixed part (step 410). If the variable part is a sub-record including an operation unit item or an operation unit sub-record, the operation target flag is also set in the last sub-record entry of the fixed part (step 414).
Next, when the request is a search condition specification (step 402), it is determined whether the sub-record is a sub-record including a search condition item (step 416). An operation target flag is set in the sub record entry (step 422). If the sub-record is not a sub-record including the search condition item, it is determined whether or not it is the last sub-record of the fixed part (step 418), and the last sub-record of the fixed part includes a variable part and the variable part includes the search condition item. If it is a record (step 420), an operation target flag is set in the sub-record entry (step 422). When the above processing is processed for all the subrecord entries of the record (step 424), the processing ends.
[0021]
Next, the operation of the record update processing unit 14 will be described with reference to FIGS.
First, it is determined whether or not the update target record or sub-record has been searched immediately before (step 500). If it has not been searched, error processing is performed (step 502) and an error is reported to the request source.
If already searched, the sub-record selection processing unit 17 is called to select a sub-record to be updated (step 504), and the storage address of the record or sub-record that has been searched immediately before is obtained (step 506). The retrieval process 13a is called (step 508), and all sub-records to be updated are retrieved.
Subsequently, it is positioned at the first subrecord entry of the record management table 45 (step 510), and if the operation target flag is set for all the subrecord entries of the fixed part (step 512), it is input to the buffer. The update data is reflected on the sub-record (step 514) and the output request flag is set (step 516). The process is repeated for all the sub-records in the fixed part (steps 518 and 520).
[0022]
Next, it is determined whether or not there is a variable part (step 522). If there is a variable part, it is positioned at the first subrecord entry of the variable part of the record management table 45 (step 524). Perform unique processing.
If the operation target flag is set in the sub-record of the variable part (step 526) and there is a buffer address (step 528), it is determined whether the update data is longer than the pre-update data input to the buffer However, if it is long, the sub-record storage address of the pre-update data is saved (step 532).
On the other hand, if the buffer address is not set (step 528), the sub-record unit input / output buffer management processing unit 18 is called with the securing request (step 534), and an output data output buffer is secured. Then, as a new storage location of the variable part sub-record, the free area address of the variable part sub-record storage file is obtained (step 536), the storage address of the sub-record entry and the variable part storage in the last sub-record of the fixed part The address is updated with this value (step 538).
[0023]
If the update data is shorter than the pre-update data (step 530), the storage address need not be updated. Next, the update data is reflected in the buffer (step 540), and an output request flag is set (step 542). After completion of the processing unique to the variable part and when there is no variable part (step 522), the sub-record unit parallel input / output processing unit 19 is called with an output request (step 546), and the update data is output and the sub-record unit The input / output buffer management processing unit 18 is called with a return request (step 548), and the data is left on the input / output buffer.
Next, it is determined whether or not there is a pre-update sub-record storage address saved in step 532 (step 550), and if not saved, the process ends. If the pre-update sub-record storage address is saved, it is positioned at the first sub-record entry of the variable part of the record management table 45 (step 552), and the saved pre-update sub-record storage address is stored in the sub-record entry. Set to address (step 554).
[0024]
Then, the sub-record unit input / output buffer management processing unit 18 is called with a secure request (step 556), a buffer is secured, empty record data is set (step 558), and an output request flag is set (step 556). 560), the sub-record unit parallel input / output processing unit 19 is called with an output request (step 562), and the area in which the sub-record before update is stored is made free.
Further, the sub-record unit input / output buffer management processing unit 18 is called with a release request (step 564), the buffer is released, and the processing is terminated.
Next, the operation of the record deletion processing unit 15 will be described with reference to FIG.
First, it is determined whether or not the record or sub-record to be deleted has been searched immediately before (step 600). If it has not been searched, error processing is performed (step 602) and an error is reported to the request source. If the search has been completed, the sub-record selection processing unit 17 is called (step 604), the sub-record to be updated is selected, and the storage address of the record or sub-record that has been searched immediately before is obtained (step 606). The process 13a is called (step 608), and all sub-records to be deleted are searched.
[0025]
Next, it is positioned at the first subrecord entry of the record management table 45 (step 610), and when the operation target flag and the input completed flag are set for all the subrecord entries (step 612), they are input to the buffer. The empty record data is reflected in the current sub-record (step 614), the output request flag is set (step 616), and the process is repeated (steps 618 and 620). Next, the sub-record unit parallel input / output processing unit 19 is called with an output request (step 622), and empty record data is output to all the sub-records to be deleted to delete the records.
Finally, it is positioned again at the first subrecord entry of the record management table 45 (step 624), and for all subrecord entries, the subrecord unit input / output buffer management processing unit 18 is called with a release request (step 626), and the buffer After repeating the process of releasing (steps 628 and 630), the process ends.
Next, the operation of the record storage processing unit 16 will be described with reference to FIGS.
[0026]
First, the sub-record selection processing unit 17 is called (step 700), and all sub-records are set as operation targets, and are related to the key item or the record searched immediately before or the free area address of the sub-record storage file of the fixed part. The storage address of the record is obtained (step 702). Subsequently, the following processing is repeated for all the subrecord entries, positioned at the first subrecord entry of the record management table 45 (steps 714 and 716).
If the sub-record is a fixed part (step 706), the sub-record unit I / O buffer management processing unit 18 is called with a secure request (step 708), the buffer is secured, and the stored data is transferred to the buffer (step 710). A record storage address and an output request flag are set (step 712). If the sub-record is a variable part (step 706), it is determined whether there is stored data (step 718), and if not, nothing is done.
[0027]
If there is stored data, the sub-record unit I / O buffer management processing unit 18 is called with an allocation request (step 720), the buffer is allocated, and the free area address of the sub-record storage file of the variable unit is obtained (step 722). Then, update the storage address of the subrecord entry and the variable part storage address in the last subrecord of the fixed part with the value (step 724), transfer the stored data to the buffer (step 726), and set the output request flag (Step 728).
Finally, when the processing of all sub-record entries is completed (step 714), the sub-record unit parallel input / output processing unit 19 is called with an output request (step 730) to store the data, and the sub-record unit input / output buffer management processing unit 18 is called with a release request (step 732), the buffer is returned, and the process is terminated.
Next, the search processing operation of the sub-record unit input / output buffer management processing unit 18 will be described with reference to FIG.
[0028]
First, it is determined whether or not a buffer address has been set for the sub-record entry of the designated record management table 45 (step 800). If it has not been set, and has been set, the buffer has already been entered in the buffer pool. If it is not a buffer (step 812), or if it is an input buffer in the buffer pool but a sub-record of the storage address set in the sub-record entry has not been input (step 814), the sub-record entry The sub-record having the storage address set to is searched from the input buffer of the buffer pool (step 802).
As a result of the above search, if it is found from the input buffers in the buffer pool (step 804), the input buffer is set to the in-use buffer, and the buffer address is set to the buffer address of the sub-record entry (step 804). 806), an input completion flag is set (step 808), and the process is terminated. If it is not found from the input buffers in the buffer pool (step 804), the sub-record unit input / output buffer management processing unit 18 is called with a secure request (step 810), a new buffer is secured, and the process ends.
[0029]
On the other hand, if the buffer at the buffer address specified in the sub-record entry is an input buffer in the buffer pool (step 812) and the sub-record at the specified storage address is input (step 814), The input buffer is used as a buffer in use (step 816), the input flag of the sub-record entry is set (step 808), and the process ends.
Next, the operations of the return process, the securing process, and the releasing process of the sub-record unit input / output buffer management processing unit 18 will be described with reference to FIGS. 16 and 17.
First, the operation of the return process will be described. If the buffer address has been set for all sub-record entries in the record management table 45 (step 822) after positioning to the first sub-record entry of the record management table 45 (step 820), the buffer is set to the input buffer of the buffer pool. (Step 824) is repeated (steps 826 and 828).
Next, the operation of the securing process will be described.
[0030]
A buffer for the length of the subrecord set in the subrecord entry of the record management table 45 is searched from the free buffer in the buffer pool (step 830). If the buffer is found from the free buffer (step 832), the buffer is in use. The buffer is set as the buffer address of the sub-record entry (step 834), the input flag and the output flag are cleared (step 836), and the process is terminated.
If it is not found from the free buffer (step 832), a buffer corresponding to the sub-record length set in the sub-record entry is searched from the input buffer of the buffer pool (step 838), and if it is found from the input buffer (step 840). ), Setting the buffer address (step 834) and clearing the flag (step 836) in the same manner, and the process is terminated.
If it is not found in the input buffer (step 840), a buffer shortage error is set (step 842), and the process is terminated.
Next, the operation of the release process will be described.
[0031]
Here, the buffer specified by the buffer address of the sub-record entry in the record management table 45 is set as an empty buffer in the buffer pool (step 850), the buffer address of the sub-record entry is cleared (step 852), and the processing is performed. finish.
Next, the operation of the input process of the sub-record unit parallel input / output means 19 will be described with reference to FIG.
First, it is positioned at the first subrecord entry of the record management table 45 (step 900), and it is checked whether the input request flag is set and the input completed flag is set for all the subrecord entries of the record management table, If it is not set (step 902), the storage address of the sub-record entry is converted into a physical address (step 904), and the physical record is requested to be input from the sub-record storage file to the set buffer (step 904). Step 906) The process is repeated (steps 908 and 910).
When all sub-record entries have been processed (step 908), check whether all requested sub-records have been input (step 912), and wait for completion if any input has not been completed (step 914) .
[0032]
The completion of sub-record input / output is reported by an input / output completion interrupt, the completed input / output request is examined (step 918), and the input / output completion is reported to the input / output request process (step 920).
When the input of all requested subrecords is completed (step 912), the input completion flag is set in the subrecord entries of all the requested subrecords (step 916), and the process is terminated.
Next, the operation of the output process of the sub-record unit parallel input / output means 19 will be described with reference to FIG.
First, it is positioned at the first sub-record entry of the record management table 45 (step 930), and when the output request flag is set for all the sub-record entries of the record management table and the output completed flag is not set ( (Step 932), the storage address of the sub-record entry is converted into a physical address (Step 934), and the physical record is requested to be output from the buffer set in the sub-record storage file (Step 936) (Step 938 and 940) is repeated.
[0033]
When all sub-record entries have been processed (step 938), check whether the output of all requested sub-records has been completed (step 942), and wait for completion if any output has not been completed (step 944). .
Completion of sub-record input / output is reported by an input / output completion interrupt in the same manner as an input request, the completed input / output request is examined (step 918), and input / output completion is reported to the input / output request process (step 920). . When the output of all subrecords requested to be output is completed (step 942), the output completion flag is set in the subrecord entries of all subrecords requested to be output (step 946), and the process is terminated.
According to the above embodiment, it is possible to perform a large number of data processing of long and large data at high speed and efficiently with the minimum necessary input / output processing using a limited input / output buffer area.
In addition, for data retrieval that requires a large amount of data to be read, the minimum input processing using a limited input / output buffer is possible without performing unnecessary input of data that is not related to the condition judgment in the record. Can be performed at high speed.
In addition, it is possible to update a part of data in a long record with a minimum of input / output processing using a limited input / output buffer without performing unnecessary input / output of data not related to the update in the record. It becomes possible to do.
[0034]
Furthermore, when the records are divided and stored in a plurality of files, the storage location management of each data is simplified, and data parallel input / output of a plurality of files and partial data input / output with an arbitrary file are possible.
Furthermore, it is possible to increase the storage space efficiency when records of variable length data format are divided and stored in a plurality of files.
Furthermore, when a variable length data format record is divided and stored in a plurality of files, the file refilling process can be performed at high speed.
Furthermore, in the record search command with the search condition, when the desired data cannot be directly searched by the key item or index, the sub record to be actually input is limited to only the part necessary for the search condition determination. The search command can be executed with a smaller number of input / output buffers than when all sub-records are always input in record units.
Also, if an input / output buffer for the record length can be prepared, the number of input / output buffer planes for inputting only the sub-records necessary for the search condition determination at once can be increased, so that the search command can be speeded up.
[0035]
In addition, since the amount of data input from the external storage device into the central processing unit is smaller than when all sub-records are always input, the search command becomes faster and the input / output channel occupation ratio can be reduced.
Furthermore, when updating records, the number of subrecords that are actually input / output is limited to only those that contain data that needs to be updated, so there is less input / output than when all subrecords are always input / output in record units. Update instructions can also be executed in the buffer. Also, since the amount of input / output data between the external storage device and the central processing unit is smaller than when all sub-records are always input / output, the update command becomes faster and the occupation rate of the input / output channel can be reduced. .
Furthermore, if all subrecord lengths that divide records are the same, and each subrecord is made into a fixed-length format file of the same capacity that can be independently input / output, the record storage address determined by the record sequence and key items is one. It can be made the same about all the subrecord realization values which comprise a record realization value. Therefore, parallel input / output of all subrecords and individual input / output of each subrecord are possible, and the storage address determination mechanism in each input / output can be unified and simplified.
[0036]
Furthermore, when the record format is variable length, the record is divided into a fixed length portion and a variable length portion, data up to a certain length is divided as a fixed length format portion with the same sub-record length, and data exceeding that is variable length portion As a sub-record is stored in the variable-length format file, the storage file for the variable-length portion only needs to have the capacity for the actual data amount of the variable-length portion. File capacity can be reduced. In addition, by having a storage address that stores the corresponding variable-length portion sub-record realization value in the final sub-record realization value of the fixed-length portion, between the fixed-length portion and variable-length portion subrecords that make up the record Can keep the connection.
Furthermore, if the file that stores the variable-length subrecord becomes full due to the addition or deletion of records, or if there are many invalid free spaces, enter only the last subrecord of the fixed-length portion. The corresponding variable length sub-record can be re-stored in another file. This makes it possible to speed up the file refilling process as compared to the case where all the subrecords are always input / output.
In addition, the said Example showed an example of this invention and this invention should not be limited to this.
[0037]
【The invention's effect】
As described above in detail, according to the present invention, a large number of data processing such as retrieval and updating of long data can be performed at high speed and efficiently with the minimum necessary input / output processing using a limited input / output buffer area. There is a remarkable effect that a data processing method can be realized.
[0038]
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a file management system according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating an example of a data storage method in the file management system according to the embodiment.
FIG. 3 is a diagram illustrating an example of a data format of a record management table in the file management system according to the embodiment.
FIG. 4 is a flowchart illustrating a processing procedure of a file definition information registration / reference processing unit according to the embodiment.
FIG. 5 is a flowchart illustrating a processing procedure of a record operation instruction execution processing unit according to the embodiment.
FIG. 6 is a flowchart illustrating a processing procedure of a record search processing unit according to the embodiment.
FIG. 7 is a flowchart illustrating a procedure of sub-record search processing according to the embodiment.
FIG. 8 is a flowchart illustrating a processing procedure of a sub-record selection processing unit according to the embodiment.
FIG. 9 is a flowchart (part 1) illustrating a processing procedure of a record update processing unit according to the embodiment.
FIG. 10 is a flowchart (No. 2) illustrating the processing procedure of the record update processing unit according to the embodiment.
FIG. 11 is a flowchart (No. 3) illustrating the processing procedure of the record update processing unit according to the embodiment.
FIG. 12 is a flowchart illustrating a processing procedure of a record deletion processing unit according to the embodiment.
FIG. 13 is a flowchart (part 1) illustrating a processing procedure of a record storage processing unit according to the embodiment.
FIG. 14 is a flowchart (part 2) illustrating the processing procedure of the record storage processing unit according to the embodiment.
FIG. 15 is a flowchart illustrating a search processing procedure of a sub-record unit input / output buffer management processing unit according to the embodiment;
FIG. 16 is a flowchart illustrating a procedure of return processing and release processing of the sub-record unit input / output buffer management processing unit according to the embodiment.
FIG. 17 is a flowchart illustrating a procedure of a securing process of a sub-record unit input / output buffer management processing unit according to the embodiment.
FIG. 18 is a flowchart illustrating a procedure of input processing of the sub-record unit parallel input / output processing unit according to the embodiment.
FIG. 19 is a flowchart illustrating a procedure of output processing of the sub-record unit parallel input / output processing unit according to the embodiment.
[Explanation of symbols]
1: Record operation command, 2: Data, 3: File definition parameter, 10: File management system, 11: File definition information registration / reference processing unit, 12: Record operation command execution processing unit, 13: Record search processing unit, 13a : Sub-record search processing, 14: record update processing section, 15: record deletion processing section, 16: record storage processing section, 17: sub-record selection processing section, 18: sub-record unit input / output buffer management processing section, 19: sub Record unit parallel input / output processing unit, 20-24: external storage device, 30: file definition information register, 45: record management table.

Claims (2)

And a memory for storing management information and a plurality of storage devices, a record that includes a variable portion of the fixed portion and the logical records of the logical record, the data processing for store and split into the plurality of storage devices In the data processing method in the apparatus,
The fixing portion of the logical record is divided into sub-records of at least one or more fixed length, said to store a plurality of physical continuous area of fixed length which is provided in the storage device, the variable portion of the logical record Is stored as a variable-length sub-record in a variable-length physical continuous area provided in the storage device,
Stores storage information for specifying a storage location in the storage device of the sub-records of the fixed length record management table stored in the memory,
Analyzes the search request in response to input of a search request, the records and the record management table specified by the record code operation instruction in the search request, a particular sub-record of prior SL designated record is stored Specify the storage device, refer to the record management table stored in the memory, acquire the storage information of the sub-record, acquire the specific sub-record from the specified storage device, and about the acquired sub-record A data processing method comprising performing a search process based on the search request. In a data processing device comprising a memory for storing management information and a plurality of storage devices, wherein a record including a fixed portion of a logical record and a variable portion of the logical record is divided and stored in the plurality of storage devices ,
The fixing portion of the logical record, and means for store at least one or more is divided into fixed length sub-records, the physical continuous area of a plurality of fixed length which is provided in front Symbol storage device,
Means for storing the variable part of the logical record as a variable- length sub-record in a variable-length physical continuous area provided in the storage device;
Means for storing the storage information for specifying a storage location in the storage device of the sub-records of the fixed length record management table stored in the memory,
Search analyzes the search request in response to an input request, the search from the record and the record management table specified in the record operating command in the request, prior Symbol the specific sub-record in the designated record is stored stored the device identifies the memory reference to the record management table stored obtains the sub record the specific sub-records stored information from said specified storage device acquisition of the search for the sub-records the acquired A data processing apparatus comprising: a search process based on a request.

Images