JPH06259303A - Decentralized file system - Google Patents

Abstract

PURPOSE:To provide the decentralized file system which shortens the time required for the transmission of a file to a decentralized processor and the synchronization of its contents and need not to pay attention to the decentralized arrangement of files. CONSTITUTION:The file used by the decentralized file system is composed of a virtual data set and when there is a request to open the file from decentralized sides 120 and 130, only a page table is sent from an original side 110. Then a necessary page is sent from the original side 110 each time the page is required on the decentralized sides 120 and 130. Further, when page data are updated on the decentralized sides 120 and 130, only the corresponding page data on the original side 110 are updated and the same page being referred to on other decentralized sides are placed in a page absent state with a command from the original side 110. Consequently, the frequency of data transfer between the original side 110 and decentralized sides 120 and 130 is decreased, so the performance of the decentralized file can be improved.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distributed file system, and more particularly to a distributed file system that makes files existing in a distributed manner on a network appear as if they were on a machine that is operating the files.

[0002]

2. Description of the Related Art Conventionally, NFS (Network File System) and AFS have been used as distributed file systems.
(Andrew file system, described in Nikkei Electronics 1990.8.20) etc. are known. The file system handles physical files stored in an external storage device such as a magnetic disk device. For example, in AFS, in order to realize a distributed file system, a distributed side processor is changed to an original side processor having a file entity. On the other hand, when there is a request for access to a certain file, the file is all transmitted to the distribution side, and a copy of the file is made on the distribution side. Then, when the file is updated in the distribution side processor, the file transfer is performed again between the original side processor and the distribution side processor to synchronize the contents of the distribution side copy file with the original file. There is.

[0003]

Therefore, in the above-mentioned conventional distributed file system, the larger the file size of the original processor accessed by the distributed processor, the larger the number of update requests from the distributed processor. Since the transmission traffic between the original-side processor and the distributed-side processor increases, the processing capacity of each processor is reduced by the capacity of the hardware of the transmission system.

Further, for example, when update requests from a plurality of distribution side processors are made to the same file at substantially the same time, the processing for synchronizing the contents of the copy file becomes complicated, which also causes There is a problem that the processing capacity of each processor is diminished.

In order to avoid the above-mentioned problems, in the conventional distributed file system, there has been a problem that a program must be created with a certain awareness of the distributed arrangement of files. .

Therefore, an object of the present invention is to solve the above-mentioned problems, shorten the time required for file transmission to the distribution side processor and the synchronization of the contents, and be fully aware that the files are distributed. It is to provide a distributed file system that does not require.

[0007]

In order to achieve the above object, according to claim 1 of the present invention, a file existing in an external storage device managed by one processor can be referred to and updated by another processor. In the distributed file system that can be used, a data section including a set of a plurality of pages obtained by redistributing a set of records, which is a logical input / output unit, into predetermined slot units, and a page table for searching the page are recorded. And a target record to be referred to or updated by a program according to a page absent bit in the page table arranged in a page fixed state in a virtual data set space secured in a virtual storage device. The target page is a virtual data set for paging between the virtual data set space and the external storage device. Ri, it is obtained by forming the file.

According to a second aspect of the present invention, a file to the original processor that manages the file is referred to or updated by another distributed processor connected to the original processor. When there is an open request, the virtual data set space is secured in the virtual storage device, the page table corresponding to the file is read from the external storage device, and the distribution side processor that has issued the file open request The original side file open / page table transmitting means for transmitting the page table read to the original side processor is provided, and when the file open request is made to the original side processor, the virtual memory device Within the virtual dataset And a distribution side file open / page table receiving means for allocating a space and arranging the page table transmitted from the original side processor in response to the file open request in the virtual data set space in a page fixed state, This is a configuration provided in the distribution side processor.

Further, according to a third aspect of the present invention, when a request for transferring a target page is issued to the original side processor from the distributing side processor to refer to or update the target page, it is necessary. After the paging of the page in the virtual data set space, the original side processor is provided with a target page transmitting means for transmitting the target page to the distribution side processor, and by the page absent bit in the page table. When the absence of a page of the target page including the target record that the program attempts to reference or update is confirmed, the target page transfer request is made to the original side processor, and the virtual data set is issued as necessary. After paging the page in space, before The target page receiving means for disposing said object pages transferred by the target page sending means to said virtual data set space, in which is configured to be provided in the dispersion-side processor.

Further, according to a fourth aspect of the present invention, when there is a request from the distribution side processor to forward the target page to the original side processor, the target page forwarding request is issued, and if necessary, the original page forwarding is performed. After paging the page in the virtual data set space, the target page transferred from the distribution side processor is arranged in the virtual data set space, and the distribution side processor making the target page forwarding request is excluded. , An update target page invalidation instruction to the other distributed processors that are referencing or updating the file to set the page fault bit corresponding to the updated target page to the page fault state Update target page invalidating means for transmitting the virtual data to the original processor. When the content of the target record included in the target page arranged in the output space is updated by a program, the target page forwarding request is issued to the original side processor to transfer the target page to the original side processor. The target page forwarding means for transmission is provided in the distribution side processor.

[0011]

The operation based on the above configuration will be described.

That is, in the configuration of the above-mentioned claim 1, since all the file formats used in the distributed file system are virtual data sets, the input / output to / from the file is the same as the input / output to / from the virtual storage device.
The read / write speed of a file from a program can be greatly improved to maximize the processing performance of the processor.

Further, according to the second aspect of the present invention, in response to the file open request from the distribution side processor, only the page table of the corresponding file is transmitted from the original side processor. Even if there is, it can be opened almost instantly, and the processing of distributed processors can be prevented from waiting due to the opening of a file, and the processing performance of the processor can be maximized.

According to the third aspect of the invention, the program of the distribution side processor issues a reference or update request to a certain record (target record) contained in the file, and the page containing the target record (target Page) does not exist in the virtual data set space of the distributed processor, the target page transfer request is made to the original side processor, so only the minimum required page data is transferred to the channel or line connecting each processor. Therefore, it is possible to suppress communication traffic and improve the use efficiency of communication resources.

Further, in the configuration of the above-mentioned claim 4, the page to be updated is transmitted by transmitting the update target page invalidation command to the third distribution side file which refers to or updates the same file. Since the page data itself is not transferred, the processing of the third distribution side processor is prevented from waiting and the processing performance of the processor is maximized. It is possible to suppress communication traffic and improve utilization efficiency of communication resources.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the distributed file system of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is a schematic block diagram showing an embodiment of the distributed file system of the present invention. In the figure, the physical device includes an original processor 110, a distributed processor (A) 120 that is channel-connected to the original processor 110, and a distributed processor (B) 130 that is line-connected to the original original processor 110, for example. And an input / output processor (IOP) that relays between the original processor 110 and an external input / output device
114 and the external storage device 11 connected to the IOP 114
It is composed of 5. And each processor
Instruction processors (IP) 111, 121, 131 for interpreting and executing programs, and program storages (PS) 112, 12 for storing the programs
2, 132 and data storages (DS) 113, 123, 133 for storing data handled by the program,
I have each. Further, as the external storage device 115, for example, a floppy disk device 115a, a magnetic disk device 115b, a magnetic tape device 115c, etc. are connected.

Further, the logical unit is a P in each processor.
Virtual storage device (VS) 1 for S112, 122, 132
1, 21 and 31 correspond to the DSs 113, 123 and 133, and extended virtual storage devices (XVS) 12, 22 and 32 correspond thereto. An address space is set in the VS 11, 21, 31 and the XVS 12, 22, 32 by an operating system (not shown).
A program is arranged in the address space of VS11, 21, 31. Further, in the address space of the XVS 12, 22, 32, a virtual data set space (VDS space) for paging the virtual data set (VDS) proposed by the present applicant in Japanese Patent Laid-Open No. 63-140348, etc. Is generated. It should be noted that there are a single virtual method and a multi-virtual method as a method of allocating a program to the address space in the VS, but the present invention can be applied to either method.

2 to 8 are diagrams showing respective processing flows in the distributed file system of the present invention, and FIGS. 2 and 3 show file opening processing flows in the original side processor and the distributed side processor, respectively. FIG. 5 shows a page data reference processing flow in the original side processor and the distribution side processor, FIGS. 6 and 7 show page data update processing flows in the original side processor and the distribution side processor, and FIG. 8 shows an update target in the distribution side processor. The processing flow of page invalidation is shown respectively. Hereinafter, FIG. 1 and the process flow will be described step by step in detail.

In the file open processing in the original processor of FIG. 2, the V received at step 201 is received.
Step 202: Check if the virtual data set corresponding to the DS file open request is already open.
Determine with. If the latter, in step 203, the VDS recorded on the magnetic disk device 115b, for example.
The page table (PTV) of the VDS space 1
After generating 2a, the process proceeds to step 205. In the case of the former, the process directly proceeds to step 205. Next, in step 205, it is determined whether the VDS open request is issued from the program on the own processor, that is, the original processor or from another processor, that is, the distributed processor. And in the former case, the file open process is completed.

If the latter, step 20
6. The page absent bit in the PTV (hereinafter, referred to as the original PTV or O.PTV) read into the DS 113 in step 6 (for each of a plurality of pages forming the VDS, whether the page is page-in in the VDS space). After generating a page absent state for all bits indicating whether or not), this is transmitted to the distributed processor that has issued the VDS open request in step 207 (the PTV transmitted to this distributed processor is Hereinafter referred to as dispersed PTV or C.PTV). The transmission in step 207 is repeated until the reception of the transmission completion code to be transmitted from the distribution side processor is confirmed in step 208 (the above corresponds to the file open / page table transmitting means).

In the file open processing in the distributed side processor of FIG. 3, a file open request of the virtual data set (VDS) is transmitted to the original side processor in step 301. Then, in step 302, it is determined whether or not the reception of the distributed PTV transmitted from the original processor is completed according to the description of FIG.
And in the latter case, it returns to step 301 again. If it is the former, the process proceeds to step 303. In step 303, the transmission completion code is transmitted to the original processor. Then, the distributed PTV transmitted in step 304 is read into the DS, and a VDS space is generated on XVS in step 305 (the above corresponds to the file open page table receiving means).

In the page data reference process in the original processor of FIG. 4, in response to the record reference request from the program for the file in which the VDS space is secured in the file open process of FIG. After making a request to read the page data including the record into the space, it is determined in step 402 whether the page data is in the VDS space. In the latter case, in step 403, the corresponding page data is paged in from the external storage device, for example, from the magnetic disk device to the VDS space, and then the process proceeds to step 404. If the former is the case, the process proceeds to step 404 as it is.

In step 404, it is determined whether the source of the read request is the program in its own processor, that is, the original processor, or the program in another processor, that is, the distributed processor. In the former case, the page data reference process is completed by reading the record requested to be referred from the page data in step 405 and passing it to the program in the original processor. If it is the latter, the page data is transmitted to the distribution side processor that issued the read request in step 406, and step 4
At 07, it is determined whether the transmission is completed. Then, if the transmission has not been completed yet, the process returns to step 406 again to transmit the next page data. When the transmission is completed, the page data reference process is completed (the above corresponds to the target page transmitting means). In the above processing, a record is an input / output unit from the program,
Since a page is an access unit of the VDS space, a record for which a reference request has been made may exist across multiple pages.

In the page data reference process in the distribution side processor of FIG. 5, VD is executed in the file open process of FIG.
In response to a record reference request from a program for a file in which S space is secured, first, step 50
After a request for reading page data including the record to the VDS space is issued in step 1, it is determined in step 502 whether the page data is in the VDS space. Then, in the latter case, in step 503, a request for the page is made to the original processor,
After the page request is continued at step 504 until the page is transmitted from the original processor, the page received at step 505 is arranged in the VDS space (the above corresponds to the target page receiving means). If it is the former, the process proceeds to step 506. In step 506, the requested record is read from the page and passed to the program. By the above, the page data reference process in the distribution side processor is completed.

In the page data update processing in the original processor of FIG. 6, in response to the record update request from the program for the file in which the VDS space is secured in the file open processing of FIG. After the request to write the page data including the record to the space is made, it is determined at step 602 whether the page data is in the VDS space. In the latter case, in step 603, the corresponding page data is paged in to the VDS space from the external storage device, for example, from the magnetic disk device, and then the process proceeds to step 604. If the former is the case, the process proceeds directly to step 604. In step 604, the VD is processed according to the record update processing from the request source program.
The page data including the record in the S space is updated. Then, in step 605, the updated page is paged out to the magnetic disk device. However, the timing of this page-out process may be either synchronous or asynchronous with respect to the record update.

In the following step 606, it is determined whether or not there is a distribution side processor to be synchronized, that is, whether or not there is a distribution side processor accessing the same file. In the latter case, the page data update process is completed. In the case of the former, an update target page invalidation command for setting the page absent bit corresponding to the page to the page absent state is issued to the distribution side processor in step 607, and this is issued in step 608 to PTV ( Repeat until the distributed PTV) updated code is received. PTV in step 608
If it is confirmed that the updated code is received, step 6
Returning to 06, it is determined whether or not there are other distributed processors to be synchronized, and all distributed processors that are using the same file are synchronized (the above is equivalent to the update target page invalidating means).

In the page data update processing in the distribution side processor of FIG. 7, the processing of steps 701 to 705 is the same as the processing of steps 501 to 505 in FIG. 5, and therefore its explanation is omitted. Then, in the following step 706, V is processed according to the record update processing from the request source program.
After updating the page data including the record in the DS space, in step 707 a target page forwarding request is issued to the original processor to cause the original processor to perform the page data update process shown in FIG. This target page forwarding request is repeated until the reply of the original side write processing end code is confirmed in step 708 (the above corresponds to the target page forwarding means).

In the update target page invalidation process in the distribution side processor of FIG. 8, the page absent bit ON transmitted from the original processor in step 607 of FIG. 6 is turned on.
The request is received by the distributing processor in step 801.
In response to this, in step 802, the page absent bit is turned on, that is, the page absent state is set. Then, the PTV updated code is transmitted to the original processor, and the update target page invalidation processing is completed.

9 to 15 are views showing the state of the distributed file system of the present invention during each of the above-mentioned processes, and FIGS. 9 and 10 show the states before and during the file open process, respectively. FIG. 12 shows a state during page data reference processing, FIGS. 13 and 14 show a state during page data update processing, and FIG. 15 shows a state during update target page invalidation processing. Hereinafter, FIGS. 9 to 15 will be described based on the above-described processing flows.

In the state of the distributed file system before the file open processing of FIG. 9, a virtual data set (original VDS or O.VD) existing in the external storage device 115 managed by the original processor 110 is used.
S) to the virtual memory device (V
S) program 21a in 21 and distribution side processor B
The file open request shown in step 301 of FIG. 3 is transmitted from the program 31a in the VS 31. When the file open process shown in FIGS. 2 and 3 is performed in response to this, the distributed file system is changed to that shown in FIG.
Transition to the state of.

In the state of the distributed file system during the file open processing of FIG. 10, the original side 11 is executed by the file open processing shown in FIGS.
0, distributed side A120, distributed side B130 data storage (DS) 113, 12 belonging to any processor
Page tables with the same contents in 3 and 133 (distributed PT
V) is generated and the extended virtual storage device (XV
S) Virtual data set spaces (VDS spaces) 12a, 22a, 32a are secured in the address spaces in 12, 22, 32. Therefore, the distribution side A120, the distribution side B13
From the programs 21a and 31a in the 0 processor,
As if it belongs to the original processor 110. V
The virtual data sets 115x and 115y (distributed VDS or C.VDS) that are the same as the DS are the self-processors 120 and 1
It will appear as if it belongs to 30 and exists. However, immediately after the opening, all the pages are in the paged-out state (that is, all the page absent bits in the PTV are in the ON state), and the VDS space 12a,
The data of 22a and 32a are all invalid.

In the state of the distributed file system during the page data reference processing of FIG. 11 (when the target page is not in the VDS space), the record "G '" requested to be referenced by the program 21a executed by the distributed processor A120 is stored. VDS space 2 on distribution side A containing page "G"
Since it is not present in 2a (the page to be referenced is paged out), the original processor 1
External storage device 1 connected to I / O 10 via IOP 114
The page "G" is read by the processing from 15 to 4 and 5. However, in the case of FIG. 11, the VDS space 1 on the original side is set because the file has just been opened.
There is no page “G” in 2a.

That is, the distribution side A makes a transmission request for the page "G" to the original side (step 50).
3), because it does not exist in the VDS space on the original side,
After the page-in operation of the page "G" is performed from the magnetic disk 115 (step 403), the page "G" is transmitted from the original side to the distribution side A (step 406). Then, the program on the distribution side A reads out the reference record “G ′” from the transmitted page “G” and passes it to the program (step 506).

Further, in the state of the distributed file system during the page data reference process of FIG. 12 (when the target page is in the VDS space), the VDS space 22 on the distribution side A is used.
Since the previously read page “G” already exists in “a”, the record “G ′” to be referred to from this is read and passed to the program.

Although not shown in the figure, in any of the above cases, the page absent bit corresponding to the page "G" in the VDS space is turned ON when the reading of the page "G" is completed (page The status is changed from (absent) to OFF (existing page).

In the state of the distributed file system during the page data update process of FIG. 13 (when the target page is in the VDS space), the page "G" including the record "G '" requested to be updated by the program 21a has already been distributed. Since it exists in the VDS space 22a on the side A, the VDS space 22a is updated by the process of FIG. 7 (step 706), and then the updated page “G” is transmitted to the original side (step 707). . In response to this, the original side updates the page "G" (steps 601 to 605) and invalidates or responds to the page that has been paged in to another distributed side processor, that is, distributed side B by the processing of FIG. A page-absence bit ON request (steps 606 to 608) is made.

Further, in the state of the distributed file system during the page data update process of FIG. 14 (when the target page is not in the VDS space), the page "G" including the record "G '" requested to be updated by the program 21a is Since it does not exist in the VDS space 22a on the distribution side A, the page "G" is received from the original side in the processing of FIG. 7 and placed in the VDS space 22a (steps 703 to 705),
The above processing is performed.

In the state of the distributed file system during the updating target page invalidation processing of FIG. 15, when the above-described page data update processing is performed on the distribution side A and the original side, the distribution side processor B130 causes the VDS space 32 to operate.
Since the page “G” before the update held in “a” is brought into the page-out state by turning on the corresponding page-absence bit, the program 31 on the distribution side B next.
When "a" needs the page "G", the page "G" is again processed from the original side by the processing of FIGS. 4 and 5.
To the distribution side B.

[0040]

As described above in detail, according to the invention described in claim 1, since all the file formats used in the distributed file system are virtual data sets,
The input and output to and from the file are all the same as the input and output to and from the virtual memory device, and the effect that the speed of reading and writing to and from the file from the program is greatly improved and the processing performance of the processor can be maximized is obtained.

According to the invention described in claim 2,
In response to a file open request from the distribution side processor, the original side processor sends only the page table of the corresponding file, so that the distribution side processor can open a file of any capacity almost instantly, It is possible to obtain the effect that the processing performance of the processor can be maximized by preventing the processing of the distributed processors from waiting due to the opening of the file.

According to the invention described in claim 3,
There is a reference and update request to a certain record (target record) contained in the file from the program of the distribution side processor, and the page (target page) containing the target record does not exist in the virtual data set space of the distributed processor. In this case, since the target page transfer request is sent to the original processor only, the minimum required page data will be transferred to the channel or line connecting each processor, suppressing communication traffic and reducing communication resources. The effect that the utilization efficiency can be improved is obtained.

According to the invention described in claim 4,
Transfers the page data itself by simply sending an invalidation page for the update target page to the third distribution side file that is referencing and updating the same file and setting the updated page to be in the page-absent state. Since it does not exist, the processing of the third distribution side processor is prevented from waiting and the processing performance of the processor is maximized, and communication traffic is suppressed and communication resource utilization efficiency is improved in the same manner as above. The effect that it can be obtained is obtained.

[Brief description of drawings]

FIG. 1 is a schematic configuration diagram showing an embodiment of a distributed file system of the present invention.

FIG. 2 is a diagram showing a file open processing flow in an original processor of the distributed file system of the present invention.

FIG. 3 is a diagram showing a file open processing flow in the distributed processors of the distributed file system of the present invention.

FIG. 4 is a diagram showing a processing flow of page data reference in an original processor of the distributed file system of the present invention.

FIG. 5 is a diagram showing a processing flow of page data reference in the distributed processors of the distributed file system of the present invention.

FIG. 6 is a diagram showing a processing flow of updating page data in the original processor of the distributed file system of the present invention.

FIG. 7 is a diagram showing a processing flow of page data update in the distributed processors of the distributed file system of the present invention.

FIG. 8 is a diagram showing a processing flow for invalidating update target pages in the distribution side processors of the distributed file system of the present invention.

FIG. 9 is a diagram showing a state of the distributed file system of the present invention before a file open process is performed.

FIG. 10 is a diagram showing a state of the distributed file system of the present invention during file open processing.

FIG. 11 is a diagram (part 1) showing the state of the distributed file system of the present invention during the page data reference process.

FIG. 12 is a diagram (No. 2) showing the state of the distributed file system of the present invention during the page data reference process.

FIG. 13 is a diagram (No. 1) showing the state of the distributed file system of the present invention during the page data update process.

FIG. 14 is a diagram (No. 2) showing the state of the distributed file system of the present invention during the page data update process.

FIG. 15 is a diagram showing a state of the distributed file system of the present invention during the update target page invalidation processing.

[Explanation of symbols]

110 Original Processor 120 Distributed Processor A 130 Distributed Processor B 111, 121, 131 Instruction Processor (IP) 112, 122, 132 Program Storage (P
S) 113, 123, 133 Data storage (DS) 114 Input / output processor (IOP) 115 External storage device 11, 21, 31 Virtual storage device (VS) 11a, 21a, 31a Program 12, 22, 32 Extended virtual storage device ( XVS) 12a, 22a, 32a Virtual data set space (VD
S space)

Claims (4)

[Claims] 1. A distributed file system in which a file existing in an external storage device managed by a certain processor can be referred to and updated by another processor, and a set of records, which is a logical input / output unit, is predetermined. A virtual data set space secured in a virtual storage device is provided, which comprises a data section composed of a set of a plurality of pages redistributed in slot units, and an index section in which a page table for searching the page is recorded. Virtual for paging a target page including a target record to be referenced or updated by a program between the virtual data set space and the external storage device according to a page absent bit in the page table arranged in a page fixed state. A distributed file system characterized in that the file is constructed by a data set. Beam. 2. When a file open request to refer to or update the file is made to the original side processor managing the file from another distributed side processor connected to the original side processor,
The virtual table space is secured in the virtual storage device, the page table corresponding to the file is read from the external storage device, and the page read to the distribution side processor that has made the file open request. The original side file open / page table transmitting means for transmitting a table is provided in the original side processor, and when the file open request is issued to the original side processor, the virtual data set space is stored in the virtual storage device. And the distribution side file open / page table receiving means for arranging the page table transmitted from the original side processor in response to the file open request in the virtual data set space in a page fixed state. Side The distributed file system according to claim 1, wherein the distributed file system is provided in a processor. 3. When an original page transfer request is issued to the original side processor from the distribution side processor to refer to or update the target page, the page of the page in the virtual data set space is transferred as necessary. After paging, the target page sending means for sending the target page to the distribution side processor, the original side processor, comprising the page absent bit in the page table,
When it is confirmed that the page of the target page including the target record to be referenced or updated by the program is absent, the target page transfer request is made to the original processor, and the virtual data set space is used as necessary. In the above configuration, the distribution side processor is provided with target page receiving means for arranging the target page transferred by the target page transmitting means in the virtual data set space after paging the page in The distributed file system according to claim 2. 4. The paging of the page in the virtual data set space, if necessary, when the original processor is requested to forward the target page by the distributed processor to the effect that the target page has been updated. After that, the target page transferred from the distribution side processor is arranged in the virtual data set space, and the file is referenced or updated except for the distribution side processor that has made the target page forwarding request. An update target page invalidating means for transmitting an update target page invalidation command to the effect that the page absent bit corresponding to the updated target page should be set to a page absent state to another distributed processor , The target provided in the processor on the original side and arranged in the virtual dataset space When the content of the target record included in the page is updated by a program, a target page forwarding means for sending the target page forwarding request to the original side processor and sending the target page to the original side processor, The distributed file system according to claim 3, wherein the distributed file system is provided in the distributed processor.