JPH07141232A - File storage management device - Google Patents

Abstract

PURPOSE:To decrease the frequency of disk access processing accompanying parity update processing. CONSTITUTION:A file is decomposed into several logical blocks LB0 to LB1, which are arranged on four data disk devices 201 to 203. A logical parity group is composed of four logical parity groups (e.g. LB0, LB1, LB2, and LB3) in total which are stored on the different disk devices 201 to 203 and those four logical blocks are exclusively ORed to generate a logical parity block (LP0 in the above case). The logical parity block is stored on a parity disk device 204. The logical blocks and logical parity block are related in a management table and the logical parity block is updated on a host memory when the logical blocks are updated. Therefore, the logical block groups which are divided into different stripes in the case of conventional technology are included in the same logical parity group.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a file storage management device for distributing files in a plurality of disk devices.

[0002]

2. Description of the Related Art As a conventional technique for managing files,
"Design and implementation of UNIX 4.3 BSD" (Maruzen, 199)
The technology described in (1 year publication) is known. This technique will be described below. FIG. 21 is an explanatory diagram of this technique. Reference numeral 1 is a host computer (hereinafter abbreviated as host), and 2 is a disk array subsystem. Reference numeral 3 denotes a predetermined disk interface (hereinafter referred to as disk I) that connects the host 1 and the disk array subsystem 2.
/ F), for example, SCSI (small compute
This is a signal line through which a signal according to the r system interface) flows, and will be referred to as a disk I / F below. An operating system 10 that manages disks in the host 1
3 (hereinafter simply referred to as OS. OS is CP of host 1)
U is executed by the main memory), and under this management, the application 101 (hereinafter simply referred to as AP. AP is executed by the CPU of the host 1 and the main memory) is operating. , So that the user can realize the desired calculation processing. The read / write request issued by the AP 101 to the disk device is processed by the file system 105 in the OS 103 for file management, and the device driver 106 issues a command to the disk device according to the command system of the disk I / F. Here, the file system 105 is a request processing unit 1053 that analyzes the disk access request of the AP 101,
A file index management unit 1054 that divides a file into logical blocks (this data segment is hereinafter referred to as a logical block), assigns a logical block number called an index to each block, and manages a file with this index, It is composed of a file index management table 1056, a buffer management unit 1055 for associating an index with the buffer 108b configured on the real memory and buffer management, and a buffer management table 1057. File system 105
The file index management unit 1054 and the buffer management unit 1057 of the AP are located in the user data area 108a.
When the data managed by 101 is stored in the disk device, it is determined which area on the disk device should be stored. Specifically, the disk device number, partition number, block (sector) number on the disk, memory address of the buffer area, etc. are determined. At the time of reading, read access is performed based on the various information thus determined. These pieces of information are registered in the management tables 1056 and 1057. The disk device receives a read / write command from the host 1 via the disk I / F 3, reads from the specified disk address, and writes. On the other hand, as a technique for improving the performance and reliability of a disk device, a RAID (Redundant Arra) composed of a plurality of disk devices developed at the University of California, Berkeley.
ys of Inexpensive Disks) is known, and this technology is based on the paper “A Cas published by the school.
e for Redundant Arrays of
Inexpensive Disks (RAID) "
In detail. The present technology will be briefly described below. FIG. 14 (a) is a diagram showing a data arrangement method of this conventional technique. In RAID, data is distributed in a plurality of disk devices and these disk devices are operated in parallel to realize high-speed transfer processing. Also,
As shown in the figure, disk addresses across different disk devices are grouped into blocks called stripes, exclusive OR of data of all data disk devices included in the same stripe is calculated, and redundant data (also called error correction code). , Redundant data also includes an error detection code), which is a kind of parity and is stored in the parity disk device. For example, file A is logical block L
It is assumed that the file B is composed of B0, LB1, LB2, and LB3, and the file B is composed of logical blocks LB10 and LB11. At that time, stripe 0 is composed of logical blocks LB0 and LB2 included in file A and LB10 and LB11 included in file B. The logical blocks LB0, LB2, LB10, and LB11 form one parity group. As a result, even if one of the disk devices fails and data cannot be read, the parity and the data in the other data disk devices included in the same parity group as the unreadable data are not read. It is possible to restore from, and realizes high reliability. The disk array subsystem of FIG.
It is a type of AID. Reference numeral 2 is a disk array subsystem, 210 is a part 210 for controlling a disk I / F used for connection with a host, 211 is a disk array control unit for performing processing such as data distribution and collection, parity calculation, etc.
Reference numerals 00 to 204 denote disk devices. The disk array subsystem 2 of the conventional example shown in FIG. 21 is designed so that it can be seen from the host 1 as one disk device. Since the host issues a read / write request assuming one disk device, the disk array control unit 211 converts this into a command for a plurality of disk devices and executes data read / write. Further, at the time of writing, it is necessary to update the above parity, and this processing is also executed.

[0003]

According to the above conventional technique, the file system of the host divides the file into logical blocks and manages them in block units.
When creating a new file, blocks are sequentially stored one by one. Therefore, when saving a file with a size of 1 block or more, these blocks are managed independently,
Since it is stored on the disk array subsystem, even consecutive blocks of the same file are arranged in a non-contiguous space of the address space on the disk array subsystem. Therefore, consecutive blocks are often included in different parity groups. Also, the disk array subsystem does not manage data in units of files, and simply stores blocks in an address space designated according to a block write request sent by the host. As a result, one parity group is
It is often composed of blocks belonging to a plurality of unrelated files. In this case, when writing one file, a plurality of blocks belonging to one file are stored in different parity groups. As a result, for example, in the worst case, when writing a plurality of blocks belonging to different parity groups, as in the case of writing the logical blocks LB1 and LB3 shown in FIG. 14A, one block is written. There is a problem that the parity update process of reading / writing the parity blocks P5 and P8 one by one occurs each time, and the speed of the file write process is significantly reduced. An object of the present invention is to provide a file storage management device in which the number of parity update processes is reduced as compared with the prior art.
Another object of the present invention is to provide a high-performance and highly reliable disk array subsystem having the file storage management device.

[0004]

In order to achieve the above object, the present invention divides data constituting the same file into a plurality of data segments, and one data segment constitutes one data block. In a file storage management device that distributes the plurality of data blocks to a plurality of external disk devices, the file storage management device includes a plurality of data blocks stored in different disk devices among the data blocks stored in the plurality of disk devices. Determining a redundant data group to be stored and a disk device that stores each of the plurality of data blocks.
Determining means, means for obtaining redundant data from data included in data blocks in the same redundant data group, and generating redundant data blocks composed of the redundant data for each redundant data group, the file, and the file. And a second storage unit for storing information on the correspondence relation between the data block and the redundant data block, and a second storage unit for storing information on the correspondence relation between the data block and the redundant data block. It is a thing.

Further, the disk array subsystem has the above file storage management device and a plurality of disk devices.

[0006]

For example, when n + 1 disk devices are connected to the host, n disks are used as data disk devices and one disk is used as a redundant data disk device. The means for determining the disk device for storing the data block determines the storage disk device for arranging the data block of the file. A group called a redundant data group is formed by n data blocks of the same file, and a redundant data block is generated by calculating, for example, exclusive OR of all the data blocks in the same redundant data group. In this way, the redundant data block is uniquely determined for each file. For each file, the data block and the corresponding redundant data block are registered in association with each other in a table that manages the relationship between the data block and the corresponding redundant data. Since there is often an opportunity to repeatedly update one redundant data while creating, adding, or changing a data block of the same file, if the above management device is adopted, the redundant data is converted into a data block in a file unit. The following effects can be obtained by managing them in association with each other. That is, according to this management device, while a file is open, it is necessary to update the difference data, which is the basis of redundant data generation, with the update of data belonging to the same redundant data group. Since the data belongs to one file, the corresponding difference data often exists in the buffer. As a result, most of the difference data is updated with the data update on the redundant data buffer, and the redundant data is read / written from the disk device each time the data is updated as in the conventional technique. As a result, the number of disk accesses required for updating redundant data can be greatly reduced.

[0007]

EXAMPLE A first example will be described. FIG. 1 shows the present invention.
It is a schematic diagram showing a file management system. File fil
e0 is composed of a data section and a parity section. Day
The data part is further divided into small blocks called logical blocks.
It In this example, the logical block is 8 KB and the file
The data part of is assumed to be 96 KB. File file
0 is LB0, LB1, ...
It can be divided into 12 logical blocks of LB11. Also,
Disk array subsystem 2 uses disk 0 (20
0), disk 1 (201), ..., disk 4 (2
It has a total of 5 discs of 04). Especially disk
4 (204) is called parity disk and stores parity
It is a dedicated disc. Above, the logical block is a disk
Array subsystem 2 disk 0 to disk 3 total
It is distributed to a group of disks called four data disks,
Is stored. In this example, LB0 is on disk 0 and LB1
To disk 1, LB2 to disk 2, and so on.
It is placed back and stored again. Where file top theory
Processing block to the number of data disks, that is, LB0
Logical parity with 4 logical blocks from LB to LB3
Group 4 LPG0, LB4 to LB7 four theories
Logical parity group LPG1, LB8 in the logical block?
To LB11 with 4 logical blocks.
Form a loop LPG2, and use byte units within each group
Read out, calculate exclusive OR for each bit, and
Generate a physical parity block. That is, LB0 + LB1 + LB2 + LB3 = LP0 (1) LB4 + LB5 + LB6 + LB7 = LP1 (2) LB8 + LB9 + LB10 + LB11 = LP2 (3) However, the symbol + here represents an exclusive OR.
Generated logical parity blocks LP0, LP1, LP2
Are stored on the parity disk or disk 4.
The data part and the parity part of the file file0 are
Each logical block that composes each logical parity block
And manage them in association with each other. If some logical block
When trying to rewrite the data of LBn, the related theory
The physical parity block LPk is also rewritten at the same time. above
A concrete example of implementing the file management method is shown below.
You FIG. 2 is a block diagram showing the system configuration of this embodiment.
is there. 1 is a host, 2 is a disk array subsystem,
3 is a disk interface (hereinafter abbreviated as I / F)
It ). Operating system on host 1
(Hereinafter abbreviated as OS) 103 operates and manages its
Below, the applications (101 and 102) (below
(Abbreviated as AP) is operating. OS103 is the following
It has each component. 104 is the data from the application
System call I / to receive disk access request etc.
F and 105 are files for managing storage of files on the disk.
File system, 106 is a disk I / F control unit 107
It is a device driver unit that controls the. File system
The system 105 has a built-in disk array management unit 1058,
This unit controls the disk array. 107 is a disk
A disk I / F control unit that controls the I / F 3. Di
The squaray subsystem 2 has five disks (200-
204), and these are disk I / F
3 are connected to the host 1 respectively. This disc
The array subsystem 2 is a file system as described above
Controlled by the disk array management unit 1058 in 105,
It is managed. The file inside the OS 103 of the host 1
The file management unit (file system) 105 operates as follows.
Disk devices 200 connected to the host with
~ 204 to decide where to store the data block
It That is, the file management unit 105 converts the file
Upon receiving the request to store the data, the unit 105
Associate the above file in 105 with the data block
Refer to the table, and in which data block the above data is
Determine if there is. If you have an existing data block
If there is, the difference data with the old data is calculated and stored in the buffer.
Store it. If it is a new data block,
Data to which disk device among multiple disk devices
The means for deciding whether to store the block is the storage disk device.
Position. And which address of the above disk device
The means for deciding whether to store a data block in space is above.
Refer to the table that manages the usage status of each disk unit
Then, the storage address is determined. In addition, the redundant data
The means for generating is the redundant data corresponding to the above data block.
Refer to the table that manages the relationship with the data block
And redundant data block corresponding to the above data block
If there is a corresponding redundant data block,
If not, which redundant data block
The method of deciding whether to store in the storage device is redundant data storage.
Determine the disk device for which of the above disk devices
Decide whether to store redundant data blocks in the address space
Means for managing the usage status of the disk device.
Table, determine the storage address, and
Management of the relationship between the
Register in the table. If the corresponding redundant data block
If the buffer exists in the buffer, the redundant data block
Calculate the exclusive OR with the lock data and put it on the buffer
save. If the corresponding redundant data block is a buffer
If it does not exist above, keep this data in the buffer.
The means for calculating existing redundant data is asynchronous after the above process.
The differential data on the buffer and the old redundant data on the disk unit.
Data with exclusive data and generate new redundant data
And write back to the disk device. 5 disk units
Connected to the store 1, but 4 units are for data disk
One is used as a redundant data disk device.
Determine the disk device to store the data block
Means file data block data disk
Decide the storage disk device so that it will be placed almost evenly on the device
To do. Redundancy due to 4 data blocks in the same file
A group called a data group is formed, and the same redundant data is
Exclusive-or of all data blocks in the data group
A redundant data block is generated by calculating. this
The redundant data block is uniquely determined for each file
To do. The update of redundant data is processed as described above. data
A table that manages the relationship between blocks and corresponding redundant data.
Table, each file has a
Register the long data block in association with it. Same file
While creating, adding, or changing a data block of
Generally has the opportunity to repeatedly update one piece of redundant data
Since there are many, if you use the above management method,
And manage redundant data by associating it with data blocks
There are the following effects. That is, this management method
According to the law, while a file is
With the update of data belonging to one redundant data group,
It is necessary to update the difference data that is the basis of redundant data generation.
However, since the data belongs to one file,
The corresponding difference data often exists on the buffer.
Become. As a result, the difference data is updated as the data is updated.
Is mostly done on the buffer for redundant data
As in the conventional technology, each time the data is updated, a redundant data
Without reading / writing data from the disk device
As a result, disk access due to redundant data update
The number of scans can be greatly reduced. The disc array is shown in Figure 3 below.
File system 103 that manages subsystem 2
The structure of each related part centering on is shown. File system
In 105, 1051 is an application file
File that performs file open processing when using
File open processing unit, 1052 is an open file
File index management table
File name management table that manages the relationship with the cable 1056.
Bull, 1053 is the file access from the application
A request processing unit for receiving a process request and performing an interpretation process, 1056
Splits the file into blocks and
Block number on the disk
File index to manage where it is stored
Management table, 1054 is file index management
Manages new registration, change, deletion, etc. of table 1056
File index manager, 1057
Buffer area where blocks are constructed in physical memory
Management information that is the management information for mapping to
Table, especially 1057a is a buffer area for data.
Data buffer management table for managing data area, 1
057b is for managing the buffer area for parity.
It is a parity buffer management table. Parity
This will be described later. 1055 is the buffer management table
1057 new registration, change, deletion, buffer allocation, etc.
The buffer management unit 1058 that manages the buffer
Disk array subsystem, disk, partition
And storage address on disk, etc.
The disk array that manages the disk array subsystem with
It is the management department. In the figure, 108 is in the main memory of the host 1.
This is the allocated buffer area. 108a is a user
User data area acquired for file access,
108b indicates that the file system 105 has activated the disk.
Data buffer area for data management during access, 1
In 08c, the file system 105 manages the parity.
Buffer area for parity, and 108d is a file system
Used temporarily by the system 105 when updating the parity
This is a work buffer area. Figure 5 shows disk array management
10 shows a block diagram of section 1058. 10581 is new
Rule block registration management unit, 10582 stores new blocks.
Machine to select the equipment (disk, partition) to be delivered
Device selection unit, 10583 is the physical device for the selected device.
Physical block mapping that determines the block address
The determination unit, 10584 determines the logical parity group,
The logical data block and logical parity block are related.
A logical parity management unit that performs the processing
Disk array usage status to manage the usage status of the disk array
It is a situation management table. Disk array usage management
The table 10585 is a disk that constitutes a disk array.
Management of disk physical block usage for each
Physical block management table 10585a and physical block
Management that manages the usage status of the sectors that make up the
It is composed of a table 10585b. 10586 is eliminated
A parity operation unit for performing other logical sum calculation, 10587
Uses parity data when one disk fails,
Degeneration processing that reproduces the data stored in the failed disk
It is a degeneration processing unit that executes the processing. Figure 6 shows a disk array
In the figure which shows the concrete example of the use situation management table 10585
is there. Physical block and sector usage for each disk
It is also configured so that the intended use can be determined. Less than,
The operation of this embodiment will be described. AP operates the file first
It is necessary to open the standing file. AP1
01 is the file name, open mode (read mode,
Read mode, read / write mode, additional mode, etc.)
Issue an open system call. Add mode and
Adds a file area to an existing file
It means a mode that indicates that it is a case of OS type
Some include in light mode. OS10
The system call I / F 104 of 3 receives this, and
If it is judged that it is a system call
The file open processing unit 1051 of the system is activated.
The section receives this, and from the file name, the file name management
The table 1052 is referred to and the file number k is changed. Also
However, in the write mode, this management table 1052
If the file is not registered, it is a new file.
I will newly register. By this process, AP can
Get the file number which is the operation number of the file. After that
Use this file number to access this file.
It FIG. 4 shows a configuration example of the file management table.
It 1052 is a file name management table and 1056 is a file name management table.
File index management table, 1057 is a buffer
The management table 108 is a buffer memory. Above
When the file number is obtained by open processing, this file is
File index corresponding to the file number as a key
The management table 1056 can be referred to. File index
The fax management table stores the contents as shown in the figure.
It is a table for each file. The outline of the contents is as follows
Is. The mode indicates the access mode of the file. Place
The owner is the user name who created the file, and the access identifier is
Indicates the access permission range of the file. Reference count
Indicates the number of references to the file. The time stamp is
The time when the file was last read or written, and the file
Shows the time when the index management table was last updated.
You The size indicates the size of the file in bytes. Bu
The number of locks represents the number of logical blocks used. Also
The buffer management table 105 corresponding to the logical block number
The pointer to 7 is stored. Buffer management table
There is one rule 1057 for each logical block.
The contents are listed. Buffer is valid for hash links
Link to a hash table to quickly determine if
Pointers, queue links for forming queues
The link pointer and flag are the buffer status
Buffer is used to determine if valid data is stored
The contents of the buffer have not been rewritten to the disk.
Indicates whether or not it is a projection. The device number is
And the partition number. Block number
No. is the disk address on the disk indicated by the device number
It is a number. The number of bytes is valid stored in this block.
The number of bytes of data. Buffer size is this buffer
Is the size in bytes of. Buffer pointer is physical
This is a pointer to the buffer memory. Logical parity glue
The group number is used when generating parity for each of the above files.
This is the number of the configured logical parity group. Logical paris
The pointer is the data buffer management table 1057.
Data logic that is valid when a and is stored in the buffer
The partition that stores the logical parity block corresponding to the block.
Pointer to the priority buffer management table 1057b
Is. Data buffer management table 1057a and
And the parity buffer management table 1057b are structural
Are the same. Data buffer management table 105
7a is from the file index management table 1056
Referring to the parity buffer management table 1057b,
From the corresponding data buffer management table 1057a
Only the points of reference are different. Things to reserve buffer
The physical memory consists of a data buffer and a parity buffer.
You can map to different areas or use the same area
It doesn't matter. Figure 3 shows a logical image
ing. Next, the file read / write operation is shown in Fig. 3 and Fig.
4 and the flow charts of FIGS. 7 to 12, FIGS.
1 showing the detailed configuration of the disk array management unit shown in FIG.
Using the schematic diagram of the data and parity update operation shown in 3.
explain. First, the read operation will be described with reference to FIG. Up
Open the file in read mode as described. That
After that, the AP issues a read system call (1100
1), the OS 103 system call I / F 104
And recognize that it is a lead system call,
The file system 105 is called (11002).
The request processing unit 1053 of the file system 105
Request (offset) issued by AP in units of
Convert to block units and sequentially apply the corresponding logical blocks.
Access (11003). Here, offset means
Amount of data to send (= transfer end address-transfer start address
Response), that is, the transfer length. First target logical block
The request processing unit 1053 sends a lock read request to the file
It is transmitted to the index management unit 1054. The department receives this
See the file index management table 1056.
Data block that stores the data of the target logical block.
A pointer to the buffer management table 1057a is obtained. One
Then, the buffer management unit 1055 receives this pointer,
Device of the disk device that stores the target logical block
Number, that is, disk number and partition number,
The storage disk block number is acquired (11005).
Here, the disk block number is the sector on the disk.
Is a logical address that corresponds to and is assigned linearly. It
In addition, this section refers to the buffer pointer and
Physical memory space is allocated to the target logical block
Determine whether or not. If the buffer is allocated
If the data of the target logical block is registered,
Flag on the buffer management table
However, if the "data valid" flag is "ON",
Read access to the disk is not required (cache hit
G) (11006). If the buffer is allocated
If not, the buffer management unit will newly
(11007). Then, in this case, and the above
When the "data valid" flag is "invalid", the device
The driver unit 106 is a disk I / F (for example, SCSI)
Read command is generated, and the disk I / F control unit 10
7 (11008). Here, the disk device
Transferred the data to the specified data buffer 108b
Until it does, the process stops, and the system enters a waiting state (1100
9). When the data transfer is completed,
Data buffer 108b
User data area 108a designated by AP (user)
Transfers the specified byte unit data (offset) to
(11010). The processing is performed again on the file system 105.
Request processing unit 1053 of all the AP specified
Judge whether the offset data has been transferred, and
However, if it is not completed, the above read
The process is repeated until the access is completed (11011). You
When all requested offsets have been transferred, the file system
The stem 105 receives A via the system call I / F 104.
Issue a system call end notification to P and return processing to AP
It Next, the operation during writing will be described. In FIG. 7, 1
The same is true up to 1003. 11004 when writing
Then, the process branches to the flowchart (A) of FIG. Phi
The index management unit 1054 receives this, and
Le index management table 1056
Data buffer management table for storing data
A pointer to the table 1057a is obtained. If applicable
Data buffer management table 1057a does not exist
When the target logical block is not stored on the disk
Meaning that it is a new logical block
Then, at the branch of (11102), the flowchart of FIG.
Branch to Details of this case will be described later. Also
The logical block already exists on the disk.
The corresponding data buffer management table,
The pointer is found. In this case, the buffer management
The unit 1055 receives this pointer and the target logical block
The device number of the stored disk device, that is, the disk
Disk number and partition number and storage disk block
Get a number. In addition, the part
Refer to the buffer or physical memory space
Determines if it is assigned to a lock. AP
The requested data write request is in byte units as described above.
Since it is specified by fusing, all of the applicable blocks
May not be rewritten. But this place
In this case, it is necessary to write in logical block units, so
Once the target logical block is read onto the data buffer
Then, update the necessary part of the logical block on the buffer,
Performs read-modify-write processing to write back to disk
It is necessary. The read modify write processing is as follows.
However, if the AP request is completely 1 logical block,
If you want to rewrite the data for
It is unnecessary. If a buffer is assigned to the logical block
If the data of the target logical block is registered,
Flag on the buffer management table
If the "data valid" flag is "ON", then
Old data of the target logical block exists in the data buffer.
Existing, no read access to the disk is required
(Cache hit) (11103). If the buffer
If is not assigned, the buffer management unit
Allocate the buffer (11104). Then, this place
And the "data valid" flag is "invalid"
Sometimes, the device driver unit 106 uses the disk I / F.
Generate a read command (eg SCSI) and
It is issued to the I / F control unit 107 (11105). here
Then, the data buffer 108b designated by the disk device
Processing is stopped until the end of data transfer to
(11106). Here is the old data lead date
Data transfer is completed. Then, it corresponds to the logical block
Refer to the buffer management table of the logical parity block
And the buffer memory for parity is mapped
It is determined whether or not (11107). If the buffer is
And the buffer contents are "valid"
If it is, proceed to FIG. Otherwise
If (buffer memory mapping is not completed),
Topping is performed (11108), and the process proceeds to (F). Just
Now we are in the process of updating the parity. Details of parity update processing
The details will be described with reference to FIG. In the case of Figure 9 (E),
Work buffer 108d0 is replaced with work buffer 108d0
Secured in the area (11109), and the area on the user area 108a
The data (new data) and the data buffer 108b0
Exclusive OR (EOR) with the above data (old data)
Calculate and generate difference data of both data (1111)
0). Here, the new data in the user area is
Buffer and update the buffer contents (1111
1). Then, the difference data on the work buffer 108d0
Data and exclusive OR (EOR) on the parity buffer
Is calculated, and the result is stored in the parity buffer 108c0.
Pay (11112). In addition, the parity buffer in the figure
The redundant data buffer is also named as the above calculation.
Due to this, new parity may not be generated in some cases.
Is. Here, the generated data is
Data, the new difference data of the old data, and the previous logical block
Exclusive theory with old difference data generated when writing
It is Riwa, and only the difference information of parity is calculated.
Yes. Therefore, eventually, the old files stored on the disk will
Calculates the exclusive OR of the difference information between the parity and this parity
Need to In this case, the parity buffer tube
"Old parity read completed" flag in the processing table 1057b.
Is off. If this flag is "O"
In the case of N ", the above processing is performed on the parity buffer.
At the end, new parity has been generated. Next,
The data buffer management table 1057a and Paris
"Buffer on Tee buffer management table 1057b
Set content "flag" to "dirty" (11113)
~ 11114). This "dirty" flag is a buffer
Indicates that the above data has not been reflected on the disc. I
Write back on the disc. This process is shown in FIG.
Is shown in the flowchart and will be described later. All AP request
Judge whether or not the block write of data is completed (1
1115), if not yet, return to FIG. Next
The case of FIG. 9F will be described. The difference from (E) is Paris
The parity buffer 108c0 has parity difference information
Means that the old parity is not stored. Therefore,
The difference information between the old and new data is generated in the same way as above, and the parity
The data is stored in the data buffer 108c0 (11116). Paris
The work buffer is unnecessary because the tea buffer is empty.
It In this case, the old parity read has not been performed yet.
Then, set the "old parity read completed" flag to "OFF".
It is set (11117). And on the user area
The data (new data) is stored in the data buffer area 1
08b and updates the buffer contents (1111
8). The same applies to the case of the above (E). Then a new
The case of writing the lock (D) will be described with reference to FIG.
It Logical parity group corresponding to the logical block
It is determined whether or not exists (11120). This format
The default is the file index management table 1056.
The same logical block as the logical block
You just need to determine if the block is already registered.
The determination method is the file index management table 10
56 has a pointer to the buffer management table 1057.
It can be judged by whether or not. If applicable logical parity
If there is no block, data buffer management table 1
057a and parity buffer management table 1057
Both of b are newly created and registered (111121-111)
22). Here, the buffs performed at 11121 and 11122
New allocation of disk management table 1057 and disk physical block
Please refer to the flowchart in Figure 11 for the lock mapping
It will be explained. First, the disk array management unit 1058
The new logical block registration management unit 10581 (FIG. 5) of
Receives a new block allocation request from the buffer management unit 1055.
And whether it is a block for parity or a block for data
(11301) to newly create the buffer management table 1
057 is created (11302). Next, the device selection unit 10
582 is a data block, a parity block, or
Also, based on information such as the logical block number,
Disk and partition to store the
Register in the buffer management table 1057 (1030
3, 10304) Next, physical block mapping decision
The part 10583 determines which physical block on the disk is concerned.
Disk array usage management
See the physical block management table in table 10585.
The storage location of other logical blocks in the same file.
Optimum to minimize seek and rotation waiting based on information
Select the appropriate block. Also, the management table 1058
Update 5. From the selection result of this physical block,
Block management block or sector number
Register in the block number field in Bull 1057
(11305). Then follow the logical parity group
Logical parity group
Is registered (11306). If it is a data block
If this is the case, manage the parity buffer corresponding to the data.
Pointer to table 1057b is a data buffer tube
Registered in the processing table 1057a, and if the parity block
If it is a check, the parity buffer management table 1
Register a NULL pointer in this field of 057b
(10308). Finally, the buffer management table 1
057 corresponding file index management table 1
Register in 056, allocate new buffer management table and
The logical block physical block mapping process is complete.
It Returning to FIG. 10, the newly registered data buffer tube
The buffer memory to the physical table 1057a.
(11123). If the parity buffer management table
Bull 1057b is also a buffer if newly created
Map memory. After that, the process at the time of the above update write
The difference data between the old and new data is calculated, and the parity
Calculate exclusive OR with the old difference data of the buffer area,
Store in the parity buffer area. If the old difference data
If there is not, the new difference data is used as is in the parity buffer area.
Stored in the area and set the "old parity read completed" flag to "OF".
Set to F ". Finally, set the new data in the user area.
Transfer to the data buffer area and return to FIG.
It During the above write processing, the data buffer and
When new information is stored in the
Set the "buffer contents" flag of the management table to "dirt"
y ”was set and the process was completed.
Old data when a write request to the buffer occurs.
This is because there is a possibility that the write processing can be reduced. this
Is called delayed writing, and at this time, appropriate timing
It is necessary to write back on the disk. buffer
This behavior is achieved by synchronizing the contents with the disc contents.
This is called sync operation. The flow chart of this sync operation
It shows in FIG. AP process called Sync Daemon is fixed
Issue sync system call periodically (1120
1). The OS receives this, and the file system buffer
The server management unit 1055 searches the list and
Look for a buffer whose "flag is" dirty "(112
03), if the searched buffer is data,
The contents of the data buffer management table
Write to a new disk (11204-11206), and
Later set the "buffer content" flag to "clean"
(11207). If it's a parity buffer
If you see "Old parity read completed" flag,
If it is "OFF" (11208), work buffer
Of the old parity is set in the buffer management table.
Lead from the disk to the work area. This old parity and
Exclusive of differential parity data on the parity buffer area
Calculates the logical sum and stores it in the buffer for parity
It Write this complete new parity to disk
And set the "old parity read completed" flag to "ON".
Also, set the "buffer contents" flag to "clean".
Set. If the "old parity read completed" flag
If it is "ON", the disc is immediately
Write to, and likewise, set the "buffer contents" flag to "cl".
Set to "ean". The above processing is performed for all "dirt".
This is done for the y "buffer.
According to this, the method of managing parity for each file in Fig. 1 is
realizable. Next, the effect of this embodiment will be described. In the conventional example
According to this, the file system 105
Determining the best disk address at that time
Therefore, as shown in FIG. 14a on the disk array subsystem.
In many cases, the block arrangement is intermittent. Optimal
Such decisions include, for example,
If there is a head, select an empty head at that time,
For example, select a sector with a low rotation waiting time. LB0
When LB3 is sequentially stored from the
Update the corresponding three parity blocks P1, P5, P8
Must be a data block and a parity block,
A total of 7 read-modify-write processes for 7 in total
It is necessary to carry out the reason. On the other hand, in the method of this embodiment,
14b, consecutive logical blocks LB0 to LB
The corresponding logical parity of 3 is uniquely determined to be LP0. Delayed letter
When the imprinting is performed as in the above embodiment, the LP0
Since the update can be executed only on the buffer, the parity update
Only one read-modify-write process required for processing
Including data update, data block and parity
5 blocks, total 5 times lead mod.
A write process will be performed. In this case, the processing effect
The rate is 1.4 times higher. Generalize the above effects
Then, a disk composed of (n + 1) disks
Write k logical blocks in array subsystem
The maximum number of parity updates in this case is (1) conventional k times (2) this embodiment k is a multiple of n (k / n) times k is not a multiple of n ((k / n) +1) times Becomes The processing efficiency including data update is about (2 / (1
+ (1 / n))) times. A second embodiment will be described.
It In the first embodiment, the number of logical blocks in the file is
I explained that it is an integer multiple of the number of data disks
However, the number of logical blocks in the file is actually an integer.
is there. For example, the file 0 (file0) shown in FIG.
It is composed of 10 logical blocks in total. data
If there are 4 disks, 8 logical blocks LB0 to LB7
The lock has two complete logical parity groups LPG0,
LPG1 can be configured, but two logics of LB8 and LB9
Block cannot form a complete logical parity group
Yes. Such a logical block is called a fragment logical block.
I will call it Ku. L of file 1 (file1)
The same applies to B4 and LB0 of file 2 (file2).
Is a fragment logic block. Flags like this
The following is an example of how to handle the ment logic block. This method
This is a complete extension of the first embodiment, and the flag in each file is
Segmented logical block only to form a logical parity group.
It is a way to make it. This method is similar to the first embodiment
Can be realized. This method is a file that is frequently added.
File already corresponds to fragment logical block
Added because logical parity block is assigned
The logical parity group composed of
Lagment logic block or complete
It is easy to configure without the need to recognize
There is In addition, this method requires only one logical block.
Logical blocks for small files made up of
Two blocks, a logical block and a logical parity block.
However, the contents of both are the same. like this
Then, when reading this file, which block should be
You may lead. That is, the data not currently in use
Disk, or both disks will be used.
When not in use, seek distance
Select a disc and lead. Such a small
In a system with many files,
By performing the above control, the speed-up effect is great. Third implementation
Here is an example: The second embodiment is very effective in speeding up small files.
While it has the effect of being easy and easy to manage,
File 2 consists of only one logical block
One logical parite for small files
Blocks must be allocated, and the disk capacity
There was a downside. Therefore, the method which improved this shortcoming
It shows in FIG. Fragment of file 0 (file0)
Logical blocks LB8 and LB9, and file 1 (file
e1) fragment logical block LB4 and file
2 (file2) fragment logical block LB0
Of the temporary logic by a total of four fragment logic blocks
Parity group VLPG0 is configured and these logical blocks
Block exclusive OR, and fragment logic block
Parity block (fragment parity block)
C) Generate FP0. A tentative logical parity group
All the fragment logical blocks contained in the
The data disk to be stored on the
Toparity block is different from the above data disks
Place it on the parity disk. Logical block of file
Flag in the file when storing
Need to check for the existence of
is there. Therefore, the file of FIG. 4 shown in the first embodiment.
Buffer management table 105 that is part of the management table
"Fragment logical block" flag in the flag area of 7
And the relevant logical block is a fragment logical block
Whether or not is recognizable. Also logical Paris
Buffer management table for parity that manages the T-block
Similarly, the "fragment logical block" is also applied to the rule 1057b.
A flag is set so that the logical parity block
It is possible to recognize whether it is a top parity block.
Ku. This fragment parity block contains multiple files.
Buffer management for fragment fragment logical block data
It will be referred to from the table 1057a. Existing
When adding a new logical block to the file,
First, the file index management table 1056 of FIG.
Management table 105 corresponding to each logical block
Referring to the pointer to 11, the new logical block number
If it is not (an integer multiple of the number of data disks-1),
The logical block becomes a fragment logical block. Yo
The "fragment logical block" flag to "ON"
If the fragment logical block is already in the same file,
If there is a link in the temporary logical parity group,
Incorporate a new fragment logical block. The provisional theory
Physical parity group is full or the same file
If there is no fragment logical block in the
Configure a logical parity group. If a new logic block
Check number matches (an integer multiple of the number of data disks n-1)
, The complete logical parity group inside the file.
Since loops can be configured, the n internal files
Calculate the exclusive OR of the Lagment logic block and
Configure a logical parity block on the
Store. At this time, all fragment logical blocks
To be stored on different data disks
Therefore, without moving the data logic block, the logic
Only the parity block needs to be newly generated. like this
In the process of growing the file, fragments
Although a logical block occurs, as shown in the first embodiment above.
The parity update processing is performed on the host main memory.
No, it doesn't write to the disc immediately,
The overhead time for updating the logical parity block is
Usually very small. As described above, according to this embodiment,
The number of file logical blocks is an integer multiple of the number of data disks
Parite saves parity disk space even when not
Management can be realized and its performance is almost the same as that of the first embodiment.
Etc. can be realized. Next, a fourth embodiment will be described. First above
In the third to third embodiments, the file management table (version
Logical block and logical parity
It was configured to maintain the correspondence of the two blocks. Book
In the embodiment, a second management method different from the above method is shown in FIG.
Show. 21052 is a file name management table, 2105
2a is a data file index for managing data files.
Pointer to the management table, 21052b is the data
Logical parity block corresponding to the logical block of the
Pack to manage the disk as one parity file.
Point to index file index management table
21056a manages the logical blocks of the data file.
Data file index management table for management, 21
056b is a parity flag for managing the logical parity block.
File index management table, 21057a and
21057b is for each logical block and logical parity block.
Buffer management table to manage the buffer corresponding to
Table. Data file index management table
21056a and parity file index tube
Process table 21056b is the file index of FIG.
It is similar to the management table 1056. Also a buffer tube
Process table 21057 is the buffer management table 1 of FIG.
Same as 057 except logical parity pointer
is there. The method of this embodiment uses one file as a data file.
File and parity file divided into two files
It makes sense. This will be described with reference to FIG. File name management
Table 21052 has two pointer fields, data
Data file index pointer 21052a and
Has a parity file index pointer 21052b
To do. file0 is a data file file0d and
Parity file file0p is divided into two parts,
Manage as an independent file. However, both of these
For the file, see the file name management table 21052 above.
Relate. Logical block and logical parity block, logical
The relation and management method of the parity group is the first embodiment described above.
Similar to the example. According to the above method, the same as the first embodiment
Similar effects can be obtained. Also, if the user needs
Depending on the file, for reasons such as work area
Does not add parity by selection when reliability is not required.
It is also possible to use this disk array subsystem
It is possible to provide reliability that matches the demands of other persons. Example 5
explain. The above embodiments are all OS 10 on the host 1.
With the method of controlling the disk array subsystem by 3
There was an application like a database management system.
In the solution, the
It is necessary to optimize the control of the
Of the disk system inside the application 101
It is necessary to control. FIG. 18 shows this example.
101 is an application, 1058 is a disk array
Disk array management unit that controls and manages subsystem 2
Is. The internal structure of 1058 is the same as that of the first to fourth embodiments.
It is like. This application is a file that the OS has
File management system, that is, without going through the file system,
-Directly disk array called device I / O
A method of controlling the subsystem is used. Application
O when a device issues a raw device I / O system call
S's system call I / F receives this and device dry
Disk I / F required by the application
Issue a disk command to the controller. OS like this
Will simply command the disk to receive the request of the application
Only a simple process of issuing a command is performed. Which disc
Whether to allocate a logical block or logical parity block to
It is the disk array pipe inside the application that determines
It becomes the role of the science department. The configuration and operation of this section are the same as those of the first embodiment
Similar to the example. According to the above embodiments, the disk array
Subsystem matched to application usage
It is possible to control in the form of high performance and high reliability.
The effect is greater. Next, a sixth embodiment will be described. Book
In the embodiment, as shown in FIG. 19, the disk array subsystem
The system 2 has a disk array management unit 1058.
Of. The disk array controller 211 communicates with the host 1.
File management information can be transferred to the disk array
It is managed inside the control unit 211. This place
In this case, the file is managed as a logical block as in the first embodiment.
Logical parity block corresponding to the logical block
to manage. Configuration and operation of disk array management unit 1058
The work is the same as in the first embodiment. According to this embodiment,
Logical blocks and theory within the Clay subsystem
Since it is possible to realize the optimal placement of the physical parity block,
The effect of higher performance is greater than that of technology. Next, the seventh embodiment
Will be explained. The first to sixth embodiments are easily networked.
Disk array system and distributed file system
Can be extended to. Figure 20 (A) shows an example.
Is shown. 5 is a network, 60-64 are
Hosts 0-4, 7 which are computers holding disks
0 is a disk, the user executes the program, host 0
A computer that issues disk access requests to
Clients, 80-83 are data disks, 84 is a disk
It is a disk for ritivity. Each host 0-4 has one device
It holds the disks 80-84. For example client
Reference numeral 70 denotes the file management method as in the first embodiment.
Suppose However, the client 70 does not
Since it does not have a disk unit for data
Disk access to hosts 0 to 4 via the network
Issue a process request. However, the client 70
The file management method can be managed in the same manner as in the first embodiment.
it can. However, in this case, the file management table shown in FIG.
Device number information of the buffer management table 1057 in the buffer
It is necessary to add the device identification number such as the host address to
is there. If you have a lot of clients,
Many disk access requests are issued to 0-4.
However, the hosts 0 to 4 are
Freely optimize and decide the storage address of the logical block
You can In addition, as another example, it is shown in FIG.
It is conceivable that the configuration is made. Host 0 is a data disk only
Is managing. Host 1 manages parity disk only
It makes sense. Client 70 sends data to host 0
Issue a disk access request to Paris to host 1.
Issue a disk access request. Of the above (A)
The difference from the example is that host 0 manages all data disks.
And host 1 manages the parity disk.
is there. Data and parity can be handled asynchronously
Each host has its own execution order and logical blocks.
Storage address can be freely optimized and determined
It Some examples have been described above.
One of the (n + 1) disks is a parity disk
However, the logical parity block is (n +
1) It is possible to disperse and arrange all the disks.
is there. Even in this case, in order to speed up by parallel processing
All logical blocks and logical partitions in the logical parity group are
Store all utility blocks on different disk units.
It is the same as in the above embodiment that it is necessary to do so. Each logic block
And the logical parity block is arbitrary on each disk.
It can be located at the address. Also, the above
All of the examples are for an array system using a magnetic disk device.
I explained about it, but instead of the magnetic disk unit, an optical disk
Disk device, magnetic tape device, or semiconductor memory device.
An array system that realizes file management like
It is also possible. In addition, a disk using a magnetic disk
Only the parity disk in the square array system
Or using a magnetic tape device or a semiconductor memory device
Can be replaced by a storage device that combines these
Is. As described above, the system of the present invention can be applied to various system configurations.
A file storage management method can be used.

[0008]

As described above, according to the present invention, since the parity is generated for each file, the parity can be localized and handled, and when the same file is repeatedly updated or added, the parity can be changed. Since most of the update processing can be performed in the buffer area on the memory of the host, there is an effect that the number of disk accesses associated with parity update can be significantly reduced and file processing can be speeded up. Further, since the storage positions of both the data and the parity can be determined on the host side, there is an effect that optimization of logical block arrangement on the disk can be realized and high-speed read / write processing with a short access time can be realized. Further, since the host side controls all of the disk array, no special circuit for controlling the disk array is required. Specifically, in FIG. 21, there are two disk I / F control units 107 and 210, but in FIG. 2, only one disk control unit 107 is required. This has the effect of building a low-cost disk array subsystem.
Further, there is an effect that it is possible to reduce an increase in processing time that occurs when a special circuit is added. In addition, since the file storage management method of the present invention can be installed inside the application, it is possible to realize optimum file storage management that matches the disk access characteristics of the application, and improve performance.
High reliability can be realized. Further, since the logical block and the logical parity block can be arranged at any place on the network, there is an effect that a highly reliable and high performance distributed file system can be easily constructed. It is also possible to put only parity in a device other than a disk device, and build a high cost performance file system by selecting a high performance disk for data and a low performance but inexpensive device for parity. It is possible. Further, since the user can decide whether or not to add the parity for each file, it is possible to provide the reliability that matches the user's request.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing correspondence between logical blocks and parity blocks.

FIG. 2 is a system configuration diagram of the first embodiment.

FIG. 3 is a block diagram of a part that realizes file management of the present invention.

FIG. 4 is an explanatory diagram of a relationship between file management tables.

FIG. 5 is a block diagram of a disk array management unit.

FIG. 6 is an explanatory diagram showing a configuration example of a disk array usage status management table.

FIG. 7 is a flowchart of disk read processing.

FIG. 8 is a flowchart (1) of disk write processing.

FIG. 9 is a flowchart (2) of disk write processing.

FIG. 10 is a flowchart (3) of disk write processing.

FIG. 11 is a flowchart of a buffer management table allocation process and a physical block mapping process.

FIG. 12 is a flowchart of disk sync processing.

FIG. 13 is a schematic diagram of update processing of a data logical block and a logical parity block.

FIG. 14 is an explanatory diagram showing a relationship between a logical block of a file and a parity by comparing the related art with the present invention.

FIG. 15 is an explanatory diagram showing a relationship between a logical block and a logical parity of a file according to the third embodiment.

FIG. 16 is an explanatory diagram showing an example of a file management table of the fourth example.

FIG. 17 is an explanatory diagram showing a relationship between a file name management table, a data file, and a parity file according to the fourth embodiment.

FIG. 18 is a block diagram showing the system configuration of a fifth embodiment.

FIG. 19 is a block diagram showing the system configuration of a sixth embodiment.

FIG. 20 is a block diagram showing the system configuration of a seventh embodiment.

FIG. 21 is a block diagram showing a system configuration centering on a file management unit of a conventional example.

[Explanation of symbols]

 1054 File index management unit 1055 Buffer management unit 1056 File index management table 1057 Buffer management table

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Takashi Oeda Takashi Oeda 292 Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa Prefecture Hitachi Ltd. Microelectronics Device Development Laboratory (72) Inventor Hiroaki Takahashi Totsuka-ku, Yokohama-shi, Kanagawa 292 Yoshida-cho, Hitachi, Ltd. Microelectronics Device Development Laboratory (72) Inventor Hitoshi Akiyama 292, Yoshida-cho, Totsuka-ku, Yokohama, Kanagawa Prefecture Hitachi, Ltd. Microelectronics Device Development Laboratory

Claims (17)

[Claims] 1. Data constituting the same file is divided into a plurality of data segments, one data block constitutes one data block, and the plurality of data blocks are distributed to a plurality of external disk devices. A file storage management device for determining a redundant data group composed of a plurality of data blocks stored in different disk devices among the data blocks stored in the plurality of disk devices, Determining means for determining the disk device storing each of the data and redundant data is obtained from the data contained in the data blocks in the same redundant data group, and a redundant data block composed of the redundant data is generated for each redundant data group. Means, the above files, and the above which constitutes the above files Data storage management comprising: first storage means for storing information on the correspondence relationship with the data block; and second storage means for storing information on the correspondence relationship between the data block and the redundant data block. apparatus. 2. The file storage management device according to claim 1, wherein information about the data block and the redundant data block regarding the same file is stored for each file. Management device. 3. The file storage management device according to claim 1, wherein the redundant data group has n data blocks, and when the number m of data blocks forming the file is not a multiple of n, By the surplus data block obtained by dividing m by n, and the data blocks forming other files,
A file storage management device comprising a redundant data group composed of n data blocks. 4. The file storage management device according to claim 1, wherein the redundant data group has n data blocks, and when the number m of data blocks forming the file is not a multiple of n, A file storage management device characterized in that one redundant data group is constituted by a surplus data block obtained by dividing m by n. 5. The file storage management device according to claim 4, wherein when the number of the surplus data blocks is one, the one data block constitutes one redundant data group, The file storage management device, wherein the redundant data block corresponding to the redundant data group is composed of the same data as the data block. 6. The file storage management device according to claim 1, wherein the second storage unit has information regarding the data block for each of the data blocks, and stores the information regarding the data block. A file storage management device, characterized in that it has, as a part, information regarding a correspondence relationship between the data block and the redundant data block. 7. The file storage management device according to claim 1, wherein one independent file is configured as a redundant data file in the entire plurality of redundant data blocks, and the second storage means is provided. A file storage management device having information for accessing the redundant data file. 8. The file storage management device according to claim 1, further comprising means for storing information on whether to generate redundant data for each file. apparatus. 9. The file storage management device according to claim 1, wherein the file storage management device is a file management system included in an operating system of a computer. 10. The file storage management device according to any one of claims 1 to 9, wherein the file storage management device is included in a redundant data group from which the redundant data is generated, out of the plurality of disk devices. There is a determining means for determining a disk device in which no data block is stored as a redundant data disk device for storing the generated redundant data block, and the determining means is one disk for each redundant data block. A file storage management device characterized in that there are a plurality of disk devices for which a device is determined and which is subject to the above determination. 11. The file storage management device according to any one of claims 1 to 9, wherein the file storage management device is included in a redundant data group from which the redundant data is generated, out of the plurality of disk devices. There is a determining means for determining a disk device in which no data block is stored as a redundant data disk device for storing the generated redundant data block, and the determining means is one disk for each redundant data block. A file storage management device, characterized in that a disk device for which a device is determined and which is a target of the above determination is a disk device for storing redundant data blocks exclusively. 12. A disk array type file system, comprising: a computer having the file storage management device according to claim 1; and a disk array device having a plurality of disk devices. 13. A disk array subsystem comprising the file storage management device according to claim 1 and a plurality of disk devices. 14. A computer system comprising a computer which is connected to a network and which has the file storage management device according to any one of claims 1 to 11, and a plurality of disk devices which are connected to the network. 15. The file storage management device according to claim 1, wherein the redundant data disk device to be determined is an optical disk device. 16. The file storage management device according to claim 1, wherein the redundant data disk device to be determined is a semiconductor storage device. 17. The file storage management device according to any one of claims 1 to 11, wherein the redundant data disk device to be determined is a magnetic tape device.