JP3725375B2 - Data reorganization method in storage media library - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a data reorganization method in an optical disk library and other write-once control storage medium libraries.
[0002]
[Prior art]
There are various uses for storing data for a long period of time, such as drawings, slips, sentences, and forms. In the case of long-term storage of data, optical disks are often used because of low bit cost and long-term storage. In particular, when the data capacity is increased, the number of cases in which a large-capacity volume composed of a plurality of discs is configured using an optical disk library is increasing.
[0003]
An optical disk library is connected to a personal computer or workstation server via a SCSI interface or the like, and generally a large-capacity logical volume (hereinafter simply referred to as a volume) is constructed by collecting a plurality of disks in the library. Therefore, the user can access as one volume without being aware of each disk.
[0004]
When multiple disc surfaces in an optical disc library are combined into a single volume, the file is usually written additionally and is rewritten at the end of the data instead of using the original space even if there is an update. It is. As a result, it is possible to skip searching for a plurality of discs in the file writing process or in reading the updated file. When considering a volume consisting of multiple discs using an optical disk library, there is an overhead due to mechanical operation in the library in addition to low access performance of the optical disk. The ability to avoid searching for discs is an important factor in terms of performance. Information about where the valid data is in which disk is stored separately as file management information on the server, and the target valid data can be acquired without directly accessing all the disks.
[0005]
[Problems to be solved by the invention]
By the way, in the case of the write-once format, the invalid area in the volume increases as the amount of update / delete files increases. For this reason, it is necessary to periodically reorganize the data in the volume.
[0006]
Conventionally, the data reorganization process for volumes consisting of multiple discs in an optical disk library is based on the invalid data that is no longer used by updating or deleting in order from the top disc surface. It was a way to go.
[0007]
However, this method has a problem that it takes an enormous amount of time to pack all data when the volume size increases due to an increase in the number of disks or a higher density. Further, during the data reorganization process, there is a problem that the volume cannot be accessed, and depending on the volume size, the operation must be stopped for about 1 to 2 weeks.
[0008]
In addition, the method of packing valid data in order from the first disk surface involves movement of written data. If the process is interrupted carelessly due to a hardware failure or the like, data loss may occur. was there.
[0009]
The object of the present invention is easy to apply to actual operation without repacking valid data from the top medium in an optical disk library or other storage medium library that generally uses a plurality of media as one logical volume. The object is to provide a data reorganization method with a low risk of data loss.
[0010]
[Means for Solving the Problems]
In the present invention, the conventional write-once method is used for writing to the volume. When a data reorganization instruction is issued, data is read in order from the top medium surface, and file management information is referenced to determine whether the data is valid or invalid. Therefore, only valid data is added to the end of the volume. As a result, the medium having only the invalid area is removed from the volume management, and the medium can be reused by formatting. As a result, each volume is constituted by a minimum number of disks, and the effective capacity of the entire library can be expanded. This method is effective when the number of media constituting the volume is plural, and the reorganization is completed when the process is performed up to the surface immediately before the final surface.
[0011]
In order to perform planned data reorganization without stopping normal operation, data reorganization can be started while the volume is online, and when normal access requests are received, data reorganization is interrupted, Furthermore, the system must be able to resume data reorganization. Therefore, to what medium and to what extent (reorganization end medium address) is stored at the end of the reorganization, and is resumed from there when the next reorganization is started.
[0012]
In addition, since the written data remains on the original medium surface as it is, no data loss occurs even if it is interrupted carelessly during the reorganization process due to a hardware failure or the like. The reorganization processing up to at least the reorganization end medium address managed by the control software in the server has been completed, and the reorganization processing may be resumed from there.
[0013]
Next, if different storage media libraries are under the same management by the control software and one volume can be configured with the media in each library, the first half of the volume and the second half of the volume can be separated into the first half. The data can be transferred from the medium constituting the medium to the medium constituting the latter half. If this processing is performed for all volumes, all the media in the library that made up the first half media become invalid areas, and data can be migrated to the other library by removing them from volume management. .
[0014]
In addition, in the method of selecting and packing only media with many invalid areas, a value of invalid capacity is set in advance as a standard for performing data reorganization, and data reorganization is performed only for media with invalid areas exceeding that value. To do. The control software always manages the valid / invalid area capacity of each medium constituting the volume. When data reorganization is started, only the corresponding medium is processed, and as a result, data reorganization can be performed efficiently. .
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. In the following embodiment, the target storage medium library is an optical disk library.
[0016]
FIG. 1 shows a configuration example of a system using an optical disk library. In the figure, an array disk 2 and an optical disk library 3 are connected to a server 1 that is a PC or a workstation by interface cables 41 and 42 such as SCSI. The array disk 2 and the optical disk library 3 constitute a hierarchical storage. In the server 1, in addition to access control of the array disk 2 and the optical disk library 3, control software 11 that performs data reorganization control of the optical disk library 3, and a file management information database (DB) 12 that holds the directory structure of the file Exists.
[0017]
Under the control of the control software 11 installed in the server 1 in advance, a plurality of disks contained in the optical disc library 3 are grouped in units of planes to form one volume. By handling each surface, it is possible to cope with an optical disk that can be used on both sides. In the example of FIG. 1, the disk surfaces 31, 32,..., 33, 34, 35 in the optical disk library 3 are grouped.
[0018]
Further, the control software 11 assigns the hard disk in the array disk 2 as a cache to the optical disk volume, stores frequently accessed data on a high-speed hard disk, and stores infrequently accessed data on a low-speed optical disk. By performing the control, hierarchical storage management that improves the overall access performance is also realized. In the example of FIG. 1, the disk surfaces 31, 32,..., 33, 34, 35 in the optical disk library 3 are grouped, and the hard disk 21 in the array disk 2 is allocated as a cache to configure the volume 5. It is shown that.
[0019]
When WRITE to the file 8 occurs in the volume 5 (6), the control software 11 first writes data to the cache 21 (61). When the update date and time of the file and the current time exceed a certain time, it is determined that the data has been determined, and the control software 11 refers to the file management information database 12 and copies the data to the corresponding optical disk. (62). This is called optical disc file update. When READ from the file 8 occurs for the volume 5 (7), the control software 11 reads the data from the cache 21 if it exists in the cache 21 (71). If it does not exist in the cache, the control software 11 refers to the file management information database 12 and copies the data from the corresponding optical disk to the cache 21 (72) and reads the data from the cache (71). In this case, the corresponding file on the optical disc remains valid, and thereafter the corresponding file (old data) on the optical disc is invalidated when being returned from the cache (when updating the optical disc file).
[0020]
FIG. 2 shows a configuration example of the optical disc library. As shown in FIG. 2, the optical disk library 3 normally includes an accessor (1111), a plurality of drives (11121, 11122), and a plurality of cells (1113), and an optical disk 1114 is stored in each cell. ing. Here, for example, it is assumed that access occurs in the order of the optical disk of the cell 11131, the optical disk of the cell 11132, and the optical disk of the cell 11133. Then, processing is performed in the following sequence under the control of the control software 11.
(1) The accessor 1111 moves the optical disk of the cell 11131 to the drive 11121 (1115), and performs I / O processing.
(2) The accessor 1111 moves the optical disk of the cell 11132 to the drive 11122 (1116), and performs I / O processing.
(3) After the accessor 1111 returns the optical disk from the drive 11121 to the cell 11131 (11171), the optical disk in the cell 11133 is moved to the drive 11121 (11172), and I / O processing is performed.
[0021]
Next, data access control of the optical disc library 3 will be described. The directory structure of the file is held in the file management information database 12, and usually the file data is stored on the optical disc. Since the write-once control is performed for the optical disc, when the file update of the optical disc occurs (that is, the updated file is written from the cache to the optical disc), the data of the same file exists twice. Therefore, the file management information database 12 holds a valid data address for each file, and realizes access to the target file data. 3 will be described as an example. It is assumed that data is written up to the area 1061 of the disk surface 106 in the volume 5 including the three disk surfaces 104, 105, and 106. When an update request for the file 1071 existing in the disk surface 104 of the volume 5 is generated (101), the control software 11 writes new data (updated file) in the area 1062 of the disk surface 106 (1011). Then, the physical address of the file 1071 in the file management information database 12 is changed from 1041 on the disk surface 104 to 1062 on the disk surface 106 (1012). As a result, the old data area 1041 of the file 1071 becomes an invalid area. When a deletion request for the file 1072 existing in the volume 5 is generated due to non-use for a long time (102), the control software 11 deletes the information about the file 1072 in the file management information database 12 (1021). ), The area 1042 becomes an invalid area, and the file 1072 is deleted from the volume 5. When a new file 1073 is created in the volume 5 in response to a request from the user (103), the control software 11 writes data in the area 1063 of the disk surface 106 (1031), and the file is stored in the file management information database 12. Information 1073 is added (1032).
[0022]
FIG. 4 shows a format example of a directory related to the optical disc library in the file management information database 12. Note that the cache directory is not directly related to the present invention, and is therefore omitted here.
[0023]
As shown in FIG. 4, the file management information database 12 is composed of a disk surface ID information 1211, a total number of files 1212 existing on each disk surface, and a file physical address 1213 table, as shown in FIG. Is done. There is a table 1211 for each volume, and there are as many tables 1212 as the number of disk surfaces constituting the volume. There are as many tables 1213 as the number of files stored on the disk surface. When file creation occurs, a table 1213 is added as the file management information database 12, and when file deletion occurs, information in the corresponding table 1213 is deleted. When the file is updated, the physical address in the disk surface of the corresponding table is updated.
[0024]
When data reorganization processing described later is performed, valid physical addresses in the table 1213 are acquired for the total number of files in the table 1212 for each disk surface based on the disk surface ID information in the table 1211. . When the entire area of the disk surface becomes invalid, the total number of files in the table 1212 on the disk surface becomes 0, and there is no information in the corresponding table 1213. Here, when the disk surface is further removed from the volume management, the information on the table 1212 corresponding to the corresponding disk surface device ID information is deleted from the table 1211.
[0025]
Next, data reorganization will be described. Data reorganization control is also handled by the control software 11 of the server 1. In the conventional method, data is packed and rewritten in order from the first disk surface. However, in the method of the present invention, valid data is additionally written from the first disk surface to the effective data final address of the same volume. .
[0026]
First, a conventional data reorganization process will be described with reference to FIG. Here, the volume 205 is composed of disk surfaces 201, 202, 203, and 204, and before data reorganization, valid data files 2051 to 2056 are stored on the disk surfaces 201, 202, and 203 as shown on the left side of FIG. Suppose that it exists rapidly. When data reorganization is performed on the volume 205 including the disk surfaces 201, 202, 203, and 204, first, the file 2051 and the file 2052 that are valid data of the area 201a of the disk surface 201 are stored on the same disk surface 201. The file 2053 and the file 2054, which are valid data of the area 202a of the disk surface 202, are written to the remaining area 201c of the disk surface 201. Next, the file 2055 that is valid data of the area 202 b of the disk surface 202 and the file 2056 that is valid data of the area 203 a of the disk surface 203 are rewritten in the head area 202 c of the disk surface 202. Since there is no valid data area on the disc surface 203, formatting is performed to make all areas empty (writable). Since the disk surface 204 is originally an empty area, it is left as it is (writable). As a result, after the data reorganization, the volume 205 is as shown on the right side of FIG.
[0027]
Next, the outline of the data reorganization processing of the present invention will be described with reference to FIG. As for the precondition, as in FIG. 5, it is assumed that the volume 305 constituted by the disk surfaces 301, 302, 303, and 304 is in the state shown on the left side of FIG. 6 before data reorganization. When data reorganization is performed, first, the file 2056 in the area 303a of the disc surface 303 is used as valid data as it is, and the file 2051 and the file 2052 that are valid data in the area 301a of the disc surface 301 are used as valid data of the disc surface 303. After the last data address, that is, in the area 303b immediately after the area 303a of the file 2056, the information is added. Next, the file 2053 which is valid data of the area 302 a of the disk surface 302 is written to the remaining area 303 c of the disk 303, and the files 2054 and 2055 of the areas 302 b and 302 c of the disk 302 are stored on the next disk surface 304. Write to the top area 304a. Since the file management information is also updated in this process, the disk surface 301 and the disk surface 302 are all invalid areas. As a result, after the data reorganization, the volume 305 becomes as shown on the right side of FIG. The disk surface 301 and the disk surface 302 can be removed from the management of the volume 305 when necessary, formatted and reused.
[0028]
Next, how to actually apply the data reorganization processing of the present invention will be described. First, the data reorganization process is basically not performed by reorganizing all the data in the volume at once, but is basically performed little by little every day. Therefore, the data reorganization process can be performed in a normal job schedule. For this purpose, a command to explicitly start and end a data reorganization process is prepared so that the data reorganization process can be handled as one job. Then, the data reorganization process can be scheduled by specifying the time and setting the job sequence in the overall system operation. As the timing for ending the data reorganization process, there is a case where a data reorganization end command is input, the reorganization process is completed, or a normal write operation occurs. Further, even if the end condition occurs, the access performance can be expected to improve if the process is terminated after the writing process of one file is completed because the related data is continuously arranged. Therefore, the end command is provided with a mode in which the file data being processed is written and then the process is ended and a mode in which the process is forcibly ended.
[0029]
At the end of the reorganization process, the point (disc address, file name) of which disk surface and how far the reorganization process has been performed is stored (for example, stored in the file management information database 12), and the normal writing process to be performed from now on And the processing start point at the start of the next data reorganization.
[0030]
Reorganization processing is performed, and the invalid disk surface does not need to be accessed basically, so that the volume can be removed from the management even when it is online. However, if both sides of the disc are used, remove them from management after both sides become invalid.
[0031]
When there is not enough space in the writing process due to normal operation writing and data reorganization, a formatted disk that has been put in the library in advance under the control of the control software 11 is automatically added to the volume. To be added. In addition, it is necessary to constantly monitor the volume capacity and the number of formatted discs that can be added, and notify the system administrator when the number of discs decreases. However, if the capacity is full during data reorganization and there is no disk that can be added, a warning message is issued, but the reorganization process itself is terminated normally. For this reason, the necessary capacity is checked in advance before data is written, so that the reorganization process does not end in the middle of the file.
[0032]
FIG. 7 shows an example of a flowchart of data reorganization processing according to the present invention based on the above. First, processing is started by a data reorganization start command. Here, an address at which the previous data reorganization is completed is acquired and set as a processing start address (step 401). When starting a new one, the start address of the top disk surface of the volume is set as the start address. Next, valid data is acquired from the disk surface with reference to the file management information in the file management information database 12 (step 402). Basically, as shown in FIG. 4, the effective data information for each disk surface is managed in the file management information database 12, and it is not necessary to access the entire area of the disk surface. Next, the obtained valid data is added to the valid data last address of the volume (step 403), and the physical address of the file is changed to the new address of the added data in the file management information (step 404). Here, it is checked whether to end the data reorganization processing. When an end notification is received by an input of a data reorganization end command or a normal write operation, or when the reorganization process is completed, the end of the data reorganization process is determined (step 405). In other cases, it is determined that the reorganization process is to be continued, and the process returns to step 402 to start from acquiring valid data again. At the end of the data reorganization process, the disk surface address at the end of the reorganization is saved and used as the start address for the next reorganization process. When the reorganization is completed, the next reorganization process starts from the top disk surface of the volume (step 406). When the end notification is forcibly received during the data writing of one file, step 404 is interrupted, and after the consistency between the file management information and the state written on the disk surface is taken, the process proceeds to step 406.
[0033]
An example in which the data reorganization processing of the present invention is incorporated in the actual operation schedule is shown in FIG. In this example, normal operation is from 8:00 to 0:00, data reorganization is performed from 1 o'clock with a margin of 1 hour, and a data reorganization end instruction is issued at 5 o'clock (operation schedule 505). An example applied to such an operation is a volume 504, which is composed of disk surfaces 501, 502,..., 503, and the area 5031 of the disk surface 503 is written. At this time, first, data is written in the area 5032 of the disk surface 503 by normal operation, and files 5041, 5042, and 5043, which are valid data of the disk surface 501, are stored on the disk surface 503 by subsequent data reorganization processing. It is written in area 5033. Then, in the next normal operation, data is written in the area 5034 of the disk 503, and the subsequent data reorganization processing writes the valid data 5044, 5045, 5046 of the disk 501 in the area 5035 of the disk surface 503. . As a result, all the disk surfaces 501 become invalid data areas, and the disk surface 502 becomes a new first surface.
[0034]
Next, an example in which data reorganization processing of the present invention is applied to transfer data between optical disk libraries will be described with reference to FIG. The old optical disk library 602 is connected to the server 601 by a SCSI interface cable 604a, and the new optical disk library 603 is similarly connected by an interface cable 604b such as SCSI. In FIG. 9, the cache array disk is omitted. Here, the disk surfaces 606e and 606f of the new optical disk library 603 are added to the volume 605 constituted by the disk surfaces 606a, 606b, 606c and 606d of the old optical disk library 602. Therefore, data reorganization processing is performed, and the disk surfaces 606a, 606b, 606c, and 606d of the old optical disk library are sequentially removed from the management of the volume 605 when they become invalid. As a result, the volume 605 is composed only of the disk surface of the new optical disk library 603, and data is transferred from the old medium to the new medium.
[0035]
Next, the case where the volume size is reduced by reorganizing only the disk surface having a large capacity of the invalid area will be described with reference to FIG. Before the data reorganization, as shown on the left side of FIG. 10, the volume 707 includes a disk surface 701 having an invalid area 10%, a disk surface 702 having an invalid area 30%, a disk surface 703 having an invalid area 40%, and an invalid area. It is assumed that a disk surface 704 having 10%, an effective area 50%, a disk surface 705 having an empty area 50%, and a disk surface 706 having an empty area 100% are formed. Here, if only the disk surface having an invalid area of 20% or more is set to be data reorganized and the reorganization processing is performed, the disk surfaces 701 and 704 are excluded, and the disk surfaces 702 and 703 are valid. Only data is added to the disk surfaces 705 and 706. As a result, after the data reorganization, the volume 707 has a disk surface 701, 704, 705, and 706 as shown on the right side of FIG. 10, and the invalid disk surfaces 702 and 703 are formatted and regenerated. It can be used.
[0036]
As mentioned above, although embodiment of this invention was described, it cannot be overemphasized that this invention is not limited to this. For example, the present invention is not limited to an optical disk library, and is generally applicable to a storage medium library in which a plurality of storage media are combined into one logical volume and data is written in a write-once format. In the embodiment, the hierarchical storage is configured by the array disk and the optical disk library. However, the present invention is not limited to the hierarchical storage.
[0037]
【The invention's effect】
As is apparent from the above description, the data reorganization method of the present invention provides the following effects.
(1) By removing a medium invalidated by data reorganization from the volume and reformatting it, the medium can be reused, and the usable area of the entire storage medium library increases.
[0038]
(2) From the operational point of view, it is not necessary to repack the valid data in order from the beginning medium of the volume, so the data reorganization process can be interrupted, and for example, the reorganization process can be executed little by little every day. There is an effect.
[0039]
(3) When considering from the viewpoint of data migration between libraries, data migration from an old medium to a new medium can be performed in the course of data reorganization processing. Therefore, data migration is possible even if the new drive cannot read the old medium.
[0040]
(4) With the conventional method, when creating a secondary medium, it was necessary to create a replica of the entire volume after data reorganization. However, if the present invention is adopted, a circle written by data reorganization is required. You only need to make a duplicate of the board. Therefore, even if the invalid area is refilled, the time required for creating the sub medium is reduced.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration example of a system applied to an optical disc library according to an embodiment of the present invention.
FIG. 2 shows an outline of the structure of an optical disc library.
FIG. 3 is a diagram for explaining volume write-once processing;
FIG. 4 is a diagram illustrating a format example of a volume file management information database;
FIG. 5 is a diagram for explaining conventional data reorganization processing;
FIG. 6 is a diagram illustrating data reorganization processing according to the present invention.
FIG. 7 is a flowchart showing an example of data reorganization processing according to the present invention.
FIG. 8 is a diagram showing an example when the data reorganization processing according to the present invention is put into actual operation;
FIG. 9 is a diagram showing an example of performing data migration by data reorganization processing according to the present invention.
FIG. 10 is a diagram showing an example in which only a disk surface with many invalid areas is selected and processed in the data reorganization processing according to the present invention.
[Explanation of symbols]
1 server
11 Control software
12 File management information database
2 Array disk
21 Hard disks in the array disk
3 Optical disk library
31-35 Disk surface in optical disk library
5 volumes
6 WRITE process for volume 5
61-62 Internal processing in WRITE process
7 READ process for volume 5
71-72 Internal processing in READ process
8 Files in the volume

Claims (6)

A data reorganization method in a storage medium library in which a plurality of medium surfaces are combined into one logical volume (hereinafter simply referred to as a volume), and data is written to the volume in a write-once format,
It is characterized in that only valid data is read in order from the medium surface at the beginning of the volume, and the read data is added to the final valid data area on the medium surface where the final valid data in the volume is written. To reorganize data. 2. The data reorganization method according to claim 1, wherein a medium having only an invalid area by data reorganization processing is removed from the volume management. 3. The data reorganization method in the storage medium library according to claim 1, wherein when the data reorganization process is interrupted, the medium surface address of the medium at that time is held, and when the data reorganization is resumed, the medium surface address is set to A data reorganization method characterized by adding data continuously. 4. A data reorganization method in a storage medium library according to claim 1, wherein one volume is composed of a group of media in different types of libraries, and valid data of a medium in a library is sequentially changed in another data reorganization process. A data reorganization method comprising transferring data to a library medium and finally eliminating a medium to be managed in the original library. 4. A data reorganization method in a storage medium library according to claim 1, wherein only a medium group having a large number of invalid areas in a volume is selected to perform data reorganization processing, and the medium is removed from volume management. A data reorganization method characterized by the above. 6. The data reorganization method in the storage medium library according to claim 1, wherein the storage medium library accommodates a large number of optical disks, and each of the plurality of disk surfaces is combined into a single volume for use. A data reorganization method characterized by the above.

Images