CN108170377B - Backup method for supporting multi-drive concurrent writing of tape library - Google Patents
Backup method for supporting multi-drive concurrent writing of tape library Download PDFInfo
- Publication number
- CN108170377B CN108170377B CN201711450059.0A CN201711450059A CN108170377B CN 108170377 B CN108170377 B CN 108170377B CN 201711450059 A CN201711450059 A CN 201711450059A CN 108170377 B CN108170377 B CN 108170377B
- Authority
- CN
- China
- Prior art keywords
- data
- writing
- tape library
- write
- executor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0613—Improving I/O performance in relation to throughput
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a backup method for supporting multi-drive concurrent writing of a tape library, which comprises the following steps: (1) initializing tape library operation; (2) generating the same number of write data actuators according to the number of the tape library drivers selected in the step (1), wherein each write data actuator corresponds to one tape library driver; (3) processing data, dividing the data into data groups and writing the data groups; (4) continuing to execute the step (3) until all the data transmitted by the data source are received and processed; (5) and ending backup and releasing resources. Compared with the prior art, the method can greatly shorten the time of the data protection window and improve the performance of data migration to the tape.
Description
Technical Field
The invention relates to the field of data storage, in particular to a backup method for supporting multi-drive concurrent writing of a tape library.
Background
With the continuous development of computers, data protection is gradually strengthened. The accumulation of large amounts of data stresses data storage. But is limited by compliance and other factors and requires that some data be protected even if it is not used for a long period of time. Migrating this data to tape is a good way.
The storage performance of a single drive of a tape library is limited by hardware conditions and is extremely limited. The migration of a large amount of data to a tape is limited by the performance bottleneck of a drive of a tape library, so that the backup window time is long, the backup efficiency is affected, and a user cannot perform data protection faster and more timely. If there is a way to distribute data to multiple tape library drives for simultaneous write data operations, then the backup performance is theoretically the sum of all tape library drive performance without other performance bottlenecks. The method greatly improves the performance of data migration to the magnetic tape.
Disclosure of Invention
The present invention aims to overcome the defects of the prior art and provide a backup method supporting multi-drive concurrent writing of a tape library, which greatly reduces the time of a data protection window and improves the performance of data migration to a tape, so that a user can better, faster and more timely perform data protection.
The purpose of the invention can be realized by the following technical scheme:
a backup method for supporting multi-drive concurrent writing of a tape library comprises the following steps:
(1) initializing tape library operation;
(2) generating the same number of write data actuators according to the number of the tape library drivers selected in the step (1), wherein each write data actuator corresponds to one tape library driver;
(3) processing data, dividing the data into data groups and writing the data groups;
(4) continuing to execute the step (3) until all the data transmitted by the data source are received and processed;
(5) and ending backup and releasing resources.
Preferably, the step (1) specifically comprises the following steps:
(1.1) selecting a tape library drive to participate in the backup;
(1.2) loading the magnetic tapes on the magnetic tape library drivers selected in the step (1.1);
(1.3) positioning the magnetic tape loaded in the step (1.2) to a writable position.
Preferably, the positioning operation of step (1.3) is executed asynchronously, and the background continues to position asynchronously to the position specified in step (1.3) after the step (1.3) is executed.
Preferably, the step (3) comprises the following steps:
(3.1) receiving data transmitted by a data source end, receiving a part of data according to the proportion of the set data quantity and the sequence of transmission, and defining the part of data as a data group;
(3.2) numbering the data groups in the step (3.1) in sequence to ensure the sequence of the data;
(3.3) selecting a writable write data executor from the write data executors generated in the step (2);
(3.4) processing the data group storage information of the data writing executor selected in the step (3.3);
(3.5) distributing the data set obtained in step (3.1) to the write data executor selected in step (3.3);
(3.6) the data writing executor organizes the data group into binary data according to a set data format;
and (3.7) writing the binary data obtained in the step (3.6) into a magnetic tape loaded in a corresponding magnetic tape library driver of a write data actuator.
Preferably, whether the write data executor of step (3.3) is writable depends on whether the magnetic tape has been positioned to a writable location and whether a data group is being written;
and if the magnetic tape is positioned to the writable position and the data group is not being written, defining the state of the data writing actuator as writable, wherein the state of the data writing actuator is not writable in the process of writing the data group, and the state of the data writing actuator is writable after the data writing group is completed. The writable state of the data writing actuator ensures that only one data group can exist in one data writing actuator at the same time, the state of the data writing actuator is not writable in the process of writing the data group, and the state of the data writing actuator is writable after the data writing group is completed.
Preferably, the data distribution operation of step (3.5) is performed asynchronously, and the step (3.5) distributes the data group to the write data executor, i.e., to the return.
Preferably, the data writing executor organizes the data group into binary data through step (3.6) and writes the data into the magnetic tape through a background writing thread, the background writing thread will continuously write the incoming data into the magnetic tape in the background, and after the writing is completed, generate the storage information corresponding to the data group, and the storage information includes the magnetic tape label, the starting physical block position, the data group size, and the data group attribute information.
Preferably, said step (3.4) comprises the steps of:
(3.4.1) judging whether the data group of which the writing operation is finished exists in the data writing executor selected in the step (3.3), if not, executing the step (3.5), otherwise, continuing to execute;
(3.4.2) acquiring the storage information of the completed write operation data group corresponding to the write data executor selected in the step (3.3);
and (3.4.3) storing the data group storage information acquired in the step (3.4.2) according to a set storage format.
Preferably, the step (5) comprises the following steps:
(5.1) sequentially selecting one write data executor generated in the step (2);
(5.2) judging whether the data group of which the writing operation is finished exists in the data writing executor selected in the step (5.1), if not, executing the step (5.5), otherwise, continuing to execute;
(5.3) acquiring the storage information of the data group which has completed the write operation and corresponds to the data writing actuator in the step (5.2);
(5.4) storing the data group storage information acquired in the step (5.3) according to a set storage format;
(5.5) continuing to execute the step (5.1) until all the write data executors generated in the step (2) have been selected;
(5.6) closing the write data executor generated in the step (2);
(5.7) releasing the tape library drive selected in step (1.1).
Preferably, the data writing executor is a logic body for caching data groups, organizing the data groups into binary data according to a set format, and writing the binary data into a magnetic tape to generate data group storage information, and each data writing executor comprises a background writing thread for writing data.
The number of the data group is unique in the invention, and the purpose of the number is to maintain the writing sequence of the data. The data set numbers are distributed with the data sets to the write data executors at step 3.5 and are also associated with the stored information for the data sets at step 3.4.2.
Compared with the prior art, the method and the device can distribute the data transmitted by the data source to a plurality of tape library drivers for writing, so that the writing performance of the data is greatly improved, and the bottleneck of the writing performance of a single tape library driver is broken through. The method can actively and effectively help the user to complete data protection more quickly and timely, so that more material resources and human resources are saved.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is a flow chart of the method of initializing the tape library operation in step (1) of the present invention;
FIG. 3 is a flow chart of the present invention for processing data in step (3);
fig. 4 is a flowchart of ending the backup releasing resources in step (5) of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, shall fall within the scope of protection of the present invention.
The operating system used in this example is 64-bit RedHat Enterprise 5, and the data uses 10 files with a file size of 1 GB. The tape library employs ULT3580-TD5, and includes 4 tape library drives, 4 tapes, and writable locations of the tapes are all 0. The size of the data group is set to a maximum of 64MB, and less than 64MB is allowed when the remaining data source is less than 64 MB.
As shown in fig. 1-4, this example comprises the following steps:
(1) initializing tape library operations
(1.1) selecting 4 tape library drives as tape library drives participating in the backup, the tape library drive device paths being/dev/nst 0,/dev/nst 1,/dev/nst 2,/dev/nst 3, respectively;
(1.2) loading the magnetic tapes on the 4 magnetic tape library drives selected in the step (1.1), wherein the magnetic tape labels are respectively E01001L5, E01002L5, E01003L5 and E01004L 5;
and (1.3) positioning the magnetic tapes loaded in the step (1.2) to writable positions, wherein the writable positions of the magnetic tapes are 0. The operation is asynchronous;
(2) generating 4 data writing actuators, wherein each data writing actuator comprises a background writing thread for writing data into a magnetic tape;
(3) processing data, dividing the data into data groups and writing the data groups;
(3.1) sequentially reading data of 10 files of 1GB, taking the read data as a data source, transmitting the data in, receiving the maximum 64MB of data according to the transmitted sequence, and defining the part of data as a data group;
(3.2) numbering the data groups in the step (3.1) in sequence to ensure the sequence of the data, wherein the numbering is started from 0 and is unique;
(3.3) selecting a writable write data executor from the write data executors generated in the step (2);
(3.4) processing the data group storage information of the data writing executor selected in the step (3.3);
(3.4.1) judging whether the data group of which the write operation is finished exists in the data writing executor selected in the step (3.3). If not, executing the step (3.5), otherwise, continuing to execute;
(3.4.2) acquiring the storage information of the data group with the write operation completed corresponding to the write data executor selected in the step (3.3), wherein the storage information comprises information such as a tape label, a data group number, a starting physical storage position, a data group size and a data group attribute;
(3.4.3) storing the data group storage information acquired in the step (3.4.2) according to a certain storage format;
(3.5) distributing the data group obtained in the step (3.1) to the data writing executor selected in the step (3.3), wherein the operation is asynchronous operation;
(3.6) organizing the data group into binary data according to a certain data format by the data writing actuator;
(3.7) writing the binary data obtained in the step (3.6) into a magnetic tape loaded in a magnetic tape library driver corresponding to a data writing actuator through a writing thread;
(4) continuing to execute the step (3) until all the data transmitted by the data source are received and processed;
(5) ending backup and releasing resources;
(5.1) sequentially selecting one write data executor generated in the step (2);
(5.2) judging whether the data group of which the write operation is finished exists in the data writing executor selected in the step (5.1). If not, executing the step (5.5), otherwise, continuing to execute;
(5.3) acquiring the storage information of the data group which has completed the write operation and corresponds to the data writing actuator in the step (5.2);
(5.4) storing the data group storage information acquired in the step (5.3) according to a certain storage format;
(5.5) continuing to execute the step (5.1) until all the write data executors generated in the step (2) have been selected;
(5.6) closing the write data executor generated in the step (2);
(5.7) releasing the tape library drive selected in step (1.1);
while the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. A backup method for supporting multi-drive concurrent writing of a tape library, the method distributing data incoming from a data source to a plurality of tape library drives for writing, the method comprising the steps of:
(1) initializing tape library operation;
(2) generating the same number of write data actuators according to the number of the tape library drivers selected in the step (1), wherein each write data actuator corresponds to one tape library driver;
(3) processing data, dividing the data into data groups and writing the data groups;
(4) continuing to execute the step (3) until all the data transmitted by the data source are received and processed;
(5) ending backup and releasing resources;
the step (3) comprises the following steps:
(3.1) receiving data transmitted by a data source end, receiving a part of data according to the proportion of the set data quantity and the sequence of transmission, and defining the part of data as a data group;
(3.2) numbering the data groups in the step (3.1) in sequence to ensure the sequence of the data;
(3.3) selecting a writable write data executor from the write data executors generated in the step (2);
(3.4) processing the data group storage information of the data writing executor selected in the step (3.3);
(3.5) distributing the data set obtained in step (3.1) to the write data executor selected in step (3.3);
(3.6) the data writing executor organizes the data group into binary data according to a set data format;
(3.7) writing the binary data obtained in the step (3.6) into the magnetic tape loaded in the magnetic tape library driver corresponding to the data writing actuator;
the step (3.4) comprises the following steps:
(3.4.1) judging whether the data group of which the writing operation is finished exists in the data writing executor selected in the step (3.3), if not, executing the step (3.5), otherwise, continuing to execute;
(3.4.2) acquiring the storage information of the completed write operation data group corresponding to the write data executor selected in the step (3.3);
(3.4.3) storing the data group storage information acquired in the step (3.4.2) according to a set storage format;
the number of the data group is unique, and the purpose of the number is to maintain the writing sequence of the data; the number of the data set is distributed to the write data executor along with the data set in step (3.5) and is also associated with the stored information of the data set in step (3.4.2).
2. The backup method for supporting multi-drive concurrent writing of a tape library according to claim 1, wherein the step (1) comprises the following steps:
(1.1) selecting a tape library drive to participate in the backup;
(1.2) loading the magnetic tapes on the magnetic tape library drivers selected in the step (1.1);
(1.3) positioning the magnetic tape loaded in the step (1.2) to a writable position.
3. The backup method for supporting multi-drive concurrent writing of a tape library according to claim 2, wherein the locating operation of step (1.3) is performed asynchronously, and the background of the step (1.3) performs the asynchronous continuous locating operation to the position specified in step (1.3).
4. A backup method for supporting multi-drive concurrent writing of a tape library as claimed in claim 1, wherein the write data executor in step (3.3) is writable depending on whether the tape has been positioned to a writable location and whether a data group is being written;
and if the magnetic tape is positioned to the writable position and the data group is not being written, defining the state of the data writing actuator as writable, wherein the state of the data writing actuator is not writable in the process of writing the data group, and the state of the data writing actuator is writable after the data writing group is completed.
5. A backup method supporting multi-drive concurrent writing for a tape library according to claim 1, wherein the data distribution operation of step (3.5) is executed asynchronously, and the step (3.5) distributes the data group to the write data executor and returns.
6. The backup method for supporting multi-drive concurrent writing of a tape library as claimed in claim 1, wherein the write data executor organizes the data group into binary data through step (3.6) and writes the data to the tape through a background write thread, the background write thread continuously writes the incoming data to the tape in the background and generates the storage information corresponding to the data group after completing the writing, the storage information includes tape label, starting physical block location, data group size and data group attribute information.
7. A backup method supporting multi-drive concurrent writing of a tape library according to claim 1, wherein said step (5) comprises the steps of:
(5.1) sequentially selecting one write data executor generated in the step (2);
(5.2) judging whether the data group of which the writing operation is finished exists in the data writing executor selected in the step (5.1), if not, executing the step (5.5), otherwise, continuing to execute;
(5.3) acquiring the storage information of the data group which has completed the write operation and corresponds to the data writing actuator in the step (5.2);
(5.4) storing the data group storage information acquired in the step (5.3) according to a set storage format;
(5.5) continuing to execute the step (5.1) until all the write data executors generated in the step (2) have been selected;
(5.6) closing the write data executor generated in the step (2);
(5.7) releasing the tape library drive selected in step (1.1).
8. The backup method of claim 1, wherein the write data executor is a logic for caching data groups, organizing the data groups into binary data according to a predetermined format, and writing the binary data into the tape to generate data group storage information, and each write data executor comprises a background write thread for writing data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711450059.0A CN108170377B (en) | 2017-12-27 | 2017-12-27 | Backup method for supporting multi-drive concurrent writing of tape library |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711450059.0A CN108170377B (en) | 2017-12-27 | 2017-12-27 | Backup method for supporting multi-drive concurrent writing of tape library |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108170377A CN108170377A (en) | 2018-06-15 |
| CN108170377B true CN108170377B (en) | 2022-04-01 |
Family
ID=62518813
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711450059.0A Active CN108170377B (en) | 2017-12-27 | 2017-12-27 | Backup method for supporting multi-drive concurrent writing of tape library |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108170377B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2023108955A (en) * | 2022-01-26 | 2023-08-07 | 富士フイルム株式会社 | Information processing apparatus, information processing method, and information processing program |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101727291A (en) * | 2008-10-14 | 2010-06-09 | 北京众志和达信息技术有限公司 | System and method for embedded integrated virtual tape library |
| CN102541957A (en) * | 2010-12-30 | 2012-07-04 | 北大方正集团有限公司 | Method and device for operating data files on basis of data flow magnetic tapes |
| CN103164172A (en) * | 2013-03-11 | 2013-06-19 | 华为技术有限公司 | Data flow storage method and device |
| CN103713860A (en) * | 2014-01-03 | 2014-04-09 | 浪潮(北京)电子信息产业有限公司 | Method and system for building virtual disk base on basis of backup architecture |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9189167B2 (en) * | 2012-05-31 | 2015-11-17 | Commvault Systems, Inc. | Shared library in a data storage system |
| US9619331B2 (en) * | 2014-01-18 | 2017-04-11 | International Business Machines Corporation | Storage unit replacement using point-in-time snap copy |
-
2017
- 2017-12-27 CN CN201711450059.0A patent/CN108170377B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101727291A (en) * | 2008-10-14 | 2010-06-09 | 北京众志和达信息技术有限公司 | System and method for embedded integrated virtual tape library |
| CN102541957A (en) * | 2010-12-30 | 2012-07-04 | 北大方正集团有限公司 | Method and device for operating data files on basis of data flow magnetic tapes |
| CN103164172A (en) * | 2013-03-11 | 2013-06-19 | 华为技术有限公司 | Data flow storage method and device |
| CN103713860A (en) * | 2014-01-03 | 2014-04-09 | 浪潮(北京)电子信息产业有限公司 | Method and system for building virtual disk base on basis of backup architecture |
Non-Patent Citations (1)
| Title |
|---|
| THStoreMan海量层次存储系统的设计与物理存储的实现;牛奎光;《中国优秀硕士学位论文全文数据库信息科技辑》;20050715;正文第2、12-44页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108170377A (en) | 2018-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8468313B2 (en) | Asynchronous replication with write concurrency grouping | |
| CN105531665B (en) | Cache for virtual memory facilities degrades | |
| US20150253992A1 (en) | Memory system and control method | |
| CN106503020B (en) | Log data processing method and device | |
| CN109598156B (en) | Method for redirecting engine snapshot stream during writing | |
| US10552045B2 (en) | Storage operation queue | |
| US8271968B2 (en) | System and method for transparent hard disk drive update | |
| US20130067147A1 (en) | Storage device, controller, and read command executing method | |
| US20150149741A1 (en) | Storage System and Control Method Thereof | |
| US8327041B2 (en) | Storage device and data transfer method for the same | |
| US9804903B2 (en) | Data processing apparatus for pipeline execution acceleration and method thereof | |
| US11340806B2 (en) | Meta data processing during startup of storage devices | |
| CN114063922A (en) | Method, device, equipment and medium for accelerating replication of master and slave library streams | |
| CN114138200A (en) | Pre-writing log method and system based on rocksDB | |
| CN104461384A (en) | Data write-in method and storage device | |
| CN108170377B (en) | Backup method for supporting multi-drive concurrent writing of tape library | |
| CN104133640B (en) | From the fast quick-recovery of dormancy | |
| CN105094711A (en) | Method and device for achieving copy-on-write file system | |
| CN114116008B (en) | Command queue management method and device, readable storage medium and electronic equipment | |
| CN112767976B (en) | Method, device, storage medium and computer equipment for stabilizing flash memory writing speed | |
| CN112114757B (en) | Storage method and system in object storage system, computing device and medium | |
| CN109542671A (en) | Verify data creation method and solid state hard disk | |
| CN108536393B (en) | Disk initialization method and device | |
| CN107273051B (en) | The reading/writing method and device of I/O data | |
| CN110347339A (en) | A kind of controllable continuous write method, scheduler and storage system for mechanical hard disk |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |