CN107220007B - A kind of VTL system and method based on blue-ray storage equipment - Google Patents
A kind of VTL system and method based on blue-ray storage equipment Download PDFInfo
- Publication number
- CN107220007B CN107220007B CN201710449246.0A CN201710449246A CN107220007B CN 107220007 B CN107220007 B CN 107220007B CN 201710449246 A CN201710449246 A CN 201710449246A CN 107220007 B CN107220007 B CN 107220007B
- Authority
- CN
- China
- Prior art keywords
- data
- vtl
- tape
- metadata
- blue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0677—Optical disk device, e.g. CD-ROM, DVD
Abstract
The invention discloses a kind of VTL system and methods based on blue-ray storage equipment, system includes VTL servers, blue light gateway server, Blu-ray Disc base management system;VTL servers externally provide the backup services based on TL agreements by iSCSI/FC, then according to the different function module of different request calls, and implementing result and data are returned to backup server;Blue light gateway server supports multiple interfaces in the form of plug-in unit, and gateway uses and manages Blu-ray Disc storehouse, while provides cache management and data read and write interface;Blu-ray Disc base management system provides data access and management interface.Metadata and user data correspond to different storage modes, are stored in respectively in different CDs, respectively using different cache management strategies;The present invention is a kind of high-throughput, high reliability, data consistency, the standby system of low cost.
Description
Technical field
It is more particularly to a kind of to be used for Virtual Tape Library the invention belongs to data storage, disaster tolerance and redundancy technique field
Management system and method for (Virtual Tape Library, the VTL) server in Blu-ray Disc stored/backed-up system.
Background technology
In recent years, related application is grown rapidly with big data for computer technology especially internet, computer data information
Amount constantly increases.The demand of resultant backup and recovery is also increasing, and the technology of backup and recovery also exists
Continue to develop.As the continuous lifting of disk size, cost constantly decline, by original tape backup, derive based on disk
Virtual Tape Library technology.And with the continuous lifting of Blue-ray disk technology, in existing Virtual Tape Library (Virtual Tape
Library on the basis of) backing up, it is virtual that Hubei University of Education with Sai Fan companies develops the blue light based on Blu-ray Disc storehouse
Tape library (BD-VTL).
Blue light Virtual Tape Library (BD-VTL) uses Blu-ray Disc (Bluray Disc) to be used as storage medium, is provided by i
Tape library (TL) backup services.The system includes two parts:Blu-ray Disc storehouse and BD-VTL management softwares.Blu-ray Disc storehouse carries
For:Data access interface, CD moving operation (be loaded into CD-ROM drive or unloaded from CD-ROM drive), data write-in (imprinting) and reading connect
Mouthful, and CD server status inquiry and management interface.
The content of the invention
In order to solve how efficiently to provide backup services by TL agreements using this WORM storage mediums of Blu-ray Disc,
The present invention provides a kind of VTL system and method based on blue-ray storage equipment.
Technical solution is used by the system of the present invention:A kind of VTL systems based on blue-ray storage equipment, its feature exist
In:Including VTL servers, blue light gateway server, Blu-ray Disc base management system;
The VTL servers, the backup services based on TL agreements are externally provided by iSCSI/FC, then according to different
The different function module of request call, and implementing result and data are returned to backup server;
The blue light gateway server, supports multiple interfaces in the form of plug-in unit, and gateway uses and manages Blu-ray Disc storehouse,
There is provided cache management and data read and write interface at the same time;The cache management, coordinates to complete, VTL clothes according to TL semantemes and gateway
Business device passes through cache management interface requests read buffer, synchronization/deletion write buffer;
The Blu-ray Disc base management system provides data access and management interface.
Technical solution is used by the method for the present invention:A kind of VTL methods based on blue-ray storage equipment, its feature exist
In:Metadata and user data correspond to different storage modes, are stored in respectively in different CDs, are delayed respectively using different
Deposit management strategy;
For metadata, the specific implementation of the method includes following principle:
Principle A1:When BD-VTL starts, corresponding metadata is loaded into caching, it is resident slow before BD-VTL is offline
Deposit;
Principle A2:Metadata read-write operation is all completed in the buffer;
Principle A3:Metadata data amount is when reaching the capacity of CD casket before offline, backwash write buffer, to ensure
The reliability of metadata;
For user data, the specific implementation of the method includes following principle:
Principle B1:When loading tape to tape drive, the metadata of tape is loaded to caching;
Principle B2:For forward/backward operation, according to tape location, give up the caching before the position, and cache the position
Caching after putting;
Principle B3:For, to the data cached of tape end, being postponed after user's valid data using TRIM operations
Deposit middle deletion;
Principle B4:When tape is unloaded from tape drive, the metadata and user data of backwash tape.
The present invention not only solves the problem of special storage medium this using Blu-ray Disc by TL agreements, and ties
TL agreements and the WORM characteristics of Blu-ray Disc are closed, adds a small amount of disk buffering, efficiently solves the competition of CD drive
Problem, and reduce the consuming of Blu-ray Disc.
Brief description of the drawings
Fig. 1 is the systematic schematic diagram of the embodiment of the present invention.
Embodiment
Understand for the ease of those of ordinary skill in the art and implement the present invention, with reference to the accompanying drawings and embodiments to this hair
It is bright to be described in further detail, it will be appreciated that implementation example described herein is merely to illustrate and explain the present invention, not
For limiting the present invention.
The technical problem to be solved in the present invention is:How using this WORM storage mediums of Blu-ray Disc TL is efficiently passed through
Agreement provides backup services.
A kind of VTL systems based on blue-ray storage equipment provided by the invention, including VTL servers, blue light gateway service
Device, Blu-ray Disc base management system;
TL virtualisation components, the backup services based on TL agreements are externally provided by iSCSI/FC, then according to different
The different function module of request call, and implementing result and data are returned to backup server.
Blue light gateway server, supports multiple interfaces, such as file system interface, block interface and to right in the form of plug-in unit
As interface.Gateway uses and manages CD server, while provides cache management and data read and write interface.Since CD casket is that WORM is set
Standby, gateway reduces the invalid data in write-in CD using write buffer;In addition the CD-ROM drive group number in CD server is limited, and client please
When seeking data, gateway can read a bulk of data from CD, be cached to read buffer.Management to caching, it is semantic according to TL
Completed with gateway cooperation.TL virtualisation components pass through cache management interface requests read buffer, synchronization/deletion write buffer.Caching
Local RAID is stored in, to improve reliability.
Blu-ray Disc base management system provides most basic data access and management interface, as CD is mobile, data are read
Take with imprinting etc..
BD-VTL storages rear end can use different interfaces, such as block, file system, object storage etc..BD-VTL is examined
Performance, VTL semantemes and storage medium characteristic are considered, reducing user using the mode of caching and data separating uses this spy
Different large capacity equipment.
Data separating refers to that user data (user data) and metadata (meta data) separate, wherein changer,
Drive and iscsi configurations are all metadata;Tape includes this two parts data, metadata include the description of tape itself with
And the tissue of user data, another part are exactly the user data of backup software write-in.
The characteristic of metadata:Data volume is small, and renewal frequency is high, and access frequency is high.Resident caching is needed, and reduces and caches back
Brush operation.The characteristic of user data:Data volume is big, and renewal and visiting frequency are difficult to control.Memory-resident during use, loads tape
During to drive, read buffer request operation is carried out at the same time, all write operations are all completed in the buffer.Tape is unloaded from drive
When, by caching backwash to CD.
BD-VTL management softwares include gateway and TL virtualisation components (VTL servers).Gateway uses and manages CD server,
There is provided cache management and data read and write interface at the same time.Since CD casket is WORM device, gateway reduces write-in light using write buffer
Invalid data in disk;In addition the CD-ROM drive group number in CD server is limited, and during client request data, gateway can be by a bulk of data
Read buffer is read into from CD.Management to caching according to TL semantemes and gateway, it is necessary to coordinate to complete.TL virtualisation components
Pass through cache management interface requests read buffer, synchronization/deletion write buffer.
TL virtualisation components externally provide TL services.It uses Blu-ray Disc to be connect as storage rear end by above-mentioned three kinds
Mouthful (file system interface, block interface and to object interface) completes data storage and TL management operations.Metadata and user
Data separating is stored, easy to cache request and synchronization.Different types of data have different cache policies.Metadata is included in TL
The configuration information of changer/drive/tape, and tape data distribution forms.The former it is slow to be saved in reading when TL is loaded
Deposit, and provide viability in TL and be resident in the buffer.The latter is then after tape is loaded into drive, is loaded and resident slow
Deposit, data are synchronized to CD casket during unloading.The user data part of TL, after tape is loaded into drive, according to tape positions,
Cache the data after the position, to reduce cache occupation amount, and ensure to back up in the soft data procedures according to read-write will not be due to
Data do not cause time-out in caching.Retrogressing (rewind) operation to tape after data reach caching, it is necessary to complete.To tape
Advance (fastforward) operation, trigger and backwash or give up unwanted data cached.
In order to support to provide the tape of vast capacity, a tape is divided into one or more small by TL virtualisation components
Storage unit.The read-write operation of multiple junior units can be converted into the read-write operation of tape, these operations can carry parallel
Hand over asynchronous completion.In addition the trim operations of support target, can reduce actual occupied space, also reduce the usage amount of CD.
Referring to Fig.1, a kind of VTL methods based on blue-ray storage equipment provided by the invention, metadata and user data pair
Different storage modes is answered, is stored in respectively in different CDs, respectively using different cache management strategies;
A) metadata:
When i.BD-VTL starts, corresponding metadata is loaded into caching, resident caching before BD-VTL is offline
Ii. metadata read-write operation is all completed in the buffer
Iii. metadata data amount is when reaching the capacity of CD casket before offline, backwash write buffer, to ensure member
The reliability of data
B) user data:
When i. loading tape to tape drive, the metadata of tape is loaded to caching
Ii. for forward/backward operation, according to tape location, give up the caching before the position, and cache the position it
Caching afterwards
Iii. for, to the data cached of tape end, being operated after user's valid data using TRIM by it from caching
Delete
When tape is iv. unloaded from tape drive, the metadata and user data of backwash tape
C) caching provides large capacity and reliability using RAID or other modes
D) data fragmentation layer:Either metadata or user data, can all be fragmented preservation, by parallel form
Blue-ray storage system is passed to, and is downloaded in caching, to improve performance, while supports TRIM to operate
The storage medium of the present invention employs substantial amounts of blu-ray media and a small amount of disk buffering, is designed plus rational,
The worm characteristics of blu-ray media can sufficiently be utilized, there is provided a kind of high-throughput, high reliability, data consistency, low cost
Standby system.
Present invention employs the realization and Blu-ray instrument characteristic for combining TL agreements, carries out caching pipe with dynamically customizing
Reason, thereby using limited spatial cache, there is provided high-throughput, and reduce the consuming of blue-ray storage.
The technology of the present invention is realized main points and is contrasted with existing scheme as follows:
It is data cached before being used for protection write-in Blu-ray Disc using RAID in 1.BD-VTL, the use of caching is in order to more
Make full use of network bandwidth (raising backup rate) and reduce the loss of Blu-ray Disc.
2. relative to tape, Blu-ray Disc can form RAID to provide Reliability Assurance, and can provide higher
Bandwidth, the order readwrite bandwidth of a CD casket (including multiple Blu-ray Discs) are suitable with disk.
3. in the prior art, Backup Data first writes disk array, then files physical tape storehouse, in order to improves
Backup bandwidth and reliability, BD-VTL use part of cache and Blu-ray Disc, can meet the needs of these two aspects.
4.BD-VTL can use file system/block/object interface when Blu-ray Disc storehouse interacts, and data can back up
To local or high in the clouds.
5.BD-VTL backup flows.
A) backup server connects BD-VTL by specific protocol (such as iscsi), and writes data;
B) data write-in local cache, and as needed, backwash data to Blu-ray Disc storehouse;
C) after Backup end, all customer data and metadata backwash to Blu-ray Disc storehouse;
6.BD-VTL recovers flow;
A) backup server connects BD-VTL by specific protocol (iSCSI/FC), and recovery operation starts;
B) data are read from CD server, and write read buffer, while are transferred to backup server;
C) after recovering, all customer data and metadata can all be removed appropriate at the time of from caching.
The deployment of BD-VTL includes two parts TL virtualisation components and gateway server.TL virtualisation components take with gateway
Business device is located in same LAN, is communicated by customized agreement.When deployment is embodied, the two can be located at same
In one platform server.
During the normal operation of BD-VTL systems, data are sent to gateway service by data backup client by VTL servers
VTL/Tape data are write corresponding directories/files by device, gateway server according to shown in attached drawing 1.Catalogue on disc cache/
File is divided into metadata catalog/file and data recording on tape catalogue file, and meta data file record CD casket is corresponding with tape to close
System and corresponding CACHE DIRECTORY.Machine readable catalogue as the corresponding container catalogue of CD belonging to it under.Each data recording on tape
It is the section file of data recording on tape under catalogue, the size of each data slicer file is no more than 64M.Under the root of disc cache
File brgw-status-file records all be unloaded but data are also present in tape name on disc cache.The name of tape
There is uniqueness in whole blue light VTL systems, it is impossible to bear the same name.
After backup client writes data into disc cache, backup client can think data backup success.Afterwards,
Write by the backstage upload program of the caching disk management system of the present invention in the CD casket of CD server.
After backup client completes data backup and unloads CD casket, the backstage upload program meeting of disk management system is cached
According to the record of brgw-status-file, check the data for unloading tape puts dirty mark, and will not also upload CD server
Data slicer file uploads to CD server.Finally, the background service program, the caching that can discharge all unloaded tapes circle or whirl in the air
Between.
When creating a virtual tape, virtual tape of the invention to CD casket maps distribution module, to select to have divided
Match somebody with somebody and free space is more than or equal to the CD casket of a GB on Tape, so can be to avoid one for distributing virtual tape
Tape is assigned to the situation of multiple CD caskets, thus further avoid from CD server recover data when, due to switching CD casket make
Into the reading I/O more than 60 seconds postpone, such readings I/O delay can cause the data recovery failure of backup software, this is also
Invention key problems-solving.
It should be appreciated that the part that this specification does not elaborate belongs to the prior art.
It should be appreciated that the above-mentioned description for preferred embodiment is more detailed, can not therefore be considered to this
The limitation of invention patent protection scope, those of ordinary skill in the art are not departing from power of the present invention under the enlightenment of the present invention
Profit is required under protected ambit, can also be made replacement or deformation, be each fallen within protection scope of the present invention, this hair
It is bright scope is claimed to be determined by the appended claims.
Claims (4)
- A kind of 1. VTL systems based on blue-ray storage equipment, it is characterised in that:Including VTL servers, blue light gateway server, Blu-ray Disc base management system;The VTL servers, the backup services based on TL agreements are externally provided by iSCSI/FC, then according to different requests Different function modules is called, and implementing result and data are returned to backup server;The blue light gateway server, supports multiple interfaces in the form of plug-in unit, and gateway uses and manages Blu-ray Disc storehouse, at the same time Cache management and data read and write interface are provided;The cache management, coordinates to complete, VTL servers according to TL semantemes and gateway Pass through cache management interface requests read buffer, synchronization/deletion write buffer;The Blu-ray Disc base management system provides data access and management interface.
- 2. the VTL systems according to claim 1 based on blue-ray storage equipment, it is characterised in that:Multiple interfaces include text Part system interface, block interface and object interface.
- A kind of 3. VTL methods based on blue-ray storage equipment, it is characterised in that:Metadata and user data correspond to different storages Mode, is stored in different CDs respectively, respectively using different cache management strategies;For metadata, the specific implementation of the method includes following principle:Principle A1:When BD-VTL starts, corresponding metadata is loaded into caching, resident caching before BD-VTL is offline;Principle A2:Metadata read-write operation is all completed in the buffer;Principle A3:Metadata data amount is when reaching the capacity of CD casket before offline, backwash write buffer, to ensure first number According to reliability;For user data, the specific implementation of the method includes following principle:Principle B1:When loading tape to tape drive, the metadata of tape is loaded to caching;Principle B2:For forward/backward operation, according to tape location, give up the caching before the position, and cache the position it Caching afterwards;Principle B3:For, to the data cached of tape end, being operated after user's valid data using TRIM by it from caching Delete;Principle B4:When tape is unloaded from tape drive, the metadata and user data of backwash tape.
- 4. the VTL methods according to claim 3 based on blue-ray storage equipment, it is characterised in that:The metadata and use User data, is all fragmented preservation, uploads to blue-ray storage system by parallel form, and download in caching.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710449246.0A CN107220007B (en) | 2017-06-14 | 2017-06-14 | A kind of VTL system and method based on blue-ray storage equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710449246.0A CN107220007B (en) | 2017-06-14 | 2017-06-14 | A kind of VTL system and method based on blue-ray storage equipment |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107220007A CN107220007A (en) | 2017-09-29 |
CN107220007B true CN107220007B (en) | 2018-04-20 |
Family
ID=59948347
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710449246.0A Expired - Fee Related CN107220007B (en) | 2017-06-14 | 2017-06-14 | A kind of VTL system and method based on blue-ray storage equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107220007B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109271241B (en) * | 2018-08-20 | 2021-10-29 | 赛凡信息科技(厦门)有限公司 | Method for scheduling operation tasks of blue-ray equipment |
CN110321076B (en) * | 2019-06-14 | 2022-12-27 | 武汉光忆科技有限公司 | Virtual tape library system based on optical disk library and management method thereof |
CN113377497B (en) * | 2021-06-08 | 2023-11-28 | 湖北第二师范学院 | Service development system based on service virtualization |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1781072A (en) * | 2003-02-05 | 2006-05-31 | 迪利根特技术公司 | Tape storage emulation for open systems environments |
CN101325610A (en) * | 2008-07-30 | 2008-12-17 | 杭州华三通信技术有限公司 | Virtual tape library backup system and magnetic disk power supply control method |
CN105022680A (en) * | 2015-08-05 | 2015-11-04 | 浪潮(北京)电子信息产业有限公司 | Data backup method and device |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2014219750A (en) * | 2013-05-02 | 2014-11-20 | 富士通株式会社 | Virtual library controller and control method |
-
2017
- 2017-06-14 CN CN201710449246.0A patent/CN107220007B/en not_active Expired - Fee Related
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1781072A (en) * | 2003-02-05 | 2006-05-31 | 迪利根特技术公司 | Tape storage emulation for open systems environments |
CN101325610A (en) * | 2008-07-30 | 2008-12-17 | 杭州华三通信技术有限公司 | Virtual tape library backup system and magnetic disk power supply control method |
CN105022680A (en) * | 2015-08-05 | 2015-11-04 | 浪潮(北京)电子信息产业有限公司 | Data backup method and device |
Also Published As
Publication number | Publication date |
---|---|
CN107220007A (en) | 2017-09-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10067712B2 (en) | Virtual disk drive system and method | |
US9489150B2 (en) | System and method for transferring data between different raid data storage types for current data and replay data | |
US8285762B2 (en) | Migration of metadata and storage management of data in a first storage environment to a second storage environment | |
US9317375B1 (en) | Managing cache backup and restore for continuous data replication and protection | |
US7840657B2 (en) | Method and apparatus for power-managing storage devices in a storage pool | |
US8694729B2 (en) | Methods for managing ownership of redundant data and systems thereof | |
US8539165B2 (en) | Methods for managing ownership of redundant data and systems thereof | |
US20060277226A1 (en) | System and method for controlling storage of electronic files | |
US20120079223A1 (en) | Methods for managing ownership of redundant data and systems thereof | |
CN103999058A (en) | Tape drive system server | |
CN107220007B (en) | A kind of VTL system and method based on blue-ray storage equipment | |
US20050273650A1 (en) | Systems and methods for backing up computer data to disk medium | |
CN107291386B (en) | A kind of cache management system and method for blue light VTL gateways | |
EP2385458A2 (en) | Virtual disk drive system and method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180420 |
|
CF01 | Termination of patent right due to non-payment of annual fee |