CN103049391B - Data processing method and equipment - Google Patents
Data processing method and equipment Download PDFInfo
- Publication number
- CN103049391B CN103049391B CN201210590148.6A CN201210590148A CN103049391B CN 103049391 B CN103049391 B CN 103049391B CN 201210590148 A CN201210590148 A CN 201210590148A CN 103049391 B CN103049391 B CN 103049391B
- Authority
- CN
- China
- Prior art keywords
- tape
- index
- data block
- information
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses a data processing technology which can reduce data volume during exporting of deduplication storage media to a tape library and save the storage space of the tape library. The method includes: sending data blocks in the deduplication storage media and storage information of the data blocks to a first target tape; generating a mapping relation between an index of each data block and the first target tape; and sending the indexes in the deduplication storage media and storage information of the indexes to a second target tape, wherein the storage information of the indexes includes storage address ranges of the indexes.
Description
Technical field
The present invention relates to technology of data copy field, relate in particular to a kind of data processing method and equipment.
Background technology
In order to prevent the loss of data, people can back up system or information conventionally, thereby standby data are saved in backup disk.But the deposit data that inevitably there will be repetition when data are backed up is to the phenomenon of backup disk, for example full backup repeatedly, or increase backup etc.
In order to reduce in backup disk, store the space waste that repeating data causes, industry disk manufacturer has proposed a kind of data de-duplication technology, utilizes data de-duplication technology can delete the repeating data in backup disk.Data de-duplication technology is specially: by backuping to multiple data blocks that the Divide File of heavily deleting in storage medium is different sizes, then adopt feature extraction algorithm to calculate respectively the index of each data block.The index calculating is mated, if there is index identical, illustrate that the data block that this index is corresponding is repetition.Thereby can only retain a data block in repeating data piece, and the data block of deleting other, and retain the index all repeating.Finally in heavily deleting storage medium, this file is retained as the set of index and the set of unduplicated data block, and wherein each index can point to a unique different data block.Thereby delete the redundant data of bringing due to the standby data of multiple standby or increasing entirely, save disk space.
But; because the cost of disk itself is higher; the data of heavily deleting in storage medium can export in the tape library that cost is lower conventionally; when controller is when heavily deleting Backup Data in storage medium and read out and send to tape library; data after heavily deleting need to be reduced to the non-heavy data of deleting; recover deleted repeating data piece, obtain complete file, and then complete file is sent to tape library.Thereby cause, heavily to delete the data volume that storage medium derives larger, causes the problem of the waste of storage space of tape library.
Summary of the invention
Embodiments of the invention provide a kind of data processing method, data layout and equipment, can reduce the data volume of heavily deleting storage medium and export to tape library, save the storage space of tape library.
For achieving the above object, embodiments of the invention adopt following technical scheme:
A first aspect of the present invention provides a kind of data processing method, is applied to the controller of heavily deleting storage medium, the described heavy index that stores multiple data blocks and data block in storage medium of deleting, corresponding at least one index of each data block; The method comprises:
Described controller sends to first object tape by heavily deleting data block in storage medium and the storage information of described data block, and the storage packets of information of wherein said data block is containing the memory address scope of described data block;
Generate the mapping relations of index and the described first object tape of described data block,, and be kept at and describedly heavyly delete in storage medium or send to the second target tape;
By described, heavyly delete index in storage medium and the storage information of described index sends to described the second target tape, the storage packets of information of wherein said index is containing the memory address scope of described index.
In conjunction with a first aspect of the present invention, in the possible implementation of another kind, before the storage information of data block and described data block is sent to first object tape, also comprise:
Obtain the capacity information of each tape in tape library;
Determine the size of data block to be sent and the size of index;
According to the size of the capacity information of described each tape and described data block, determine that first object tape determines described first object tape, according to the size of the capacity information of described each tape and described index, determine described the second target tape, wherein, described first object tape comprises one or more tapes, the capacity summation of described first object tape is more than or equal to the size of described data block, described the second target tape comprises one or more tapes, and the capacity summation of described the second target tape is more than or equal to the size of described index.
In conjunction with a first aspect of the present invention and above-mentioned possible implementation, in the possible implementation of another kind, described method also comprises:
The indication of described the second target tape is read in reception, and reads described the second target tape according to described indication;
Obtain the memory address scope of the index of storing in described the second target tape, and read index from the memory address scope of described index;
According to the mapping relations of described index and described first object tape, determine the first object tape of the data block storage that described index is corresponding;
Obtain the memory address scope of the data block of storing in described first object tape, and from the memory address scope of described data block read block.
In conjunction with a first aspect of the present invention and above-mentioned possible implementation, in the possible implementation of another kind, determine described first object tape determining first object tape according to the size of the capacity information of described each tape and described data block, after determining described the second target tape according to the size of the capacity information of described each tape and described index, described method also comprises:
Preserve the information of described first object tape and the information of described the second target tape;
In reception, read the indication of described the second target tape, and before reading described the second target tape according to described indication, described method also comprises:
According to the information of the information of described first object tape and described the second target tape, determine that described first object tape and described the second target tape can use.
A second aspect of the present invention, provides a kind of controller, comprising:
Read module, for from heavily deleting storage medium read block; From described heavy deleting, storage medium, read index;
Sending module, for data block that described read module is read and the storage information of described data block, send to first object tape, the storage information of the index that described read module is read and described index sends to the second target tape, the storage packets of information of wherein said data block is containing the memory address scope of described data block, and the storage packets of information of described index is containing the memory address scope of described index;
Generation module, for generating the mapping relations of index and described first object tape of described data block, and is kept at and describedly heavyly deletes in storage medium or send to described the second target tape.
In conjunction with a second aspect of the present invention, in the possible implementation of another kind, this controller also comprises:
Acquisition module, for obtaining the capacity information of the each tape of tape library;
Determination module, for determining the size of data block and the size of index to be sent;
Tape determination module, for the capacity information of each tape that obtains according to described acquisition module and the size of described determination module established data piece, determine that first object tape determines described first object tape, the size of the definite index of the capacity information of the each tape obtaining according to described acquisition module and described determination module is determined described the second target tape, wherein, described first object tape comprises one or more tapes, the capacity summation of described first object tape is more than or equal to the size of described data block, described the second target tape comprises one or more tapes, the capacity summation of described the second target tape is more than or equal to the size of described index.
In conjunction with a second aspect of the present invention and above-mentioned possible implementation, in the possible implementation of another kind, this controller also comprises:
The first read module, receives the indication of reading described the second target tape, and reads described the second target tape according to described indication; Obtain the memory address scope of the index of storing in described the second target tape, and read index from the memory address scope of described index;
The second read module, for according to the mapping relations of described index and described first object tape, determines the first object tape of the data block storage that described index is corresponding; Obtain the memory address scope of the data block of storing in described first object tape, and from the memory address scope of described data block read block.
In conjunction with a second aspect of the present invention and above-mentioned possible implementation, in the possible implementation of another kind, this controller also comprises:
Preserve module, for determining that according to the size of the capacity information of described each tape and described data block first object tape determines described first object tape at described tape determination module, after determining described the second target tape according to the size of the capacity information of described each tape and described index, preserve the information of described first object tape and the information of described the second target tape;
Inspection module, for receiving the indication of reading described the second target tape at described the first read module, and before reading described the second target tape according to described indication, according to the information of the information of described first object tape and described the second target tape, determine that described first object tape and described the second target tape can use.
A third aspect of the present invention, provides a kind of data handling system, comprising:
Controller, for will heavily deleting the data block of storage medium and the storage information of described data block sends to first object tape, the storage packets of information of wherein said data block is containing the memory address scope of described data block; Generate the mapping relations of index and the described first object tape of described data block, and be kept at and describedly heavyly delete in storage medium or send to described the second target tape; By described, heavyly delete index in storage medium and the storage information of described index sends to the second target tape, the storage packets of information of wherein said index is containing the memory address scope of described index;
Heavily delete storage medium, the described heavy index that stores multiple data blocks and data block in storage medium of deleting, corresponding at least one index of each data block;
And tape library, described tape library comprises first object tape and the second target tape.
The data processing method that the embodiment of the present invention provides, data layout and equipment, by Backup Data is sent to first object tape and the second target tape with the form of data block and index, and the mapping relations of the generating indexes data block place tape corresponding with this index, with in prior art, by heavily deleting data, revert to tediously long complete data file and send to compared with tape library, with the form of heavily deleting data, send the data to tape library, saved and sent tediously long repeating data piece, both reduced and heavily deleted the data traffic between storage medium and tape library, also saved the storage space of tape library.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, to the accompanying drawing of required use in embodiment or description of the Prior Art be briefly described below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skills, do not paying under the prerequisite of creative work, can also obtain according to these accompanying drawings other accompanying drawing.
A kind of data processing method process flow diagram of Fig. 1 for providing in one embodiment of the invention;
A kind of heavy process flow diagram of deleting data from heavily delete storage medium export to tape library of Fig. 2 for providing in another embodiment of the present invention;
Fig. 3 imports to the process flow diagram of heavily deleting storage medium for a kind of heavy data of deleting that provide in another embodiment of the present invention from tape library;
The composition schematic diagram that Fig. 4 is a kind of data layout based on data de-duplication of providing in another embodiment of the present invention;
The composition schematic diagram that Fig. 5 is a kind of controller of providing in another embodiment of the present invention;
The composition schematic diagram that Fig. 6 is a kind of controller of providing in another embodiment of the present invention;
A kind of data handling system composition schematic diagram of Fig. 7 for providing in another embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the present invention's part embodiment, rather than whole embodiment.Based on the embodiment in the present invention, those of ordinary skills, not making the every other embodiment obtaining under creative work prerequisite, belong to the scope of protection of the invention.
One embodiment of the invention provides a kind of data processing method, and as shown in Figure 1, the method comprises:
101, controller sends to first object tape by heavily deleting data block in storage medium and the storage information of described data block, and the storage packets of information of wherein said data block is containing the memory address scope of described data block.
Wherein, described controller is the controller of heavily deleting storage medium, can be control system or other control modules of computing machine, storage array.The described heavy storage medium of deleting refers to the disk that adopts data de-duplication technology save data, and in heavily deleting storage medium, the form of save data is attached most importance to and deleted data.The described heavy index that stores multiple data blocks and data block in storage medium of deleting, corresponding at least one index of each data block.Wherein, heavy to delete data block described in storage medium be unique unduplicated described.Heavily delete data for file is saved as to multiple index, and data block corresponding to each index, and delete the data block repeating, make an only corresponding unique unduplicated data block of multiple identical index.
In the present embodiment, heavily delete the Backup Data that stores data block and index composition in storage medium, controller can read data block and the index heavily deleted in storage medium, and Backup Data is sent to tape library according to the form of index and data block, so that tape library is preserved Backup Data according to the form of index and data block, tape library comprises first object tape and the second target tape.The described heavy storage medium of deleting can be disk, floppy disk, hard disk or solid state hard disc etc., and the embodiment of the present invention does not limit this.Described tape library can be physical tape storehouse, can be also VTL.Wherein, if described tape library is physical tape storehouse, the data block and the index that are written in physical tape storehouse are to store according to the data block of controller transmission and the organizational form of index.If described tape library is VTL, because the dividing mode of tape or the organizational form of data of the VTL of different vendor may be different, in order to guarantee the consistance of data, the organizational form of the data that read from VTL must be identical with the organizational form of data that is written to VTL.
102, generate the mapping relations of index and the described first object tape of described data block.
Wherein, the mapping relations of the index of described data block and described first object tape, are the mapping relations of the data block place tape that described index is corresponding with described index, for determining the memory location of the data block that index is corresponding.
Optionally, generate the mapping relations of index and the described first object tape of described data block, described mapping relations can be kept to described heavy deleting in storage medium, when reading index, just can know the data block place tape that this index is corresponding according to described mapping relations.
Optionally, generate the mapping relations of index and the described first object tape of described data block, a part for the storage information using the mapping relations of described index and described first object tape as index sends in the second target tape together with index; Or, the mapping relations of described first object tape can be not yet as a part for the storage information of described index, be saved in the second target tape.Similarly, just can be according to the mapping relations in the storage information of this index when reading index, determine the data block place tape that this index is corresponding.
103, by described, heavyly delete index in storage medium and the storage information of described index sends to the second target tape, the storage packets of information of wherein said index is containing the memory address scope of described index.
It should be noted that there is no fixing sequencing between step 101,102 and 103, can adjust according to actual needs the sequencing of above step.For example, can first send the storage information of index and index, then send the address realm of data block and data block, finally generate the mapping relations between described index and described data block place tape.
The data processing method that the embodiment of the present invention provides, by Backup Data is sent to first object tape and the second target tape with the form of data block and index, and the mapping relations of the generating indexes data block place tape corresponding with this index, with in prior art, by heavily deleting data, revert to tediously long complete data file and send to compared with tape library, with the form of heavily deleting data, send the data to tape library, saved and sent tediously long repeating data piece, both reduced and heavily deleted the data traffic between storage medium and tape library, also saved the storage space of tape library.
Another embodiment of the present invention provides a kind of data processing method, and as shown in Figure 2, heavily deleting the flow process that storage medium exports to tape library by data can comprise:
201, controller and tape library connect, and obtain the magnetic tape information of tape library.
Wherein, after controller and tape library connect, carry out the initial setting up of tape library information, communicate with tape library, obtain the magnetic tape information in tape library.For example, described magnetic tape information can comprise: tape quantity, tape bar code (barcode) information, driver information of number etc.Wherein, the magnetic tape information of described tape library comprises the capacity information of each tape in tape library.
202,, according to data block to be sent and index, from described magnetic tape information, determine described first object tape and described the second target tape.
Wherein, described according to data block to be sent and index, from described magnetic tape information, determine described first object tape and described the second target tape, specifically comprise: determine the size of data block to be sent and the size of index; According to the size of the capacity information of described each tape and described data block, determine that first object tape determines described first object tape, according to the size of the capacity information of described each tape and described index, determine described the second target tape, wherein, described first object tape comprises one or more tapes, the capacity summation of described first object tape is more than or equal to the size of described data block, described the second target tape comprises one or more tapes, and the capacity summation of described the second target tape is more than or equal to the size of described index.
Wherein, the size of described data block is the required storage size taking of the described data block of storage, and the size of described index is the required size that takies storage space of the described index of storage.According to described data block to be sent and index, determine described first object tape and described the second target tape, comprise according to heavily deleting the data block of having preserved in storage medium and the data volume size of index, or the size of the data block that the needs of determining according to user are derived and the data volume of index is determined described first object tape and described the second target tape from described magnetic tape information.For example, the storage size of supposing the magnetic tape reel in tape library is 200MB, heavily delete the data block of preserving 100MB index and 1000MB in storage medium, can choose in tape library appoint magnetic tape reel as the second target tape, be used for preserving index, using other the 5 dish tapes in tape library as first object tape, for save data piece.
203, preserve the information of described first object tape and the information of described the second target tape.
Wherein, preserve the information of described first object tape and the information of described the second target tape can determine first object tape and the second target tape in step 202 after.For example, can preserve the bar code information of first object tape and the bar code information of the second target tape etc.Like this, when needs are during from tape library reading out data, can first judge whether the index that will read and first object tape and the second target tape at data block place connect, and can proper communication.
Optionally, also can when sending index and data block, upgrade the information of first object tape and the information of described the second target tape.For example, need the data block deriving to need polydisc tape could preserve, every during to a dish virgin tape transmission data block, the bar code information that just can add this virgin tape in the information of first object tape.Further, can also record which data block and send in any dish tape, thus the mapping relations of generated data piece and first object tape.Similarly, the mapping relations of all right generating indexes and the second target tape.Like this, while sending data block or index, all can upgrade these mapping relations at every turn.Wherein, mapping relations can identify and represent with the identification information of the sign of the sign of data block, index or tape, or also can directly adopt index as sign, and the embodiment of the present invention does not limit the representation of mapping relations.
204, by heavily deleting data block in storage medium and the storage information of described data block, send to first object tape, the storage packets of information of wherein said data block is containing the memory address scope of described data block.
Wherein, the address realm of described data block is for describing start address and the end address of described data block, concrete, the describing mode of the address realm of data block can be described for the mode that adopts start address and displacement, or can be also to adopt the mode of start address and end address to be described, or can also be that the number of data block and the big or small mode of each data block that adopts storage information to belong to same data block set is afterwards described, the embodiment of the present invention limit the specific descriptions mode of address realm.
Concrete, because data are to adopt binary mode to preserve, for in tape library to distinguishing between different pieces of information piece, can, when data block is sent to first object tape, to first object tape, send the memory address scope of this data block.For example, data block A, B and C are three different data blocks, data block A, B and C are rolled up to (Block Volume) as a data block, it is data block set, the descriptor of adding data block set above of the set that can form in these three data blocks, this descriptor can the quantity of data of description piece and the size of each data block.2 kilobyte that for example Block Volume information can be described after Block Volume information are first data blocks, be data block A, 3 kilobyte after data block A are second data blocks, i.e. data block B, 5 kilobyte after data block B are data block C, etc.Read after the data block set of A, B and C composition, can read the Block Volume information of next data block set, by that analogy.
205, generate the mapping relations of index and the described first object tape of described data block.
Wherein, generate the mapping relations of index and the described first object tape of described data block, described mapping relations can be kept to described heavy deleting in storage medium, when reading index, just can know the data block place tape that this index is corresponding according to described mapping relations.Or, generating the mapping relations of index and the described first object tape of described data block, a part for the storage information using the mapping relations of described index and described first object tape as index sends in the second target tape together with index; Similarly, just can be according to the mapping relations in the storage information of this index when reading index, determine the data block place tape that this index is corresponding.
For example, the mapping relations of described index and described first object tape can be the mapping relations table of the bar code information composition of index and first object tape.Or the mapping relations of described index and described first object tape can be also the index stores information binding together with index, the bar code information that comprises first object tape in the storage information of index.Or, the mapping relations of described index and described first object tape can be included in the storage information of index, but be stored in the second target tape with described index binding, while reading index, also can read the mapping relations of described index and described first object tape.
206, the storage information of the index of described data block and described index is sent to the second target tape, the storage packets of information of wherein said index is containing the memory address scope of described index.
Wherein, with the transmission form class of the data block in step 204 seemingly, the address realm of described index is for describing start address and the end address of described index, concrete, the describing mode of the address realm of index can be described for the mode that adopts start address and displacement, or can be also to adopt the mode of start address and end address to be described, or can also be that the number of index and the big or small mode of each index that adopts storage information to belong to same data block set is afterwards described, the embodiment of the present invention does not limit the specific descriptions mode of address realm.
Concrete, in order to distinguish between to different index, can, when index being sent to the second target tape, to the second target tape, send the memory address scope of this index in the second target tape.For example, index a, b and c are three different index, using index a, b and c as an index volume (Index Volume), it is index set, the descriptor of adding index set above of the set that can form at these three index, this descriptor can be described the quantity of index and the size of each index.100 bytes that for example Index Volume information can be described after Index Volume information are first index, i.e. index a, and 200 bytes after index a are second index, i.e. index b, 300 bytes after index b are index c, etc.Read after the index set of a, b and c composition, can read the Index Volume information of next index set, by that analogy.
Further, the storage information of described index also comprises the mapping relations of described index and described first object tape.For the ease of the tape at data block place corresponding to recording indexes, can be when index being sent to the second target tape, not only to the second target tape, send the memory address scope of this index, also to the second target tape, send the disc information at the data block place that this index is corresponding.Still, take above-mentioned index a, b and c as example, the data block that index a, b and c are corresponding is respectively data block A, B and C.The descriptor of adding index set above of the set that index a, b and c can form at these three index as an index set (Index Volume), this descriptor can be described the quantity of index, the size of each index, and the magnetic tape information at data block place corresponding to each index.100 bytes that for example Index Volume information can be described after Index Volume information are first index, be index a, 200 bytes after index a are second index, be index b, 300 bytes after index b are index c, and these three index respectively data block of correspondence are all kept in tape 1.When reading the index set of a, b and c composition, just can read the data block in tape 1.
In the present embodiment, the flow process of heavily deleting data and export to from heavily deleting storage medium tape library has more than been described, corresponding, as shown in Figure 3, data are imported to from tape library to heavily to delete the flow process of storage medium as follows:
207, controller and tape library connect, and obtain the magnetic tape information of tape library.
Wherein, after controller and tape library connect, can communicate with tape library, obtain the current magnetic tape information having connected.For example, described magnetic tape information can comprise: tape quantity, tape bar code (barcode) information, driver information of number etc.The tape that magnetic tape information is corresponding can carry out proper communication.
208, receive the indication of reading described the second target tape and/or first object tape, determine the second target tape and/or the first object tape that need to read.
Wherein, the magnetic tape information having connected can be presented to user, by user, be selected the tape that need to read, or also can initiate read index and the data block in whole tapes by user.
209,, according to the information of the information of the magnetic tape information of tape library and described first object tape and described the second target tape, determine whether described first object tape and described the second target tape can be used; If described first object tape and described the second target tape are all available, perform step 210; If described first object tape is available when different with described the second target tape, do not carry out the action of reading of index and data block.
Wherein, the magnetic tape information getting in step 207 is mated with first object magnetic tape information and the second target magnetic tape information, if determine that first object tape and the second target tape all connect with controller, and connect availablely, can read the data in tape.Otherwise, if any one tape in first object tape or the second target tape is unavailable, can point out user's read error, no longer read action.
210, obtain the memory address scope of the index of storing in described the second target tape, and read described index from the memory address scope of described index.
Wherein, in step 206, the storage information of index and index is kept in the second target tape, here can from Index Volume information, obtain the memory address scope of index, thereby determine between the index after index volume information and how to divide.
Further, if the magnetic tape information at data block place corresponding index is also written in Index Volume information in step 206, can also from Index Volume information, obtain the mapping relations of index and described first object tape so here.
211,, according to the mapping relations of described index and described first object tape, determine the first object tape of the data block storage that described index is corresponding.
Wherein, can from the mapping relations table of index and the described first object tape preserved, inquire about the first object magnetic tape information that obtains the data block place that index is corresponding, also can from the storage information of index, obtain the first object magnetic tape information at the data block place that index is corresponding, thereby from first object tape, read the data block that index is corresponding.
212, obtain the memory address scope of the data block of storing in described first object tape, and from the memory address scope according to described data block read block.
Wherein, while data block being exported to tape library in step 204, in first object tape, write the memory address scope of data block.Here can read the memory address scope of the data in Block Volume information, thus the dividing condition of specified data piece, read block from first object tape.
The data processing method that the embodiment of the present invention provides, by Backup Data is sent to first object tape and the second target tape with the form of data block and index, and the mapping relations of the generating indexes data block place tape corresponding with this index, with in prior art, by heavily deleting data, revert to tediously long complete data file and send to compared with tape library, with the form of heavily deleting data, send the data to tape library, saved and sent tediously long repeating data piece, both reduced and heavily deleted the data traffic between storage medium and tape library, also saved the storage space of tape library.
And, heavily delete storage medium from tape library during reading out data, also can read respectively index and data block corresponding to index according to the storage information of the storage information of index and data block, reduce the data volume that data are read out from tape library, improve the efficiency of data processing.
Another embodiment of the present invention provides a kind of data layout, and described data layout is heavily to delete data are exported to after tape library in storage medium, the data layout of preserving in described tape library, and tape library can be with this data layout save data.Concrete, this data layout comprises: be stored in index part in the second target tape, be stored in the data block portions in first object tape and be stored in the second target tape or described heavy mapping relations part of deleting in storage medium.
Wherein, described index part comprises: memory address scope and the index of index.One or more index can form index set, and the memory address scope of index can be used as the head of index set, describe the storage condition of each index in index set.
Wherein, described data block portions comprises: memory address scope and the data block of data block.One or more data blocks can composition data set of blocks, and the memory address scope of data block can be used as the head of data block set, the storage condition of each data block in data of description set of blocks.
Wherein, described mapping relations part comprises: the mapping relations between the tape at the data block place that described index and described index are corresponding.These mapping relations can, for the mapping relations table between the identification information of the tape that comprises index and data block place corresponding to index, be kept at and heavily delete in storage medium or be kept in tape library, or can also be kept in third party's storage medium.
Further, as shown in Figure 4, in described index part, can comprise described mapping relations part.Concrete, when heavily deleting index in storage medium and write tape, write the storage information of this index simultaneously, wherein said storage information comprises the magnetic tape information at the memory location of index and data block place corresponding to index.The magnetic tape information at the data block place that index is corresponding can also be described at the head of index set like this.
The data layout that the embodiment of the present invention provides, by the memory location of identification index and the memory location of data block, and preserve the magnetic tape information at the data block place that index is corresponding, compared with the data layout of tediously long complete file in prior art, can index and the form save data of data block corresponding to index, make not support that the tape library of data de-duplication technology can be heavily to delete the form save data of data, both reduced and heavily deleted the data traffic between storage medium and tape library, also saved the storage space of tape library.
Another embodiment of the present invention provides a kind of controller, is applied to the controller of heavily deleting storage medium, the described heavy index that stores multiple data blocks and data block in storage medium of deleting, corresponding at least one index of each data block; As shown in Figure 5, this controller comprises: read module 301, sending module 302, generation module 303.
Read module 301, for from heavily deleting storage medium read block; From described heavy deleting, storage medium, read index;
Sending module 302, for data block that described read module 301 is read and the storage information of described data block, send to first object tape, the index that described read module 301 is read and the storage information of described index send to the second target tape, the storage packets of information of wherein said data block is containing the memory address scope of described data block, and the storage packets of information of described index is containing the memory address scope of described index;
Generation module 303, for generating the mapping relations of index and described first object tape of described data block.
Further, described generation module 303 also for: generate the mapping relations of index and the described first object tape of described data block, and be kept at described heavy deleting in storage medium.
Further, described generation module 303 also for: generate the mapping relations of index and the described first object tape of described data block, and send to described the second target tape.
Further, this controller also comprises: acquisition of information module 304, determination module 305, tape determination module 306.
Acquisition of information module 304, for obtaining the capacity information of the each tape of tape library;
Determination module 305, for determining the size of data block and the size of index to be sent;
Tape determination module 306, for the capacity information of each tape that obtains according to described acquisition of information module 304 and the size of described determination module 305 established data pieces, determine that first object tape determines described first object tape, the size of the definite index of the capacity information of the each tape obtaining according to described acquisition of information module 304 and described determination module 35 is determined described the second target tape, wherein, described first object tape comprises one or more tapes, the capacity summation of described first object tape is more than or equal to the size of described data block, described the second target tape comprises one or more tapes, the capacity summation of described the second target tape is more than or equal to the size of described index.
Further, this controller also comprises: the first read module 307, the second read module 308.
The first read module 307, receives the indication of reading described the second target tape, and reads described the second target tape according to described indication; Obtain the memory address scope of the index of storing in described the second target tape, and read index from the memory address scope of described index;
The second read module 308, for according to the mapping relations of described index and described first object tape, determines the first object tape of the data block storage that described index is corresponding; Obtain the memory address scope of the data block of storing in described first object tape, and from the memory address scope of described data block read block.
Further, this controller also comprises: preserve module 309, inspection module 310.
Preserve module 309, for determining that according to the size of the capacity information of described each tape and described data block first object tape determines described first object tape at described tape determination module, after determining described the second target tape according to the size of the capacity information of described each tape and described index, preserve the information of described first object tape and the information of described the second target tape;
Inspection module 310, for receiving the indication of reading described the second target tape at described the first read module 309, and before reading described the second target tape according to described indication, according to the information of the information of described first object tape and described the second target tape, determine that described first object tape and described the second target tape can use.
It should be noted that, the corresponding content of the specific descriptions of part of module in can reference method embodiment in the embodiment of the present invention, the embodiment of the present invention is no longer described in detail here.
The controller that the embodiment of the present invention provides, by Backup Data is sent to first object tape and the second target tape with the form of data block and index, and the mapping relations of the generating indexes data block place tape corresponding with this index, with in prior art, by heavily deleting data, revert to tediously long complete data file and send to compared with tape library, with the form of heavily deleting data, send the data to tape library, saved and sent tediously long repeating data piece, both reduced and heavily deleted the data traffic between storage medium and tape library, also saved the storage space of tape library.
Invent another embodiment a kind of controller is provided, be applied to the controller of heavily deleting storage medium, the described heavy index that stores multiple data blocks and data block in storage medium of deleting, corresponding at least one index of each data block; As shown in Figure 6, this controller comprises: transmitter 41, processor 42.
Transmitter 41, for will heavily deleting the data block of storage medium and the storage information of described data block sends to first object tape, the storage packets of information of wherein said data block is containing the memory address scope of described data block;
Processor 42, for generating the mapping relations of index and described first object tape of described data block;
Described transmitter 41, also, for heavyly deleting the index of storage medium and the storage information of described index sends to the second target tape by described, the storage packets of information of wherein said index is containing the memory address scope of described index.
Further, described transmitter 41 also for, the index of the described data block that described processor 42 is generated and the mapping relations of described first object tape are kept at described heavy deleting in storage medium.
Further, described transmitter 41 also for, the index of the described data block that described processor 42 is generated and the mapping relations of described first object tape send to described the second target tape.
Further, this controller also comprises: receiver 43.
Receiver 43, for obtaining the capacity information of the each tape of tape library;
Described processor 42, also for determining the size of data block and the size of index to be sent; According to the size of the capacity information of described each tape and described data block, determine that first object tape determines described first object tape, according to the size of the capacity information of described each tape and described index, determine described the second target tape, wherein, described first object tape comprises one or more tapes, the capacity summation of described first object tape is more than or equal to the size of described data block, described the second target tape comprises one or more tapes, and the capacity summation of described the second target tape is more than or equal to the size of described index.
Further, described receiver 43, also for receiving the indication of reading described the second target tape, and reads described the second target tape according to described indication; Obtain the memory address scope of the index of storing in described the second target tape, and read index from the memory address scope of described index;
Described processor 42, also, for according to the mapping relations of described index and described first object tape, determines the first object tape of the data block storage that described index is corresponding;
Described receiver 43, also for obtaining the memory address scope of the data block that described first object tape stores, and from the memory address scope of described data block read block.
Further, this controller also comprises: storer 44.
Described storer 44, for determining that according to the size of the capacity information of described each tape and described data block first object tape determines described first object tape, after determining described the second target tape according to the size of the capacity information of described each tape and described index, preserve the information of described first object tape and the information of described the second target tape;
Described processor 42, also for receiving the indication of reading described the second target tape at described receiver 43, and before reading described the second target tape according to described indication, according to the information of first object tape and the information of described the second target tape of storage in described storer 44, determine that described first object tape and described the second target tape can use.
The controller that the embodiment of the present invention provides, by Backup Data is sent to first object tape and the second target tape with the form of data block and index, and the mapping relations of the generating indexes data block place tape corresponding with this index, with in prior art, by heavily deleting data, revert to tediously long complete data file and send to compared with tape library, with the form of heavily deleting data, send the data to tape library, saved and sent tediously long repeating data piece, both reduced and heavily deleted the data traffic between storage medium and tape library, also saved the storage space of tape library.
Invent another embodiment a kind of data handling system is provided, as shown in Figure 7, comprising: controller 51, heavily delete storage medium 52 and tape library 53.
Wherein, described controller 51, for will heavily deleting the data block of storage medium 52 and the storage information of described data block and send to the first object tape of tape library 53, the storage packets of information of wherein said data block is containing the memory address scope of described data block; Generate the mapping relations of index and the described first object tape of described data block; The the second target tape that described heavy storage information of deleting index in storage medium 52 and described index is sent to tape library 53, the storage packets of information of wherein said index is containing the memory address scope of described index.
The described heavy index that stores multiple data blocks and data block in storage medium 52 of deleting, corresponding at least one index of each data block;
Described tape library 53 comprises first object tape and the second target tape.
The data handling system that the embodiment of the present invention provides, by Backup Data is sent to first object tape and the second target tape with the form of data block and index, and the mapping relations of the generating indexes data block place tape corresponding with this index, with in prior art, by heavily deleting data, revert to tediously long complete data file and send to compared with tape library, with the form of heavily deleting data, send the data to tape library, saved and sent tediously long repeating data piece, both reduced and heavily deleted the data traffic between storage medium and tape library, also saved the storage space of tape library.
Through the above description of the embodiments, those skilled in the art can be well understood to the mode that the present invention can add essential common hardware by software and realize, and can certainly pass through hardware, but in a lot of situation, the former is better embodiment.Based on such understanding, the part that technical scheme of the present invention contributes to prior art in essence in other words can embody with the form of software product, this computer software product is stored in the storage medium can read, as the floppy disk of computing machine, hard disk or CD etc., comprise that some instructions are in order to make a computer equipment (can be personal computer, server, or the network equipment etc.) carry out the method described in each embodiment of the present invention.
The above; be only the specific embodiment of the present invention, but protection scope of the present invention is not limited to this, any be familiar with those skilled in the art the present invention disclose technical scope in; can expect easily changing or replacing, within all should being encompassed in protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection domain of described claim.
Claims (9)
1. a data processing method, is applied to the controller of heavily deleting storage medium, the described heavy index that stores multiple data blocks and data block in storage medium of deleting, corresponding at least one index of each data block; It is characterized in that, the method comprises:
Described controller sends to first object tape by heavily deleting data block in storage medium and the storage information of described data block, and the storage packets of information of wherein said data block is containing the memory address scope of described data block;
Generate the mapping relations of index and the described first object tape of described data block, and be kept at and describedly heavyly delete in storage medium or send to the second target tape;
By described, heavyly delete index in storage medium and the storage information of described index sends to described the second target tape, the storage packets of information of wherein said index is containing the memory address scope of described index.
2. data processing method according to claim 1, is characterized in that, before the storage information of data block and described data block is sent to first object tape, also comprises:
Obtain the capacity information of each tape in tape library;
Determine the size of data block to be sent and the size of index;
According to the size of the capacity information of described each tape and described data block, determine that first object tape determines described first object tape, according to the size of the capacity information of described each tape and described index, determine described the second target tape, wherein, described first object tape comprises one or more tapes, the capacity summation of described first object tape is more than or equal to the size of described data block, described the second target tape comprises one or more tapes, and the capacity summation of described the second target tape is more than or equal to the size of described index.
3. data processing method according to claim 2, is characterized in that, described method also comprises:
The indication of described the second target tape is read in reception, and reads described the second target tape according to described indication;
Obtain the memory address scope of the index of storing in described the second target tape, and read index from the memory address scope of described index;
According to the mapping relations of described index and described first object tape, determine the first object tape of the data block storage that described index is corresponding;
Obtain the memory address scope of the data block of storing in described first object tape, and from the memory address scope of described data block read block.
4. data processing method according to claim 3, is characterized in that,
Determine described first object tape determining first object tape according to the size of the capacity information of described each tape and described data block, after determining described the second target tape according to the size of the capacity information of described each tape and described index, described method also comprises:
Preserve the information of described first object tape and the information of described the second target tape;
In reception, read the indication of described the second target tape, and before reading described the second target tape according to described indication, described method also comprises:
According to the information of the information of described first object tape and described the second target tape, determine that described first object tape and described the second target tape can use.
5. a controller, is characterized in that, comprising:
Read module, for from heavily deleting storage medium read block; From described heavy deleting, storage medium, read index;
Sending module, for data block that described read module is read and the storage information of described data block, send to first object tape, the storage information of the index that described read module is read and described index sends to the second target tape, the storage packets of information of wherein said data block is containing the memory address scope of described data block, and the storage packets of information of described index is containing the memory address scope of described index;
Generation module, for generating the mapping relations of index and described first object tape of described data block, and is kept at and describedly heavyly deletes in storage medium or send to described the second target tape.
6. controller according to claim 5, is characterized in that, also comprises:
Acquisition of information module, for obtaining the capacity information of the each tape of tape library;
Determination module, for determining the size of data block and the size of index to be sent;
Tape determination module, for the capacity information of each tape that obtains according to described acquisition of information module and the size of described determination module established data piece, determine that first object tape determines described first object tape, the size of the definite index of the capacity information of the each tape obtaining according to described acquisition module and described determination module is determined described the second target tape, wherein, described first object tape comprises one or more tapes, the capacity summation of described first object tape is more than or equal to the size of described data block, described the second target tape comprises one or more tapes, the capacity sum total of described the second target tape is more than or equal to the size of described index.
7. controller according to claim 6, is characterized in that, also comprises:
The first read module, receives the indication of reading described the second target tape, and reads described the second target tape according to described indication; Obtain the memory address scope of the index of storing in described the second target tape, and read index from the memory address scope of described index;
The second read module, for according to the mapping relations of described index and described first object tape, determines the first object tape of the data block storage that described index is corresponding; Obtain the memory address scope of the data block of storing in described first object tape, and from the memory address scope of described data block read block.
8. controller according to claim 7, is characterized in that, also comprises:
Preserve module, for determining that according to the size of the capacity information of described each tape and described data block first object tape determines described first object tape at described tape determination module, after determining described the second target tape according to the size of the capacity information of described each tape and described index, preserve the information of described first object tape and the information of described the second target tape;
Inspection module, for receiving the indication of reading described the second target tape at described the first read module, and before reading described the second target tape according to described indication, according to the information of the information of described first object tape and described the second target tape, determine that described first object tape and described the second target tape can use.
9. a data handling system, is characterized in that, comprising:
Controller as described in any one in claim 5-8;
Heavily delete storage medium, the described heavy index that stores multiple data blocks and data block in storage medium of deleting, corresponding at least one index of each data block;
And tape library, described tape library comprises first object tape and the second target tape.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210590148.6A CN103049391B (en) | 2012-12-29 | 2012-12-29 | Data processing method and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210590148.6A CN103049391B (en) | 2012-12-29 | 2012-12-29 | Data processing method and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103049391A CN103049391A (en) | 2013-04-17 |
| CN103049391B true CN103049391B (en) | 2014-05-07 |
Family
ID=48062038
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210590148.6A Active CN103049391B (en) | 2012-12-29 | 2012-12-29 | Data processing method and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103049391B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103559106B (en) * | 2013-10-14 | 2016-03-02 | 华为技术有限公司 | A kind of backup method of data, Apparatus and system |
| CN103577565B (en) * | 2013-10-25 | 2017-01-04 | 华为技术有限公司 | A kind of method and apparatus that file is exported to tape |
| CN106713489A (en) * | 2017-01-17 | 2017-05-24 | 郑州云海信息技术有限公司 | Deduplication based synchronous remote copying system and method |
| CN106843760A (en) * | 2017-01-17 | 2017-06-13 | 郑州云海信息技术有限公司 | It is a kind of based on the asynchronous remote copy system deleted and method again |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8190836B1 (en) * | 2008-04-30 | 2012-05-29 | Network Appliance, Inc. | Saving multiple snapshots without duplicating common blocks to protect the entire contents of a volume |
| US8108638B2 (en) * | 2009-02-06 | 2012-01-31 | International Business Machines Corporation | Backup of deduplicated data |
| US9058298B2 (en) * | 2009-07-16 | 2015-06-16 | International Business Machines Corporation | Integrated approach for deduplicating data in a distributed environment that involves a source and a target |
| US8356017B2 (en) * | 2009-08-11 | 2013-01-15 | International Business Machines Corporation | Replication of deduplicated data |
| US9063666B2 (en) * | 2010-03-25 | 2015-06-23 | International Business Machines Corporation | File index, metadata storage, and file system management for magnetic tape |
-
2012
- 2012-12-29 CN CN201210590148.6A patent/CN103049391B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN103049391A (en) | 2013-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102929748B (en) | Data back up method and device | |
| CN103136243B (en) | File system duplicate removal method based on cloud storage and device | |
| CN107436725A (en) | A kind of data are write, read method, apparatus and distributed objects storage cluster | |
| CN104461390A (en) | Method and device for writing data into imbricate magnetic recording SMR hard disk | |
| CN103049391B (en) | Data processing method and equipment | |
| CN102821111A (en) | Real-time synchronizing method for file cloud storage | |
| CN103678143A (en) | File storage method and device and electronic equipment | |
| US10572335B2 (en) | Metadata recovery method and apparatus | |
| CN103324533A (en) | distributed data processing method, device and system | |
| CN103034592A (en) | Data processing method and device | |
| CN110324429A (en) | Backup method and back-up device based on Distributed Storage | |
| US10042570B2 (en) | Tape backup and restore in a disk storage environment with intelligent data placement | |
| CN102222033B (en) | A kind of method and device for preserving small computer system interface access error | |
| CN102142010A (en) | Method and equipment for inputting data to multimedia service database on embedded equipment | |
| CN105373339A (en) | Hard disk data copy method and system | |
| US9785517B2 (en) | Rebuilding damaged areas of a volume table using a volume data set | |
| CN101630332A (en) | Data storage management method, data storage management device and data storage management system | |
| CN103164172A (en) | Data flow storage method and device | |
| CN104238960A (en) | Hard disk formatting method, block data storage method based on hard disk and block data storage device based on hard disk | |
| CN112269543A (en) | Storage logical volume management method, device and related components | |
| CN105528344A (en) | Method and apparatus for determining media information of read data in storage device | |
| CN105354149A (en) | Memory data search method and apparatus | |
| CN101739308B (en) | Method for generating image file and storage system for image file | |
| CN105242985B (en) | Data recovery method and device | |
| CN109542674A (en) | Snapshot creation method, device, equipment and the medium of distributed system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |