CN107704336A - A kind of date storage method and device - Google Patents
A kind of date storage method and device Download PDFInfo
- Publication number
- CN107704336A CN107704336A CN201710900323.XA CN201710900323A CN107704336A CN 107704336 A CN107704336 A CN 107704336A CN 201710900323 A CN201710900323 A CN 201710900323A CN 107704336 A CN107704336 A CN 107704336A
- Authority
- CN
- China
- Prior art keywords
- data
- data fragmentation
- fragmentation
- burst
- group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
- G06F11/1044—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices with specific ECC/EDC distribution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
- G06F11/1048—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices using arrangements adapted for a specific error detection or correction feature
- G06F11/106—Correcting systematically all correctable errors, i.e. scrubbing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a kind of date storage method and device, is related to technical field of memory, can reduce network consumption, improves the performance of data storage.This method includes：The data fragmentation and local verification burst, the data fragmentation of first data for obtaining the first data include N number of data fragmentation, and N >=2, N are integer；To locally it be verified in the different storage devices that burst is stored in the first data center DC corresponding to J data fragmentation in N number of data fragmentation and the J data fragmentation, and remaining N J data fragmentations are temporarily stored in the first DC, and issued burst is locally verified corresponding to each group of data fragmentation in the N J data fragmentations and this group of data fragmentation in the 2nd DC corresponding with this group of data fragmentation, wherein, corresponding 2nd DC of each group of data fragmentation.
Description
Technical field
It is related to technical field of memory, more particularly to a kind of date storage method and device.
Background technology
In technical field of memory, in order to improve the security of data, while the cost of storage is reduced, can typically use and entangle
Code (Erasure Coding, EC) redundant fashion is deleted to store data.EC redundancies are exactly that the data that needs store are carried out
Burst obtains multiple data fragmentations, and the plurality of data fragmentation is verified, and obtains corresponding multiple verification bursts.Then will
Obtained data fragmentation and verification burst is stored in data center (Data Center, DC).When the partial data burst of storage
During damage, the partial data burst of the damage can be recovered by verifying in burst and unspoiled data fragmentation.
When storing burst (including data fragmentation and verification burst), there is two ways more conventional at present.One kind be with
Data fragmentation carries out EC redundant storages across the DC modes stored, i.e., multiple data fragmentations is stored separately in multiple DC.For example,
As shown in Figure 1, it is assumed that by taking 3 DC as an example.DC1 carries out EC redundancies after data 1 are received, to the plurality of data and obtains data point
Piece 1-6 and verification burst.Wherein verifying burst includes local verification burst 1-3 and global verification burst 1-3.Data fragmentation 1-
3 and corresponding local verification burst 1 be stored in DC1, the local burst 1 that verifies is used for any data in data fragmentation 1-3
When burst damages, the data fragmentation of the damage is reconstructed using unspoiled data fragmentation.Data fragmentation 4-6 and corresponding local
Verification burst 2 is stored in DC2, when local verification burst 2 is used for any data burst damage in data fragmentation 4-6, profit
The data fragmentation of the damage is reconstructed with unspoiled data fragmentation.Overall situation verification burst 1-3 and corresponding verification burst 3 store
It is local when verifying any global verification burst damage that burst 3 is used in global verification burst 1-3 in DC3, using not damaging
Bad global verification burst reconstructs the global verification burst of the damage.Overall situation verification burst 1-3 is used in data fragmentation 1-6
When data fragmentation within any 3 or 3 damages, unspoiled data fragmentation is utilized to reconstruct the data fragmentation of damage.At this
In kind storage mode, because each data fragmentation of data 1 is to write DC1 and DC2 by DC1, therefore data 1 are being read
During each data fragmentation, also to be read by DC1.That is, when reading data 1 every time, DC1 will read number from DC2
According to burst 4-6, thus due to across DC reading and caused by network consumption it is bigger.
Another kind is to carry out EC redundant storages by XOR (eXclusive OR) mode.For example, as shown in Fig. 2 with 3 numbers
Exemplified by center.The data fragmentation 7-9 of data 2 and corresponding local verification burst 4 are stored in DC1, the data fragmentation of data 3
A, B, C and corresponding local verification burst 5 are stored in DC2.Wherein, data 2 and data 3 are separate.Number is stored in DC3
It is stored according to 2 and the XOR bursts of data 3 and corresponding local verification burst 6 in DC2.Wherein, the sum of data storage 2 in DC3
The XOR bursts for including data fragmentation 7 and data fragmentation A according to 3 XOR bursts (are used in DC1 or DC2 failures, recover data point
Piece 7 or data fragmentation A), data fragmentation 8 and data fragmentation B XOR bursts (are used in DC1 or DC2 failures, recover data point
Piece 8 or data fragmentation B), data fragmentation 9 and data fragmentation C XOR bursts (are used in DC1 or DC2 failures, recover data point
Piece 9 or data fragmentation C).In this storage mode, because the data fragmentation of a data is stored entirely in a DC, because
This is when reading the data, and without carrying out data transmission between DC, network consumption during so as to read across DC is zero.But
When deleting data, such as, it is necessary to by data 2 or data 3 when deleting data 3, and new and data 2 carry out XOR data hair
DC3 is delivered to, to regenerate corresponding XOR bursts.Therefore, when deleting data, network consumption is caused due to across DC deletion
It is larger.
As can be seen that after with currently used data storage method data storage, or it is that network disappears when reading data
Consume it is larger, or be that network consumption is larger when deleting data.
The content of the invention
The application provides a kind of date storage method and device, can reduce network consumption, improve the performance of data storage.
In a first aspect, the application provides a kind of date storage method, including：Obtain data fragmentation and the local of the first data
Burst is verified, the data fragmentation of first data includes N number of data fragmentation, and N >=2, N are integer；By in N number of data fragmentation
The different storages that burst is stored in the first data center DC are locally verified corresponding to J data fragmentation and the J data fragmentation
In device, and remaining N-J data fragmentation is temporarily stored in the first DC, and will be every in the N-J data fragmentation
Burst is locally verified corresponding to one group of data fragmentation and this group of data fragmentation to issue in the 2nd DC corresponding with this group of data fragmentation；
Wherein, the N-J data fragmentation is divided at least one set, and each group of data fragmentation includes at least one data fragmentation, each group
Corresponding 2nd DC of data fragmentation, J >=1, J are integer.
In the date storage method that the application provides, due to storing all data point of the first data in the first DC
Piece, therefore all data fragmentations of the first data can be directly read when reading the first data from the first DC.And deleting
During the first data, each data fragmentation of the first data stored in the first DC and each 2nd DC can be directly deleted, is discharged
Corresponding memory space, without transmitting the data fragmentation of the first data between DC.The data storage provided using the application
Method, no matter read data or delete data, between the first DC and each 2nd DC will not transmitted data slice, therefore keep away
Exempted from due to across DC reading or delete and caused by network consumption, improve the performance of data storage.
Optionally, this method also includes：When total accounting is more than or equal to default first threshold value, delete this first
N-J data fragmentation of interim storage in DC, total accounting are the total of the data fragmentation occupancy of interim storage in the first DC
Capacity accounts for the ratio of the first DC capacity.
Optionally, delete in the first DC before N-J data fragmentation of interim storage, this method also includes：Judging should
Whether the access frequency of the first data, which is less than or equal to default first, accesses thresholding；Interim storage in the deletion the first DC
The N-J data fragmentation, including：If the access frequency is less than or equal to the first access thresholding, the first DC is deleted
The N-J data fragmentation of middle interim storage.
Based on the optional mode of above two, extra data space consumption can be reduced.
Optionally, if access frequency, which is less than or equal to first, accesses thresholding, the N-J of interim storage in the first DC is deleted
After individual data fragmentation, this method also includes：
Thresholding is accessed when the access frequency is more than or equal to default second, and total accounting is less than first threshold value
When, the N-J data fragmentation is copied in the 2nd DC stored from the N-J data fragmentation；By N-J data of copy point
Piece is temporarily stored in the first DC.
Based on the optional mode, the N-J data fragmentation can be faced when N-J data fragmentation turns into dsc data
When be stored in the first DC, so as to avoid due to frequently across DC reading and caused by network consumption it is excessive the problem of.
Optionally, after the data fragmentation and the local verification burst that obtain the first data, this method also includes：Judgement always accounts for
Than whether being more than or equal to default second threshold value；By J data fragmentation in N number of data fragmentation and the J data
Locally verified corresponding to burst in the different storage devices that burst is stored in the first data center DC, and by remaining N-J
Data fragmentation is temporarily stored in the first DC, and by each group of data fragmentation in the N-J data fragmentation and this group of data
Burst is locally verified corresponding to burst to issue in the 2nd DC corresponding with this group of data fragmentation, including：It is determined that total accounting is small
When second threshold value, will locally it be verified corresponding to J data fragmentation in N number of data fragmentation and the J data fragmentation
Burst is stored in the different storage devices in the first data center DC, and remaining N-J data fragmentation is temporarily stored in
In first DC, and will locally it be verified corresponding to each group of data fragmentation and this group of data fragmentation in the N-J data fragmentation
Burst is issued in the 2nd DC corresponding with this group of data fragmentation.
Based on the optional mode, data storage device can judge which kind of storage side performed based on the first DC total accounting
Formula, so as to improve the first DC storage performance.
Optionally, burst will locally be verified corresponding to J data fragmentation in N number of data fragmentation and the J data fragmentation
It is stored in the different storage devices in the first DC, and remaining N-J data fragmentation is temporarily stored in it in the first DC
Afterwards, this method also includes：Client-server reads the J data fragmentation of storage and being somebody's turn to do for interim storage from the first DC
N-J data fragmentation；The J data fragmentation and the N-J data fragmentation are combined as the first data by the client-server.
Second aspect, the application provide a kind of data storage device, including：Acquiring unit, for obtaining the first data
Data fragmentation and local verification burst, the data fragmentation of first data include N number of data fragmentation, and N >=2, N are integer；Storage
Unit, for will locally verify burst storage corresponding to J data fragmentation in N number of data fragmentation and the J data fragmentation
In different storage devices in the first data center DC, and remaining N-J data fragmentation is temporarily stored in the first DC
In, and issue burst is locally verified corresponding to each group of data fragmentation in the N-J data fragmentation and this group of data fragmentation
In the 2nd DC corresponding with this group of data fragmentation；Wherein, the N-J data fragmentation is divided at least one set, each group of data point
Piece includes at least one data fragmentation, and corresponding 2nd DC of each group of data fragmentation, J >=1, J are integer.
Optionally, data storage device also deletes unit：The deletion unit, in the memory cell by N number of data
Burst is locally verified corresponding to J data fragmentation and the J data fragmentation in burst to be stored in the first data center DC
In different storage devices, and remaining N-J data fragmentation is temporarily stored in the first DC, and by the N-J data
Locally verified corresponding to each group of data fragmentation and this group of data fragmentation in burst after burst is stored in the 2nd DC, when
When total accounting is more than or equal to default first threshold value, the N-J data fragmentation of interim storage in the first DC is deleted,
Total accounting is the ratio for the capacity that the total capacity that the data fragmentation of interim storage in the first DC takes accounts for the first DC.
Optionally, unit is deleted, is additionally operable to before N-J data fragmentation of interim storage in deleting the first DC, is judged
Whether the access frequency of the first data, which is less than or equal to default first, accesses thresholding；In the deletion element deletion the first DC
N-J data fragmentation of interim storage, is specifically included：If the access frequency is less than or equal to the first access thresholding, delete
Except the N-J data fragmentation of interim storage in the first DC.
Optionally, memory cell, be additionally operable to the interim storage in the DC of element deletion the first is deleted N-J data fragmentation it
Afterwards, when access frequency is more than or equal to the default second access thresholding, and total accounting is less than the first threshold value, from the N-J
The N-J data fragmentation is copied in the 2nd DC that individual data fragmentation is stored, and the N-J data fragmentation of copy is deposited temporarily
Storage is in the first DC.
Optionally, memory cell, it is additionally operable to after the data fragmentation that acquiring unit obtains the first data, judges total accounting
Whether default second threshold value is more than or equal to, and total accounting is that the data fragmentation of interim storage in the first DC takes
Total capacity accounts for the ratio of the first DC capacity；The memory cell is individual by J data fragmentation in N number of data fragmentation and the J
Locally verified corresponding to data fragmentation in the different storage devices that burst is stored in the first data center DC, and by remaining N-
J data fragmentation is temporarily stored in the first DC, and by each group of data fragmentation in the N-J data fragmentation and this group of number
Issue in the 2nd DC corresponding with this group of data fragmentation, specifically include according to burst is locally verified corresponding to burst：It is determined that this is total
When accounting is less than second threshold value, by sheet corresponding to J data fragmentation in N number of data fragmentation and the J data fragmentation
Ground verification burst is stored in the different storage devices in the first data center DC, and remaining N-J data fragmentation is interim
It is stored in the first DC, and by corresponding to each group of data fragmentation in the N-J data fragmentation and this group of data fragmentation
Ground verification burst is issued in the 2nd DC corresponding with this group of data fragmentation.
The technique effect for the data storage device that the application provides may refer to each of above-mentioned first aspect or first aspect
The technique effect of individual implementation, here is omitted.
The third aspect, the application provide a kind of client-server, including：Reading unit, for being filled in data storage
Burst will locally be verified corresponding to J data fragmentation in N number of data fragmentation and the J data fragmentation by, which putting, is stored in the first DC
Different storage devices in after, the J data fragmentation of storage and the N-J number of interim storage are read from the first DC
According to burst；Assembled unit, for the J data fragmentation that reads reading unit and the N-J data fragmentation be combined as this
One data.
Optionally, in first aspect and second aspect, the calculation of the first threshold value is as follows：X=(n/K/ (n+
M)) * h%；Wherein, X represents first threshold value, and K represents the 2nd DC number, and K >=1, K are integer, and m represents K the 2nd DC
The middle total quantity for verifying burst for allowing storage, n represent the total quantity for allowing the data fragmentation of storage in K the 2nd DC, h% tables
Show dsc data ratio.
Fourth aspect, this application provides a kind of data storage device, including processor, memory, communication interface；This is deposited
Reservoir, for storing computer executed instructions；The communication interface sends and received N number of data fragmentation and local verification burst；
The processor, it is connected by the bus with the memory and communication interface, when data storage device is run, the computing device
The computer executed instructions stored in the memory, to realize the various realization sides in first aspect performed by data storage device
The method of data storage described in formula.
The technique effect for the data storage device that the application provides may refer to each of above-mentioned first aspect or first aspect
The technique effect of individual implementation, here is omitted.
5th aspect, this application provides a kind of client-server, including processor, memory, communication interface；This is deposited
Reservoir, for storing computer executed instructions；The communication interface sends and received N number of data fragmentation；The processor, by this
Bus is connected with the memory and communication interface, when client-server is run, is stored in the computing device memory
Computer executed instructions, to realize the method for the data storage in first aspect involved by client-server.
6th aspect, the application also provide a kind of computer-readable storage medium, finger are stored with the computer-readable storage medium
Order, when run on a computer so that computer performs any optional mode institute of above-mentioned first aspect or first aspect
The method stated.
7th aspect, present invention also provides a kind of computer program product for including instruction, when it is transported on computers
During row so that computer performs the method described in any optional mode of above-mentioned first aspect or first aspect.
Brief description of the drawings
Fig. 1 is the schematic diagram that a kind of data fragmentation that prior art provides carries out EC redundant storages across the DC modes stored；
Fig. 2 is a kind of schematic diagram that EC redundant storages are carried out by XOR modes that prior art provides；
Fig. 3 is a kind of schematic diagram for distributed memory system that the application provides；
Fig. 4 is a kind of structural representation one for data storage device that the application provides；
Fig. 5 is a kind of flow chart one of the one embodiment for date storage method that the application provides；
Fig. 6 is the schematic diagram one for the data storage method that the application provides；
Fig. 7 is the schematic diagram two for the kind data storage method that the application provides；
Fig. 8 is a kind of flowchart 2 of the one embodiment for date storage method that the application provides；
Fig. 9 is the schematic diagram three for the data storage method that the application provides；
Figure 10 is the schematic diagram four for the kind data storage method that the application provides；
Figure 11 is a kind of flow chart 3 of the one embodiment for date storage method that the application provides；
Figure 12 is a kind of flow chart four of the one embodiment for date storage method that the application provides；
Figure 13 is a kind of flow chart of the one embodiment for method for reading data that the application provides；
Figure 14 A are a kind of structural representation two for data storage device that the application provides；
Figure 14 B are a kind of structural representation three for data storage device that the application provides；
Figure 14 C are a kind of structural representation four for data storage device that the application provides.
Embodiment
First, the terms " system " and " network " are often used interchangeably herein.The terms " and ", only
Only it is a kind of incidence relation for describing affiliated partner, expression may have three kinds of relations, for example, A and B, can be represented：Individually deposit
In A, while A and B be present, these three situations of individualism B.In addition, character "/" herein, typicallys represent forward-backward correlation object
It is a kind of relation of "or".When the application refers to ordinal numbers such as " first ", " second ", " the 3rd " or " the 4th ", except non-root
The meaning of its certain order of representation according to context, it is appreciated that being only that differentiation is used.
The date storage method that the application provides, can apply in distributed memory system.As shown in figure 3, it is this Shen
A kind of distributed memory system that please be provide, including multiple DC.Each DC is provided with corresponding data storage device, for performing
The date storage method that the application provides.Wherein, a DC is made up of multiple servers, and data storage device can be integrated in
On a server in the plurality of server, a server can also be newly increased in the DC as data storage device.
As shown in figure 4, a kind of data storage device provided for the application, including：Processor, memory, communication interface
And bus.Wherein, bus couple processor, memory and communication interface, and processor, memory and communication interface it
Between realize data transfer.For example, processor receives order by bus from communication interface, the order received is decrypted, according to
The order of decryption performs calculating or data processing.Memory can include program module and data module, be calculated for storing
Machine instructs and data cached, such as kernel (kernel), middleware (middleware), application programming interfaces (AP) and application
Deng.Program module can be made up of software, firmware, hardware or therein at least two.Communication interface may be coupled to and other
DC carries out information exchange, to realize the control to other DC.
As shown in figure 5, a kind of flow chart of one embodiment of the date storage method provided for the application, this method bag
Include following steps：
Step 501, data storage device obtains the data fragmentation of the first data and local verification burst, first data
Data fragmentation includes N number of data fragmentation, and N >=2, N are integer.
Wherein, the first data are the data for needing to carry out EC redundant storages.Data storage device can receive first
Burst processing can be carried out to first data, after data so as to obtain N number of data fragmentation.For example, it is assumed that first data
Length be n, first data can averagely be divided into N number of data fragmentation, the length of each data fragmentation by data processing equipment
Spend for N/n.After N number of data fragmentation is obtained, EC redundant computations can be carried out to N number of data fragmentation, obtain N number of number
According to multiple local verification bursts corresponding to burst.At the same time it can also obtain global verification burst corresponding to N number of data fragmentation.
Step 502, data storage device is corresponding by J data fragmentation of J data fragmentation in N number of data fragmentation and this
Local verification burst be stored in the different storage devices in the first data center DC, and by remaining N-J data fragmentation
It is temporarily stored in the first DC, and each group of data fragmentation in the N-J data fragmentation is corresponding with this group of data fragmentation
Local verification burst give in the 2nd DC corresponding to this group of data fragmentation.
Wherein, the N-J data fragmentation is divided at least one set, and each group of data fragmentation includes at least one data point
Piece, corresponding 2nd DC of each group of data fragmentation, J >=1, J are integer.
Exemplary, data storage device, can will when storing the data fragmentation of the first data and local verification burst
The N data amount checks are equally divided into N/J groups according to burst, and each group includes J data fragmentation.That is, one group in the N/J groups
Burst is locally verified corresponding to data fragmentation and this group of data fragmentation to be stored in the first DC, remaining N/J-1 groups data hair point
Piece (common N-J data fragmentation) is temporarily stored in the first DC.Each group of data fragmentation in the N/J-1 group data fragmentations and should
Local check sheet corresponding to group data fragmentation is respectively stored in corresponding 2nd DC.
Or N number of data fragmentation can also be grouped by the way of non-divide equally, one group includes J data point
Piece, remaining N-J data fragmentation can be divided into K groups, and each group includes (N-J)/K.Wherein, K represents the 2nd DC
Number.
It should be noted that in distributed memory system, obtain data fragmentation and distribute each data fragmentation storage position
The DC corresponding to data storage device put is properly termed as the first DC, and other DC in addition to the first DC are properly termed as second
DC。
It is exemplary, include 3 (K=3) individual DC, respectively DC1, DC2 and DC3 in storage system in a distributed manner, first
N (N=6) individual data fragmentation of data is data fragmentation a, b, c, d, e and f, and the local verification burst of the first data includes this
Ground verification burst a, local verification burst b and local verification burst c.
If data storage device 1 corresponding with DC1 receives the first data, and obtains the data fragmentation of first data
Verify burst with local, then data storage device 1 can according to can be as shown in Figure 6 data storage method data storage point
Piece a, b, c, d, e and f, and local verification burst a, local verification burst b and local verification burst c.Wherein, data fragmentation
A and data fragmentation b is first group of data fragmentation, corresponding local verification burst a；That is data fragmentation c and data fragmentation d is second group
Data fragmentation, corresponding local verification burst b；That is data fragmentation e and data fragmentation f is the 3rd group of data fragmentation, corresponding local school
Test burst c.First group of data fragmentation and local verification burst a are stored in DC1.Wherein, data fragmentation a, data fragmentation b and sheet
Ground verification burst a is stored in the different storage devices (for example, different storage hard disk etc.) in DC1, when first group of data fragmentation
In the damage of any one data fragmentation when, you can the damage is reconstructed by local verification burst a and unspoiled data fragmentation
Bad data fragmentation.Remaining data fragmentation c, d, e, f (i.e. second group and the 3rd group of data fragmentation) are temporarily stored in DC1
In.Wherein, data fragmentation c, d, e, f can be temporarily stored in the different storage devices in DC1, can also be temporarily stored in same
In one storage device.
Second group of data fragmentation and local verification burst b are sent to DC2 by data storage device 1, and second is stored in DC2
Group data fragmentation and local verification burst b.When any one data fragmentation damage in second group of data fragmentation, you can pass through
Local verification burst b and unspoiled data fragmentation reconstruct the data fragmentation of the damage.Wherein, data fragmentation c, data fragmentation
D and local verification burst b can be stored in the different storage devices in DC3.
3rd group of data fragmentation and local verification burst c are sent to DC3 by data storage device 1, and the 3rd is stored in DC3
Group data fragmentation and local verification burst c.When any one data fragmentation damage in the 3rd group of data fragmentation, you can pass through
Local verification burst c and unspoiled data fragmentation reconstruct the data fragmentation of the damage.Wherein, data fragmentation e, data fragmentation
F and local verification burst c can be stored in the different storage devices in DC3.
It should be noted that if data storage device 1 also obtains the overall situation corresponding with data fragmentation a, b, c, d, e and f
Verify burst, including global verification burst a, global verification burst b and global verification burst c.So data storage device 1
This 3 global bursts that verify can be respectively stored in a DC.I.e. as shown in fig. 6, data storage device 1 verifies the overall situation
Burst a is stored in DC1, and overall situation verification burst b is stored in DC2, and overall situation verification burst c is stored in DC3.Work as data
When data fragmentation within any 3 or 3 in burst a, b, c, d, e and f damages, the overall situation can be passed through and verify burst
A, global verification burst b and global verification burst c and unspoiled data fragmentation recover impaired data fragmentation.
Or data storage device 1 can also according to can be as shown in Figure 7 data storage method data storage burst a,
B, c, d, e and f, it is local to verify burst a, local verification burst b and local verification burst c and global verification burst a, complete
Office verification burst b and global verification burst c.Wherein, data fragmentation a, b, c is first group of data fragmentation, corresponding local verification
Burst a；Data fragmentation d, e, f are second group of data fragmentation, corresponding local verification burst b；Overall situation verification burst a, global verification
Burst b and the global verification bursts of global mono- group of verification burst c, corresponding local verification burst c.
Data storage burst a, b, c and corresponding local verification burst a in different storage devices in DC1.When first
When any one data fragmentation in group data fragmentation damages, you can pass through local verification burst a and unspoiled data fragmentation
Reconstruct the data fragmentation of the damage.And second group of data fragmentation of interim storage in DC1, wherein in second group of data fragmentation
Data fragmentation d, e, f, can be temporarily stored in the different storage devices in DC1 or be temporarily stored in same
In storage device.
Second group of data fragmentation d, e, f and corresponding local verification burst b are sent to DC2 by data storage device 1,
Data storage burst d, e, f and corresponding local verification burst b in different storage devices in DC2.When second group of data fragmentation
In the damage of any one data fragmentation when, you can the damage is reconstructed by local verification burst b and unspoiled data fragmentation
Bad data fragmentation.
Overall situation verification burst a, the overall situation are verified burst b, global verification burst c and corresponding local by data storage device 1
Verify burst c and be sent to DC3, global verification burst a is stored in the different storage devices in DC3, the overall situation verifies burst b, complete
Office verification burst c and corresponding local verification burst c.As overall situation verification burst a, global verification burst b, global verification burst c
In any one overall situation verification burst damage when, global verify burst and local verification burst c reconstructs this using unspoiled
The global verification burst of damage.
In a kind of possible design, data storage device is also based on the first DC total accounting to judge which kind of is performed
Storage mode.Wherein, total accounting is that the total capacity that interim storage data fragmentation takes in the first DC accounts for the first DC capacity
Ratio.Exemplary, based on such as Fig. 5, as shown in Figure 8, after above-mentioned steps 501, this method also includes：
Step 503, data storage device judges whether the first DC total accounting is more than or equal to default second
Limit value.
Data storage device can it is each complete once to store after, count always accounting for for the first DC after this storage
Than.When there are new data to need to be stored, current first DC total accounting can also be counted.Store the new data it
Before, data storage device judges to perform any Stored Procedure according to newest total accounting.
In this application, when total accounting is less than second threshold value, then perform the application described in step 504 and carry
The Stored Procedure of confession.When total accounting is more than or equal to second threshold value, then depositing described in step 504 can be performed
Store up flow.
Step 504, data storage device uses conventional EC redundant storages mode, stores the data fragmentation and sheet of the first data
Ground verifies burst.
Wherein, conventional EC redundant storages mode can carry out EC redundancies by way of across the DC storage of data fragmentation to deposit
The Stored Procedure of storage.Arrangement mode corresponding to EC redundant storage modes is stored in data storage device.
Exemplary, with the data fragmentation of the first data as is seen in fig. 6 or fig. 7, local verification burst and distribution
Exemplified by storage system.Data storage device 1 determines that DC1 total accounting is more than before 6 data fragmentations of the first data are stored
Second threshold value, i.e. DC1 are currently not suitable for carrying out full dose storage (the owning the first data i.e. as described in step 502
In data fragmentation storage DC1).So, as shown in figure 9, data storage device 1 can be by 6 data fragmentations and each verification
Burst is averagely stored in 3 DC.That is data storage burst a, data fragmentation b, local verification burst a and global verification in DC1
Data storage in data storage burst c, data fragmentation d, local verification burst b and global verification burst b, DC3 in burst a, DC2
Burst e, data fragmentation f, local verification burst c and global verification burst c.
Or as shown in Figure 10,6 data fragmentations can also be centrally stored in DC1 and DC2 by data storage device 1
In.That is data storage burst d, e, f and correspondingly in data storage burst a, b, c and corresponding local verification burst a, DC2 in DC1
Local verification burst b.Storage overall situation verification burst a, the overall situation verify burst b in DC3, the overall situation verifies burst c and corresponding
Local verification burst c.
Optionally, in this application, second threshold value can also be set, by judge the first DC total accounting whether
More than or equal to first threshold value, to determine the need for performing data deletion action, that is, delete these and be temporarily stored in
Data fragmentation in first DC.
Exemplary, data storage device can periodically detect the current total accountings of the first DC, or complete one
The first DC total accounting is detected after secondary new data storage operation.When the first DC total accounting is more than or equal to default first
During threshold value, you can triggering deletion action, delete the data fragmentation of interim storage in the first DC.
Wherein, data storage device can be with the data fragmentation of interim storage in the DC of random erasure the first, can also be according to number
Deleted according to the storage time of burst.For example, in all data fragmentations for being currently temporarily stored in the first DC, the first data
N-J data fragmentation be the most long data fragmentation of storage time, then data storage device may be selected to delete the N-J number
According to burst.
Optionally, after storage is into storage system, the access frequency of the data can change with time and become data
Change.For example, in certain time period, the access frequency of the data is higher.It is assumed that when access frequency accesses higher than default second
When thresholding, the data can be described as dsc data.Within another period, the access frequency of the data is relatively low.It is assumed that when visit
When asking that frequency accesses thresholding less than default first, the data are properly termed as cold data.Wherein, the second access thresholding is more than
First accesses thresholding.
So, the access frequency based on data, data storage device can be when performing data deletion action, according to first
The access frequency of the data fragmentation of interim storage in DC, selection deletion is the data fragmentation of cold data, to reduce extra data
Memory space consumes.
Exemplary, with the first data instance, as shown in figure 11, another date storage method provided for the application
The flow chart of one embodiment, describe after the first data complete storage in the way of above-mentioned steps 502, in data storage
During, method how to delete the data fragmentation of cold data.Specifically, this method includes：
Step 1101, when total accounting is more than or equal to default first threshold value, data storage device judges first
Whether the access frequency of data, which is less than or equal to default first, accesses thresholding.
When total accounting is more than or equal to default first threshold value, data memory device determines to need execution data to delete
Division operation.So data memory device can obtain the access frequency of each data fragmentation stored in current first DC
Rate, to judge which data is changed into cold data.
It should be noted that after the completion of each data storage that data storage device is received, can be periodic
Count and record the access frequency of each data.
With the first data instance, after the first data complete storage, data storage device periodically counts and records this
The access frequency of first data.When it is determined that need perform data deletion action when, data storage device can obtain to this first
The newest record of data is judged.If the newest access frequency of first data, which is less than or equal to first, accesses thresholding,
Illustrate that now first data are cold data.If the newest access frequency of first data is more than or equal to the second access door
Limit, illustrates that now first data are dsc data.
So, when needing to perform data deletion action, data storage device passes through the data stored to each
Access frequency judgement, it may be determined which is cold data in these stored data, and which is dsc data.
Step 1102, if the access frequency is less than or equal to the first access thresholding, data storage device then deletes this
N-J data fragmentation of interim storage in first DC.
It is understood that when data storage device determines that the access frequency of the first data is less than or equal to first and accessed
Thresholding, that is, when to determine the first data be cold data, data storage device can delete N-J number of interim storage in the first DC
According to burst.
Exemplary, by taking the example shown in Fig. 6 or Fig. 7 as an example.First data store distribution in the manner depicted in FIG. 6
After in formula storage system, when data storage device 1 determine need perform data deletion action, and determine the first data be cold data
When, data storage device 1 can delete data fragmentation c, d, e and f of the first data of interim storage in DC1.After deletion, the
The storage mode of one data is as shown in Figure 9.
Or first data stored in a manner of shown in Fig. 7 in distributed memory system after, when data storage device 1
It is determined that needing to perform data deletion action, and the first data are determined when being cold data, data storage device 1 can be deleted in DC1
Data fragmentation d, e and f of first data of interim storage.After deletion, the storage mode of the first data is as shown in Figure 10.
Optionally, the first threshold value of execution deletion action is used to determine whether, can be as obtained by calculating equation below：
X=(n/K/ (n+m)) * h%；
Wherein, X represents the first threshold value, and K >=1, K are integer, and m represents the verification burst for allowing storage in K the 2nd DC
Total quantity, n represents the total quantity for allowing the data fragmentation of storage in K the 2nd DC, and h% represents dsc data ratio.Namely
Say, can account for data storage burst on each 2nd DC on K the 2nd DC allows all bursts (including data fragmentation of storage
With verification burst) ratio, be multiplied with dsc data ratio, obtain first threshold value.
Exemplary, it is assumed that dsc data ratio h%=10%, n=18, m=12, wherein, 12 verification bursts include 3
Local verification burst and 9 global verification bursts.So the first threshold value X=(18/2/ (18+12)) * 10%=3%.I.e. pair
For first DC, the data point in the first DC (are temporarily stored in store ephemeral data burst by 3% volume space
Piece).That is, by take 3% volume space, reduce across DC read frequency, and read the first data when
Prolong.
In one example, the first data are temporarily stored in after N-J data fragmentation in the first DC be deleted, or
After the first data are stored using conventional EC redundant storages mode, the access frequency of the first data improves, and turns into dsc data.That
In order to avoid causing network consumption excessive due to frequently across DC data of reading first, data storage device can be by the N-
J data fragmentation is stored in the first DC again, to allow the reading of the first data to be completed in the first DC, without across
DC is read.
Exemplary, as shown in figure 12, the stream of one embodiment of another date storage method provided for the application
Cheng Tu.This method includes：
Step 1201, thresholding is accessed when access frequency is more than or equal to default second, and total accounting is less than first
During limit value, N-J data fragmentation is copied in the 2nd DC that data storage device is stored from the N-J data fragmentation.
It is understood that data storage device is in periodic statistics and when recording the access frequency of the first data, can be with
The access frequency of first data is judged simultaneously.If it find that the access frequency of the first data is more than or equal to second
Thresholding is accessed, that is, when to determine first data be dsc data, data storage device then may determine that now the first DC total accounting
Whether the first threshold value is less than.If the first DC total accounting is less than the first threshold value, data storage device can be according to record
The N-J data fragmentation storage location, copy the N-J data in the 2nd DC stored from the N-J data fragmentation
Burst.
Step 1202, N-J data fragmentation of copy is temporarily stored in the first DC by data storage device.
Exemplary, by taking the example shown in Fig. 9 or Figure 10 as an example.Whole numbers of the first data are not stored in DC1
In the case of according to burst, if the process that data storage device 1 is counted and recorded in the access frequency to the first data
In, the second access thresholding that the access frequency of the first data is more than or equal to is determined, and DC1 total accounting is less than the first thresholding
Value.So data storage device 1 can determine not deposit in DC1 according to the storage location of each burst of the first data of record
The storage location of the data fragmentation of storage.
If as shown in figure 9, storage location of the data storage device 1 according to each burst of the first data of record, really
Data fragmentation c, d of fixed first data are stored in DC2, and data fragmentation e, f are stored in DC3.So data storage device 1 is
Data fragmentation c, d, e and f can be copied from DC2 and DC3, then the data fragmentation c, d, e and f are stored in DC1.With
So that the storage mode of the first data is as shown in Figure 6.So as to which when reading the first data, the first data can be read from DC1
All data fragmentations, without each burst of frequently across DC data of reading first, i.e., read in DC1 data fragmentation a,
B, data fragmentation c, d are read in DC2 and data fragmentation e, f are read in DC3.So as to avoid due to frequently across DC reading
The problem of network consumption is excessive caused by and.
As shown in Figure 10, if data storage device 1 is according to the storage location of each burst of the first data of record, really
Data fragmentation d, e, f of fixed first data are stored in DC2.So data storage device 1 can copy data fragmentation from DC2
D, e and f, then the data fragmentation d, e and f are stored in DC1.To cause the storage mode of the first data such as Fig. 7 institutes
Show.So as to when reading the first data, all data fragmentations of the first data can be read from DC1, without frequently across
DC reads each burst of the first data, i.e., data fragmentation a, b, c are read in DC1, and data fragmentation d, e, f are read in DC2.
So as to avoid due to frequently across DC reading and caused by network consumption it is excessive the problem of.
The date storage method provided based on above-mentioned the application, data storage device store the first data in the first DC
J data fragmentation and the remaining N-J data fragmentations of interim storage in the case of, as shown in figure 13, the application provides also
A kind of method for reading data, including：
Step 1301, client-server reads J data fragmentation of storage and the N-J of interim storage from the first DC
Data fragmentation.
Step 1302, the J data fragmentation and the N-J data fragmentation are combined as the first data by client-server.
It is exemplary, the example with reference to shown in Fig. 6, pass through data storage device 1 corresponding with DC1 during due to the first data
DC1, DC2 and DC3 are write, therefore, when accessing the first data, it is also desirable to each data of the first data are read from DC1
Burst.So, the date storage method provided based on the application, is deposited due to having carried out interim full dose to the first data in DC1
Store up, be i.e. store data fragmentation a, b in DC1, go back interim storage remaining data fragmentation c, d, e and f.Therefore, client
Server 1 can read data fragmentation a, b, c, d, e and f directly from DC1, and be combined as the first data, without across DC
Read each burst of the first data, avoid due to the data fragmentation of the first data is transmitted between DC and caused by net
Network consumes.
Similarly understand, can be directly to the first DC and each the when needing to delete the first data in storage system
Two DC send control information, to indicate that each DC deletes the data fragmentation of the first data and corresponding verification burst, with release
Corresponding memory space, without carrying out each data fragmentation of the first data between DC, and then avoid network consumption.
From above-described embodiment as can be seen that no matter the date storage method provided based on the application, read data or deleted
Except data, between the first DC and K the 2nd DC will not transmitted data slice, therefore avoid due to across DC readings or deletion
Network consumption caused by and, improve the performance of data storage.
The above-mentioned scheme mainly provided from the angle of interaction between each network element the application is described.It is appreciated that
, for data storage device in order to realize above-mentioned function, it comprises perform the corresponding hardware configuration of each function and/or software
Module.Those skilled in the art should be readily appreciated that, with reference to the list of each example of the embodiments described herein description
Member and algorithm steps, the application can be realized with the combining form of hardware or hardware and computer software.Some function is actually
Performed in a manner of hardware or computer software driving hardware, application-specific and design constraint bar depending on technical scheme
Part.Professional and technical personnel can realize described function using distinct methods to each specific application, but this
Realize it is not considered that exceeding scope of the present application.
The application can carry out the division of functional module according to above method example to data storage device, for example, can be with
Corresponding each function divides each functional module, and two or more functions can also be integrated in a processing module
In.Above-mentioned integrated module can both be realized in the form of hardware, can also be realized in the form of software function module.Need
It is noted that the division in the application to module is schematical, only a kind of division of logic function can when actually realizing
To there is other dividing mode.
In the case where dividing each functional module using corresponding each function, Figure 14 A show institute in above-described embodiment
A kind of possible structural representation for the data storage device being related to, data storage device include：Acquiring unit 1401, storage are single
Member 1402, delete unit 1403.Acquiring unit 1401 performs the step 501 in Fig. 5 and Fig. 8 for data storage device；Storage
Unit 1402 is used to support that data storage device performs the step 502 in Fig. 5, the step in step 502-504, Figure 12 in Fig. 8
Rapid 1201-1202；Unit 1403 is deleted to be used to support data storage device to perform the step 1101-1102 in Figure 11.Wherein, on
The function description of corresponding function module can be quoted by stating all related contents for each step that embodiment of the method is related to, herein
Repeat no more.
In the case of using integrated unit, Figure 14 B show data storage device involved in above-described embodiment
A kind of possible structural representation.Data storage device includes：Processing module 1411 and communication module 1412.Processing module
1411 actions for being used for data storage device are controlled management.For example, processing module 1411 is used to support data storage to fill
Put in the step 1101-1102 and Figure 12 in the step 501-504, Figure 11 in the step 501-502, Fig. 8 in execution Fig. 5
Step 1201-1202, and/or other processes for techniques described herein.Communication module 1412 is used to support data to deposit
The communication of storage device and other network entities, such as the communication between other DC with being shown in Fig. 3.Data storage device may be used also
With including memory module 1413, program code and data for data storage storage device.
Wherein, processing module 1411 can be processor or controller, such as can be central processing unit (Central
Processing Unit, CPU), general processor, digital signal processor (Digital Signal Processor, DSP),
Application specific integrated circuit (Application-Specific Integrated Circuit, ASIC), field programmable gate array
It is (Field Programmable Gate Array, FPGA) or other PLDs, transistor logic, hard
Part part or its any combination.What it can realize or perform with reference to described by present disclosure various exemplary patrols
Collect square frame, module and circuit.The processor can also be the combination for realizing computing function, such as include one or more micro- places
Manage device combination, combination of DSP and microprocessor etc..Communication module 1412 can be transceiver, transmission circuit or communication interface
Deng.Memory module 1413 can be memory.
When processing module 1411 is processor, communication module 1412 is communication interface, when memory module 1413 is memory,
Data storage device involved by the application can be the data storage device shown in Figure 14 C.
Refering to shown in Figure 14 C, the data storage device includes：Processor 1421, communication interface 1422, memory 1423 with
And bus 1424.Wherein, communication interface 1422, processor 1421 and memory 1423 are connected with each other by bus 1424；Always
Line 1424 can be Peripheral Component Interconnect standard (Peripheral Component Interconnect, PCI) bus or extension
Industrial standard architectures (Extended Industry Standard Architecture, EISA) bus etc..The bus can be with
It is divided into address bus, data/address bus, controlling bus etc..For ease of representing, only represented in Figure 14 C with a thick line, but not table
Show only a bus or a type of bus.
The step of method or algorithm with reference to described by present disclosure, can be realized in a manner of hardware, also may be used
By be by computing device software instruction in a manner of realize.Software instruction can be made up of corresponding software module, software mould
Block can be stored on random access memory (Random Access Memory, RAM), flash memory, read-only storage (Read
Only Memory, ROM), Erasable Programmable Read Only Memory EPROM (Erasable Programmable ROM, EPROM), electricity can
EPROM (Electrically EPROM, EEPROM), register, hard disk, mobile hard disk, read-only optical disc
(CD-ROM) or in the storage medium of any other form well known in the art.A kind of exemplary storage medium is coupled to place
Device is managed, so as to enable a processor to from the read information, and information can be write to the storage medium.Certainly, store
Medium can also be the part of processor.Processor and storage medium can be located in ASIC.In addition, the ASIC can position
In core network interface equipment.Certainly, processor and storage medium can also be present in core network interface as discrete assembly and set
In standby.
In the specific implementation, the application also provides a kind of computer-readable storage medium, wherein, the computer-readable storage medium can store
There is program, the program may include the part or all of step in each embodiment for the date storage method that the application provides when performing
Suddenly.Described storage medium can be magnetic disc, CD, read-only memory (English：Read-only memory, referred to as：ROM)
Or random access memory (English：Random access memory, referred to as：RAM) etc..
Present invention also provides it is a kind of comprising instruction computer program product, when run on a computer so that
Computer performs the part or all of step in each embodiment for the date storage method that above-mentioned the application provides.
It is required general that those skilled in the art can be understood that the technology in the application can add by software
The mode of hardware platform is realized.Based on such understanding, the technical scheme in the application is substantially in other words to prior art
The part to contribute can be embodied in the form of software product, and the computer software product can be stored in storage medium
In, such as ROM/RAM, magnetic disc, CD, including some instructions to cause a computer equipment (can be personal computer,
Server, either vpn gateway etc.) perform method described in some parts of each embodiment of the present invention or embodiment.
In this specification between each embodiment identical similar part mutually referring to.Implement especially for device
For example, because it is substantially similar to embodiment of the method, so description is fairly simple, related part is referring in embodiment of the method
Explanation.
Invention described above embodiment is not intended to limit the scope of the present invention..
Claims (13)
- A kind of 1. date storage method, it is characterised in that including：The data fragmentation and local verification burst, the data fragmentation of first data for obtaining the first data include N number of data point Piece, N >=2, N are integer；It is stored in burst is locally verified corresponding to J data fragmentation in N number of data fragmentation and the J data fragmentation In different storage devices in first data center DC, and remaining N-J data fragmentation is temporarily stored in the first DC In, and burst hair will be locally verified corresponding to each group of data fragmentation in the N-J data fragmentation and this group of data fragmentation Give in the 2nd DC corresponding to this group of data fragmentation；Wherein, the N-J data fragmentation is divided at least one set, and each group of data fragmentation includes at least one data point Piece, corresponding 2nd DC of each group of data fragmentation, J >=1, J are integer.
- 2. according to the method for claim 1, it is characterised in that methods described also includes：When total accounting is more than or equal to default first threshold value, the N-J of interim storage in the first DC is deleted Individual data fragmentation, total accounting are that the total capacity that the data fragmentation of interim storage in the first DC takes accounts for the first DC Capacity ratio.
- 3. according to the method for claim 2, it is characterised in that the N- for deleting interim storage in the first DC Before J data fragmentation, methods described also includes：Judge whether the access frequency of first data is less than or equal to default first and accesses thresholding；The N-J data fragmentation for deleting interim storage in the first DC, including：If the access frequency, which is less than or equal to described first, accesses thresholding, the institute of interim storage in the first DC is deleted State N-J data fragmentation.
- 4. according to the method for claim 3, it is characterised in that if the access frequency is less than or equal to described the One accesses thresholding, then deletes in the first DC after the N-J data fragmentation of interim storage, methods described also includes：Thresholding is accessed when the access frequency is more than or equal to default second, and total accounting is less than first thresholding During value, the N-J data fragmentation is copied in the 2nd DC stored from the N-J data fragmentation；The N-J data fragmentation of copy is temporarily stored in the first DC.
- 5. according to the method described in claim any one of 2-4, it is characterised in that the calculation of first threshold value is such as Under：X=(n/K/ (n+m)) * h%；Wherein, X represents first threshold value, and K represents the number of the 2nd DC, and K >=1, K are integer, and m is represented described in K Allow the total quantity of the verification burst of storage in 2nd DC, n represents the total of the data fragmentation for allowing storage in K the 2nd DC Quantity, h% represent dsc data ratio.
- 6. according to the method described in claim any one of 1-5, it is characterised in that it is described obtain the first data data fragmentation and After local verification burst, methods described also includes：Judge whether total accounting is more than or equal to default second threshold value, total accounting is to be deposited temporarily in the first DC The total capacity that the data fragmentation of storage takes accounts for the ratio of the capacity of the first DC；Burst is locally verified corresponding to the J data fragmentation by N number of data fragmentation and the J data fragmentation to deposit Store up in the different storage devices in the first DC, and remaining N-J data fragmentation is temporarily stored in the first DC, with And burst will be locally verified corresponding to each group of data fragmentation in the N-J data fragmentation and this group of data fragmentation and is issued with being somebody's turn to do In 2nd DC corresponding to group data fragmentation, including：When it is determined that total accounting is less than second threshold value, by J data fragmentation in N number of data fragmentation and institute State and locally verified in the different storage devices that burst is stored in the first DC corresponding to J data fragmentation, and by remaining N-J Data fragmentation is temporarily stored in the first DC, and by each group of data fragmentation in the N-J data fragmentation and the group Burst is locally verified corresponding to data fragmentation to issue in the 2nd DC corresponding with this group of data fragmentation.
- 7. according to the method described in claim 1,5 or 6, it is characterised in that the J number by N number of data fragmentation Locally verified in the different storage devices that burst is stored in the first DC according to corresponding to burst and the J data fragmentation, and will After remaining N-J data fragmentation is temporarily stored in the first DC, methods described also includes：Client-server reads the J data fragmentation of storage and the N-J of interim storage from the first DC Data fragmentation；The J data fragmentation and the N-J data fragmentation are combined as first data by the client-server.
- A kind of 8. data storage device, it is characterised in that including：Acquiring unit, for the data fragmentation for obtaining the first data and local verification burst, the data fragmentation of first data Including N number of data fragmentation, N >=2, N are integer；Memory cell, for will corresponding to J data fragmentation in N number of data fragmentation and the J data fragmentation it is local Verification burst is stored in the different storage devices in the first data center DC, and remaining N-J data fragmentation is deposited temporarily Store up in the first DC, and by corresponding to each group of data fragmentation in the N-J data fragmentation and this group of data fragmentation Local verification burst is issued in the 2nd DC corresponding with this group of data fragmentation；Wherein, the N-J data fragmentation is divided at least one set, and each group of data fragmentation includes at least one data point Piece, corresponding 2nd DC of each group of data fragmentation, J >=1, J are integer.
- 9. data storage device according to claim 8, it is characterised in that the data storage device also deletes unit：The deletion unit, for when total accounting is more than or equal to default first threshold value, deleting in the first DC The N-J data fragmentation of interim storage, total accounting are that the data fragmentation of interim storage in the first DC takes Total capacity accounts for the ratio of the capacity of the first DC.
- 10. data storage device according to claim 9, it is characterised in thatThe deletion unit, it is additionally operable to before the N-J data fragmentation of interim storage in deleting the first DC, judges Whether the access frequency of first data, which is less than or equal to default first, accesses thresholding；The N-J data fragmentation of interim storage, is specifically included in first DC described in the deletion element deletion：If the access frequency, which is less than or equal to described first, accesses thresholding, the institute of interim storage in the first DC is deleted State N-J data fragmentation.
- 11. the data storage device according to claim 9 or 10, it is characterised in thatThe memory cell, it is additionally operable to the N-J data of the interim storage in the first DC described in the deletion element deletion After burst, thresholding is accessed when the access frequency is more than or equal to default second, and total accounting is less than described the During one threshold value, the N-J data fragmentation is copied in the 2nd DC stored from the N-J data fragmentation, and will copy The N-J data fragmentation be temporarily stored in the first DC.
- 12. according to the data storage device described in claim any one of 9-11, it is characterised in that the meter of first threshold value Calculation mode is as follows：X=(n/K/ (n+m)) * h%；The total amount of data fragmentation,Wherein, X represents first threshold value, and K represents the number of the 2nd DC, and K >=1, K are integer, and m is represented described in K Allow the total quantity of the verification burst of storage in 2nd DC, n represents the total of the data fragmentation for allowing storage in K the 2nd DC Quantity, h% represent dsc data ratio.
- 13. according to the data storage device described in claim any one of 8-12, it is characterised in thatThe memory cell, it is additionally operable to after the data fragmentation that the acquiring unit obtains first data, judgement always accounts for Than whether being more than or equal to default second threshold value, total accounting is the data fragmentation of interim storage in the first DC The total capacity of occupancy accounts for the ratio of the capacity of the first DC；The memory cell is by local school corresponding to J data fragmentation in N number of data fragmentation and the J data fragmentation Test in the different storage devices that burst is stored in the first DC, and remaining N-J data fragmentation is temporarily stored in described In one DC, and by locally verification divides corresponding to each group of data fragmentation in the N-J data fragmentation and this group of data fragmentation Piece is issued in the 2nd DC corresponding with this group of data fragmentation, is specifically included：When it is determined that total accounting is less than second threshold value, by J data fragmentation in N number of data fragmentation and institute State and locally verified in the different storage devices that burst is stored in the first DC corresponding to J data fragmentation, and by remaining N-J Data fragmentation is temporarily stored in the first DC, and by each group of data fragmentation in the N-J data fragmentation and the group Burst is locally verified corresponding to data fragmentation to issue in the 2nd DC corresponding with this group of data fragmentation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710900323.XA CN107704336B (en) | 2017-09-28 | 2017-09-28 | Data storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710900323.XA CN107704336B (en) | 2017-09-28 | 2017-09-28 | Data storage method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107704336A true CN107704336A (en) | 2018-02-16 |
| CN107704336B CN107704336B (en) | 2021-08-13 |
Family
ID=61175122
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710900323.XA Active CN107704336B (en) | 2017-09-28 | 2017-09-28 | Data storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107704336B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109003216A (en) * | 2018-08-20 | 2018-12-14 | 西安创艺教育培训中心有限公司 | Cloud education and study platform based on two-dimensional code scanning |
| CN111435323A (en) * | 2019-01-15 | 2020-07-21 | 阿里巴巴集团控股有限公司 | Information transmission method, device, terminal, server and storage medium |
| CN113194117A (en) * | 2021-03-22 | 2021-07-30 | 海南视联通信技术有限公司 | Data processing method and device based on video network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7562203B2 (en) * | 2006-09-27 | 2009-07-14 | Network Appliance, Inc. | Storage defragmentation based on modified physical address and unmodified logical address |
| CN101510223A (en) * | 2009-04-03 | 2009-08-19 | 成都市华为赛门铁克科技有限公司 | Data processing method and system |
| CN105573680A (en) * | 2015-12-25 | 2016-05-11 | 北京奇虎科技有限公司 | Storage method and device for replicated data |
| CN106201338A (en) * | 2016-06-28 | 2016-12-07 | 华为技术有限公司 | Date storage method and device |
-
2017
- 2017-09-28 CN CN201710900323.XA patent/CN107704336B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7562203B2 (en) * | 2006-09-27 | 2009-07-14 | Network Appliance, Inc. | Storage defragmentation based on modified physical address and unmodified logical address |
| CN101510223A (en) * | 2009-04-03 | 2009-08-19 | 成都市华为赛门铁克科技有限公司 | Data processing method and system |
| CN105573680A (en) * | 2015-12-25 | 2016-05-11 | 北京奇虎科技有限公司 | Storage method and device for replicated data |
| CN106201338A (en) * | 2016-06-28 | 2016-12-07 | 华为技术有限公司 | Date storage method and device |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109003216A (en) * | 2018-08-20 | 2018-12-14 | 西安创艺教育培训中心有限公司 | Cloud education and study platform based on two-dimensional code scanning |
| CN111435323A (en) * | 2019-01-15 | 2020-07-21 | 阿里巴巴集团控股有限公司 | Information transmission method, device, terminal, server and storage medium |
| CN111435323B (en) * | 2019-01-15 | 2023-06-20 | 阿里巴巴集团控股有限公司 | Information transmission method, device, terminal, server and storage medium |
| CN113194117A (en) * | 2021-03-22 | 2021-07-30 | 海南视联通信技术有限公司 | Data processing method and device based on video network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107704336B (en) | 2021-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109725822B (en) | Method, apparatus and computer program product for managing a storage system | |
| TWI539277B (en) | System, method and computer program product for increasing spare space in memory to extend a lifetime of the memory | |
| CN104503708B (en) | The method and device of data hash storage | |
| US8799571B1 (en) | System and method for configuring a device array upon detecting addition of a storage device | |
| CN104040516B (en) | Method, apparatus and system for data deduplication | |
| US20110107053A1 (en) | Allocating Storage Memory Based on Future Use Estimates | |
| CN109213420A (en) | Date storage method, apparatus and system | |
| CN107408070A (en) | More transaction journals in distributed memory system | |
| CN106909317A (en) | The data storage on memory node | |
| CN103136243A (en) | File system duplicate removal method and device based on cloud storage | |
| CN107704336A (en) | A kind of date storage method and device | |
| CN107203345A (en) | A kind of many copy storage rapid verification coherence methods and its device | |
| CN109582213A (en) | Data reconstruction method and device, data-storage system | |
| US11385823B2 (en) | Method, electronic device and computer program product for rebuilding disk array | |
| TW202040477A (en) | Personal asset change record storage method, system, device and equipment | |
| CN106445409A (en) | Distributed block storage data writing method and device | |
| CN110096220A (en) | A kind of distributed memory system, data processing method and memory node | |
| US20200183883A1 (en) | Storing an asset update record | |
| CN110389710A (en) | The method and apparatus for distributing storage resource | |
| US8924642B2 (en) | Monitoring record management method and device | |
| CN111459948B (en) | Transaction integrity verification method based on centralized block chain type account book | |
| US9292213B2 (en) | Maintaining at least one journal and/or at least one data structure by circuitry | |
| CN115639961A (en) | Data writing method, storage system and server | |
| KR20180114819A (en) | System and method for identifying ssds with lowest tail latencies | |
| US20150242145A1 (en) | Storage apparatus and method of controlling storage apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |