CN105610921A - Erasure code filing method based on data cache in cluster - Google Patents
Erasure code filing method based on data cache in cluster Download PDFInfo
- Publication number
- CN105610921A CN105610921A CN201510979326.8A CN201510979326A CN105610921A CN 105610921 A CN105610921 A CN 105610921A CN 201510979326 A CN201510979326 A CN 201510979326A CN 105610921 A CN105610921 A CN 105610921A
- Authority
- CN
- China
- Prior art keywords
- filing
- node
- data
- deblocking
- cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention discloses an erasure code filing method based on data cache in a cluster. The method comprises following steps of (1), according to a user access request, reading demanded data subblocks from production nodes in which the data subblocks are located to a node buffer; sending the data subblocks to an access node; forwarding the data subblocks to a filing node according to a filing data state table; (2), updating the number of the data subblocks of each data node by the filing node according to the received data subblocks; judging whether the filing node receives all data subblocks of current strips; if so, filing the current strips; otherwise, reading cold data subblocks from a production cluster, filing after the strips are gathered completely. In adoption of the erasure code filing method provided by the invention, compared with the normal erasure code filing method, because the user access cached data subblocks are adopted, the number of times of reading the data subblocks from the production cluster by the filing node is greatly reduced; the erasure code filing operation is carried out under the condition of not influencing the user access; and the filing efficiency is improved.
Description
Technical field
The invention belongs to computer memory technical field, more specifically, relate to the correcting and eleting codes archiving method based on data buffer storage under a kind of cluster.
Background technology
At information age data volume rapid growth of today, Mass storage cluster has obtained application more and more widely. According to the height of data access frequency, can successively the data in storage cluster be divided into dsc data (HotData), warm data (WarmData) and cold data (ColdData). Along with the passing of system operation time, the visiting frequency of dsc data can decline, and becomes warm data, finally becomes cold data, and dsc data conventionally leaves in and produces cluster (ProductionCluster) by three copies or many copies mode. A kind of method that improves production group system utilization rate is that copy data low visiting frequency is moved to correcting and eleting codes filing cluster (ArchivalCluster), and this filing cluster adopts RS code to carry out organize data wherein; Correcting and eleting codes filing (Erasure-codedArchival) refers to the operation that data is moved to correcting and eleting codes storage from copy storage.
It is two separate processes that existing correcting and eleting codes filing and user access copy cluster; The filing of existing correcting and eleting codes comprises that (1) obtains needs filing data at the data bitmap of producing in cluster; (2) from produce cluster, read copy data; (3) will read copy data and carry out correcting and eleting codes coding, generate verification piecemeal; (4) a copy data and corresponding verification piecemeal are write to correcting and eleting codes filing cluster; (5) reclaim remaining cold data on production cluster, delete the data trnascription of producing all participation correcting and eleting codes filings in cluster. User accesses copy cluster and comprises that (1) obtains desired data piecemeal at the metadata information of producing in cluster; (2) deblocking place node is sent to read request; (3) this node reads corresponding data and is chunked into node buffering area; (4) back end sends deblocking, and deblocking is forwarded to client.
For this correcting and eleting codes filing, if the request of data filing is excessive, adopt off-line filing, the two is independent for data access and filing; But at current large data age, there is hardly special time of filing, need to adopt online elite archiving; For online elite archiving, produce on the node in cluster and remained dsc data and cold data, there are the following problems to adopt above-mentioned correcting and eleting codes archiving method:
(1) filing request is directly handed down to production cluster, now produces cluster and is also providing access services for user; Filing request brings taking of the taking of extra external memory access, memory cache, Internet resources, causes the response time that user asks greatly to increase;
(2), for filing task, because the node of producing in cluster need to be processed user access request and data filing request simultaneously, there is resource contention between the two; In the high situation of certain storage cluster node users request of access frequency, the filing request of this node will slowly can not get response, cause filing node cannot obtain in time deblocking, and then affect whole filing efficiency.
Summary of the invention
For above defect or the Improvement requirement of prior art, the invention provides the correcting and eleting codes archiving method based on data buffer storage under a kind of cluster, its object is to improve correcting and eleting codes filing efficiency.
For achieving the above object, according to one aspect of the present invention, provide the correcting and eleting codes archiving method based on data buffer storage under a kind of cluster, specifically comprised the following steps:
(1) according to user access request, desired data piecemeal is produced to node from its place and read node buffering area, and deblocking is sent to access node, be transmitted to filing node according to filing data state table simultaneously;
(2) filing node, according to the deblocking receiving, upgrades the deblocking number of each back end; And judge that whether filing node receives the total data piecemeal of current band, if so, files current band; If not, read cold data block from producing cluster, after band is gathered together enough, file.
Above-mentioned correcting and eleting codes archiving method, owing to having adopted the deblocking of user's access cache, than common correcting and eleting codes filing, can significantly reduce the number of times of filing node reading out data piecemeal from produce cluster, thereby can in the situation that substantially not affecting user's access, carry out archive operation, improve filing efficiency;
On the other hand, the deblocking of user's access cache is forwarded to filing node, avoid user's access and the filing of common correcting and eleting codes to producing the repetition read requests of identical data piecemeal in node, can effectively reduce the data volume of producing Internet Transmission in cluster, the load that alleviates whole network.
Preferably, above-mentioned steps (1), specifically comprises following sub-step:
(1.1) access node obtains required deblocking in the address producing cluster from management server;
(1.2) above-mentioned deblocking is sent to read request in the address corresponding node generating in cluster;
(1.3) production node deblocking being generated cluster from its place reads node buffering area;
(1.4), according to filing data state table, judge that whether this deblocking has been forwarded to filing node, is if so, forwarded to access node by this deblocking; If not, this deblocking is forwarded to simultaneously to access node and filing node; This step directly utilize switch in cluster by the data retransmission of user's access to filing node.
Preferably, above-mentioned steps (2), specifically comprises following sub-step:
(2.1) filing node receives deblocking (dsc data), upgrades the deblocking number of the each back end comprising in filing node;
(2.2) the current band of filing node initializing, the node of analyzing band forms, and which back end is the data block that judgement receives belong to, and initializes storage of variables; In this step, node comprises back end and check-node;
(2.3) judge whether to receive the total data piecemeal of current band by filing node, if so, enter step (2.7); If not, enter step (2.4);
(2.4), in the time interval that is T at duration, filing node receives deblocking from user cache;
(2.5) whether k the deblocking that judges current band collects together, if so, enters step (2.7); If not, enter step (2.6); Wherein, k value equals the number of back end;
(2.6) filing node directly reads cold data from produce cluster, with μ the band that gather together enough, then enters step (2.3);
Wherein, the deblocking number reading from back end j is Numj，Numj＝max{(μ-aj-λ (j)), 0}, the maximum band number that μ can file in time T for filing node, ajFor the deblocking number that comprises back end j in filing node, λ (j) is illustrated in the deblocking number that in next time interval T, filing node will receive from back end j;
(2.7) file current band, and the value of bar reel number i is added to 1, and upgrade filing data state table;
(2.8) judge whether the filing of whole bands according to filing data state table, if not, entered step (2.3); If so, finish filing;
Said process, filing node, in not affecting user's access, utilizes data cached reception user piecemeal, can play the effect that improves filing efficiency.
In general, the above technical scheme of conceiving by the present invention compared with prior art, can obtain following beneficial effect:
(1) the correcting and eleting codes archiving method based on data buffer storage under this cluster provided by the invention, in the situation that not affecting user's access, the data cached filing efficiency that improves of utilizing user to access: by the data cached piecemeal of user's access is directly sent to filing node, than common correcting and eleting codes archiving method, method provided by the invention can reduce the number of times that disk reads, read, produce node memory space hold and produce these three links of node network interface card transmission filing data piecemeal thereby accelerate to produce node external memory deblocking, improve filing efficiency; Only, in the situation that a band does not all receive, just can directly read and produce cluster deblocking, and the number of times of reading out data piecemeal and piece number are much lower than data volume and the frequency of user's access, substantially do not affect user's access;
(2) the correcting and eleting codes archiving method based on data buffer storage under this cluster provided by the invention, has reduced the data volume of producing Internet Transmission in cluster; In existing common correcting and eleting codes filing, deblocking need to be sent to the switch cluster from node buffering area, then be transmitted to filing node; And in the correcting and eleting codes archiving method providing of the present invention, can directly utilize switch in cluster by the data retransmission of user's access to filing node, avoid, in common correcting and eleting codes archiving method, data are sent to the switch cluster network from node buffering area, thereby can effectively reduce the data volume of producing Internet Transmission in cluster, the load that alleviates whole network.
Brief description of the drawings
Fig. 1 is the flow chart of the correcting and eleting codes archiving method based on data buffer storage under cluster of the present invention;
Fig. 2 is the schematic diagram of the correcting and eleting codes archiving method based on data buffer storage under the cluster that provides of embodiment;
Fig. 3 is that user accesses the schematic diagram of filing with common correcting and eleting codes;
Fig. 4 is user access based on data buffer storage and the schematic diagram of data filing.
Detailed description of the invention
In order to make object of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, the present invention is further elaborated. Should be appreciated that specific embodiment described herein, only in order to explain the present invention, is not intended to limit the present invention. In addition,, in each embodiment of described the present invention, involved technical characterictic just can combine mutually as long as do not form each other conflict.
For ease of understanding, the terminological interpretation relating to is in the present invention described as follows:
Produce cluster: in referring to provide filing data in archiving process, still need to provide the node general name of data access service to upper layer application.
Correcting and eleting codes filing cluster: filed the cluster of rear storage partial data band, can have been substituted by the free space of producing cluster itself.
Management server: manage the metadata information of overall cluster, comprise the node location information of deblocking in cluster and copy thereof.
Coding nodes: in the data filing process of a band, implement the node of coding calculating operation.
Node data piecemeal and band: in the cataloged procedure of filing, the unit of reading out data is exactly deblocking. In storage cluster, a band is exactly to be made up of the deblocking unit of same offset address on different nodes, can recover independently the information aggregate of fail data.
Data bitmap: be kept in the management server of storage cluster, represent the corresponding relation between deblocking and back end.
NUMj: in archiving process, when the required deblocking of current filing band does not collect neat, filing node directly reads NUM to producing back end j in clusterj＝max{(μ-aj-λ (j)), 0} deblocking, the band number that μ can file in time T for filing node, ajFor the deblocking number that comprises back end j in filing node, λ (j) is illustrated in the deblocking number that in next section of time T, filing node receives from node j.
Filing data state table: the global table that management server is safeguarded, whether record is produced deblocking in cluster and be archived, and in an embodiment, represents respectively that this deblocking ' is not archived ' and ' being archived ' with ' 0 ' and ' 1 '.
The invention provides the correcting and eleting codes archiving method based on data buffer storage under a kind of cluster, the data cached correcting and eleting codes archiving process of accelerating of mainly utilizing user to access, its idiographic flow as shown in Figure 1, comprises the steps:
(1) according to user access request, desired data piecemeal is produced to node from its place and read node buffering area, then deblocking is sent to access node, be transmitted to filing node according to filing data state table simultaneously; This step specifically comprises following sub-step:
(1.1) access node is asked according to user, and searching and managing server obtains the deblocking of user's request at the node location of producing in cluster;
(1.2) the production clustered node position obtaining according to step (1.1), the node reading out data of producing cluster is chunked into node buffering area;
(1.3) produce querying node management server, obtain filing data state table; According to filing data state table, judge the filing state of user's request data piecemeal;
(1.4) produce node and send deblocking in buffering area;
(1.5) deblocking is sent to access node by switch, with relative users request, meanwhile, the deblocking not being archived is forwarded to filing node, with corresponding filing request;
(1.6) judge whether user's request all finishes dealing with, and if yes, user accesses end; If NO, forward step (1.1) to, continue to carry out next user's request;
(2) filing node, according to the user accesses data receiving, upgrades the deblocking number of each back end, if filing node receives the total data piecemeal of current band, directly files; If cannot receive current band total data piecemeal by user accesses data, read cold data from producing cluster, then file; This step specifically comprises following sub-step:
(2.1) filing node initializing filing band, current filing band is numbered i, i=0; Initialize the deblocking number a that comprises back end j in filing nodejSet a be [0,0,0,0];
(2.2) filing node receives user accesses data (dsc data), and with new set a;
(2.3) filing node judges whether to receive the total data piecemeal of filing band i, if so, enters step (2.7); If not, enter step (2.4);
(2.4) filing node latency period T; Within the time interval of duration T, filing node constantly receives deblocking from user cache; In the present embodiment, T is 1 second;
(2.5) filing node, with new set a, judges whether to receive the total data piecemeal of filing band i again, if so, enters step (2.7); If not, enter step (2.6);
(2.6) filing node directly reads cold data from produce cluster, and the deblocking number reading from back end j is Numj，Numj＝max{(μ-aj-λ (j)), 0}, with μ the band that gather together enough, then enters step (2.3);
(2.7) file current band, and bar reel number i value is added to 1;
(2.8) upgrade the filing data state table in management server;
(2.9) judge whether to produce all filings of band in cluster, if not, entered step (2.3); If so, finish filing.
Set forth the correcting and eleting codes archiving method based on data buffer storage provided by the invention below in conjunction with specific embodiment; What in example, adopt is (k+r, k) RS coded system; Wherein, k refers to the number of initial data piecemeal, the verification piecemeal number that r presentation code generates, and in k deblocking and r verification piecemeal, any k all can decode k initial data piecemeal.
As shown in Figure 2, be the exemplary plot that adopts the correcting and eleting codes archiving method based on data buffer storage under RS (6,4) code clustering in embodiment; Wherein, in correcting and eleting codes filing cluster, always have six nodes, wherein four is back end, and two is inspection node, adopts the coded system of (6,4) RS to realize and ensure data integrity, and node serial number is { 1,2,3,4,5,6};
Wherein, come from the logical address of producing the arbitrary deblocking in cluster all corresponding to one of them in four back end in correcting and eleting codes cluster, filing node is filed band and is counted μ=3 in time T.
Correcting and eleting codes archiving method based on data buffer storage under the cluster that the present embodiment provides, specific as follows:
(1), in embodiment, the deblocking number set of user's access is for the first time [2,4,5,3],, obtains respectively 2,4,5 and 3 deblockings from back end 1, back end 2, back end 3 and back end 4 that is;
Access node is searching and managing server first, obtains whole address informations of this deblocking set; Meanwhile, filing node initializing band i; Now, in filing node, the deblocking number of each back end is [0,0,0,0]; Wherein, the initial value of i is 0;
(2) produce cluster reading out data block collection to buffering area; While access for the first time, deblocking set is forwarded to access node and filing node by switch simultaneously;
(3) access node obtains deblocking set, response user request; Meanwhile, filing node obtains deblocking set, and the deblocking number of upgrading each back end is [2,4,5,3];
1. indicated flow process in above-mentioned steps (1) (2) (3) corresponding diagram 2;
(4) filing node carries out the calculating of band i coding, and 4 deblockings and 2 verification piecemeals are sent to correcting and eleting codes filing cluster, and i increases by 1, and now i is 1; Filing node is removed the deblocking of having filed band 0;
(5) if filing node is received all deblockings of band i, file; And 4 deblockings and 2 verification piecemeals are sent to correcting and eleting codes filing cluster, and i increases by 1, and now i is 2;
Delete the deblocking of the band 1 of having filed in filing node; In filing node, each node data piecemeal number is [0,2,3,1];
2. indicated flow process in above-mentioned steps (4) (5) corresponding diagram 2;
(6) if do not receive the total data piecemeal of band i in filing node, enter wait state; Meanwhile, user carries out back-call, and deblocking set is [0,3,3,4], in tentation data block collection, does not have and the deblocking of accessing for the first time repetition, and deblocking set is forwarded to access node and filing node by switch simultaneously;
(7) access node relative users back-call; Meanwhile, filing node receives deblocking set [0,3,3,4], and upgrading node data piecemeal number is [0,5,6,5];
3. indicated flow process in above-mentioned steps (6) (7) corresponding diagram 2;
(8) after filing node stand-by period section T, if do not receive yet the total data piecemeal of band i, directly from producing reading out data piecemeal cluster;
In next time period T, the deblocking λ of user's access is [1,1,1,1], according to NUMj＝max{(μ-aj-λ (j)), 0}, obtaining the deblocking number reading from four back end is [2,0,0,0]; Need to read two corresponding deblockings from back end 1; Wherein, the band that filing node can be filed in time T is counted μ=3; The deblocking number a that comprises back end j in filing nodejSet a be [0,5,6,5];
(9) user accesses data block collection is [1,1,2,1] for the third time, and deblocking set is forwarded to access node and filing node by switch simultaneously; Meanwhile, filing node is pressed directly reading out data piecemeal from produce cluster of deblocking set [2,0,0,0];
(10) filing node receives from the deblocking set [1,1,2,1] of user cache and the deblocking set [2,0,0,0] of directly reading, and upgrading node data piecemeal number is [3,6,8,6];
4. indicated flow process in above-mentioned steps (8) (9) (10) corresponding diagram 2;
(11) filing node completes the filing calculating of 3 bands, and the deblocking of each band and verification piecemeal are sent to correcting and eleting codes filing cluster, deletes the deblocking of having filed in filing node; Now i is 5, and in filing node, the number of each node data piecemeal is [0,3,5,3];
5. indicated flow process in above-mentioned steps (11) corresponding diagram 2;
(12) repeating step (1)～(11), complete until produce the filing of whole bands in cluster.
In prior art, the process of common correcting and eleting codes filing as shown in Figure 3, be specially: filing node directly reads the required k of a band deblocking from produce cluster, then obtain r verification piecemeal of this band, finally k deblocking and r verification piecemeal are sent to the preservation of correcting and eleting codes filing cluster; Each filing is all that reading out data piecemeal calculates verification piecemeal again from produce cluster, then deblocking and verification piecemeal is sent to correcting and eleting codes filing cluster, until filing all completes;
And the correcting and eleting codes archiving method based on data buffer storage that this invention provides is specific as follows:
(1) filing node initializing band, the deblocking of reception user access cache;
(2) if the k of a current filing band deblocking all receives, just enter step (4); Otherwise wait for the data that send in next time period T from the data buffer storage of user's access;
(3) judge whether to receive the total data piecemeal of current band, if so, enter step (4); If NO, from produce cluster, read corresponding deblocking with μ the band that gather together enough;
(4) filing node is to the calculating of encode of current band, k deblocking and r and verification piecemeal sent to correcting and eleting codes and file in cluster and preserve;
(5) repeating step (2), to (4), completes until produce the filing of whole bands in cluster.
In embodiment, adopt the deblocking of the reception user access cache in step (1), than common correcting and eleting codes filing, significantly reduce the number of times of filing node reading out data piecemeal from produce cluster, thereby can in the situation that substantially not affecting user's access, carry out archive operation, improve filing efficiency; On the other hand, by switch, the deblocking of user's access cache is forwarded to filing node, avoid user's access and the filing of common correcting and eleting codes to producing the repetition read requests of identical data piecemeal in node, can effectively reduce the data volume of producing Internet Transmission in cluster, the load that alleviates whole network.
Those skilled in the art will readily understand; the foregoing is only preferred embodiment of the present invention; not in order to limit the present invention, all any amendments of doing within the spirit and principles in the present invention, be equal to and replace and improvement etc., within all should being included in protection scope of the present invention.
Claims (3)
1. the correcting and eleting codes archiving method based on data buffer storage under cluster, is characterized in that, described in entangleDeleting a yard archiving method specifically comprises the following steps:
(1), according to user access request, desired data piecemeal is read to node buffering from producing nodeDistrict, and deblocking is sent to access node, simultaneously according to filing data state table by described dataPiecemeal is transmitted to filing node;
(2) filing node, according to the deblocking receiving, upgrades the deblocking of each back endNumber; And judge that whether filing node receives the total data piecemeal of current band, if so, willCurrent band is filed; If not, read cold data block from producing cluster, after band is gathered together enoughFile.
2. correcting and eleting codes archiving method as claimed in claim 1, is characterized in that, described step (1)Specifically comprise following sub-step:
(1.1) access node obtains required deblocking on the ground producing cluster from management serverLocation;
(1.2) described deblocking is sent to read request in the address corresponding node generating in cluster;
(1.3) production node deblocking being generated cluster from its place reads node buffering area;
(1.4) judge whether described deblocking has been forwarded to filing node, if so, by this numberBe forwarded to access node according to piecemeal; If not, this deblocking be forwarded to access node simultaneously and returnShelves node.
3. correcting and eleting codes archiving method as claimed in claim 1 or 2, is characterized in that, described step(2) specifically comprise following sub-step:
(2.1) filing node receives deblocking, upgrades the each back end comprising in filing nodeDeblocking number;
(2.2) the current band of filing node initializing, the node of analyzing band forms, and judgement receivesWhich back end is data block belong to, and initializes storage of variables; Described node comprise back end andCheck-node;
(2.3) judge whether to receive the total data piecemeal of current band by filing node, if so,Enter step (2.7); If not, enter step (2.4);
(2.4), in the time interval that is T at duration, filing node receives deblocking from user cache;
(2.5) whether k the deblocking that judges current band collects together, if so, enters step (2.7);If not, enter step (2.6); Wherein, k value equals the number of back end;
(2.6) filing node directly reads cold data from produce cluster, with μ the band that gather together enough, soAfter enter step (2.3); Wherein, the maximum bar that μ can file in time interval T for filing nodeBand number;
(2.7) file current band, and the value of bar reel number i is added to 1, and upgrade filing data state table;
(2.8) judged whether the filing of whole bands according to filing data state table, if not,Enter step (2.3); If so, finish filing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510979326.8A CN105610921B (en) | 2015-12-23 | 2015-12-23 | Correcting and eleting codes archiving method based on data buffer storage under a kind of cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510979326.8A CN105610921B (en) | 2015-12-23 | 2015-12-23 | Correcting and eleting codes archiving method based on data buffer storage under a kind of cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105610921A true CN105610921A (en) | 2016-05-25 |
| CN105610921B CN105610921B (en) | 2018-09-07 |
Family
ID=55990455
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510979326.8A Expired - Fee Related CN105610921B (en) | 2015-12-23 | 2015-12-23 | Correcting and eleting codes archiving method based on data buffer storage under a kind of cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105610921B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108304264A (en) * | 2018-01-22 | 2018-07-20 | 华中科技大学 | A kind of correcting and eleting codes archiving method based on SPARK streaming computings |
| CN110190926A (en) * | 2019-04-26 | 2019-08-30 | 华中科技大学 | Correcting and eleting codes restorative procedure, correcting and eleting codes update method and system based on network query function |
| CN112764680A (en) * | 2021-01-20 | 2021-05-07 | 华云数据控股集团有限公司 | Disk stripe selection method and storage system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103177111A (en) * | 2013-03-29 | 2013-06-26 | 西安理工大学 | System and method for deleting repeating data |
| CN103605582A (en) * | 2013-11-27 | 2014-02-26 | 华中科技大学 | Erasure code storage and reconfiguration optimization method based on redirect-on-write |
-
2015
- 2015-12-23 CN CN201510979326.8A patent/CN105610921B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103177111A (en) * | 2013-03-29 | 2013-06-26 | 西安理工大学 | System and method for deleting repeating data |
| CN103605582A (en) * | 2013-11-27 | 2014-02-26 | 华中科技大学 | Erasure code storage and reconfiguration optimization method based on redirect-on-write |
Non-Patent Citations (2)
| Title |
|---|
| JIANZHONG HUANG等: "CoRec: A Cooperative Reconstruction Pattern for Multiple Failures in Erasure-Coded Storage Clusters", 《IEEE》 * |
| 黄建忠等: "面向纠删码存储集群的弹性Ｉ/Ｏ调度机制研究", 《计算机研究与发展》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108304264A (en) * | 2018-01-22 | 2018-07-20 | 华中科技大学 | A kind of correcting and eleting codes archiving method based on SPARK streaming computings |
| CN108304264B (en) * | 2018-01-22 | 2022-02-15 | 华中科技大学 | Erasure code filing method based on SPARK streaming calculation |
| CN110190926A (en) * | 2019-04-26 | 2019-08-30 | 华中科技大学 | Correcting and eleting codes restorative procedure, correcting and eleting codes update method and system based on network query function |
| CN112764680A (en) * | 2021-01-20 | 2021-05-07 | 华云数据控股集团有限公司 | Disk stripe selection method and storage system |
| CN112764680B (en) * | 2021-01-20 | 2024-02-06 | 华云数据控股集团有限公司 | Disk stripe selection method and storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105610921B (en) | 2018-09-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9830101B2 (en) | Managing data storage in a set of storage systems using usage counters | |
| CN103548003B (en) | Method and system for improving the client-side fingerprint cache of deduplication system backup performance | |
| CN103392167B (en) | The caching method of distributed memory system, node | |
| US11232073B2 (en) | Method and apparatus for file compaction in key-value store system | |
| CN106951375B (en) | Method and device for deleting snapshot volume in storage system | |
| CN109947668A (en) | The method and apparatus of storing data | |
| CN107786638B (en) | Data processing method, device and system | |
| CN107958079A (en) | Aggregate file delet method, system, device and readable storage medium storing program for executing | |
| CN101488104A (en) | System and method for implementing high-efficiency security memory | |
| CN110958300B (en) | Data uploading method, system, device, electronic equipment and computer readable medium | |
| CN104092670A (en) | Method for utilizing network cache server to process files and device for processing cache files | |
| CN104125163A (en) | Data processing method, device and terminal | |
| CN107368608A (en) | The HDFS small documents buffer memory management methods of algorithm are replaced based on ARC | |
| WO2012106923A1 (en) | Object adjusting method, migration control device, node device and storage system | |
| US20190278757A1 (en) | Distributed Database Management System with Dynamically Split B-Tree Indexes | |
| CN113254270B (en) | Self-recovery method, system and storage medium for storing cache hot spot data | |
| CN108347459A (en) | A kind of high in the clouds data quick storage method and device | |
| CN103067479A (en) | Network disk synchronized method and system based on file coldness and hotness | |
| WO2024041434A1 (en) | Storage system and data processing method | |
| CN105610921A (en) | Erasure code filing method based on data cache in cluster | |
| CN106991118A (en) | Entangled based on CEPH and delete fault-tolerant reading document method and device | |
| CN104917788A (en) | Data storage method and apparatus | |
| CN104424189A (en) | Positioning resolving method and positioning resolving system based on cloud platform | |
| CN117270794B (en) | Redis-based data storage method, medium and device | |
| CN104915376B (en) | A kind of archival compression method of file in cloud storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180907 Termination date: 20191223 |