CN111400083A - Data storage method and system and storage medium - Google Patents
Data storage method and system and storage medium Download PDFInfo
- Publication number
- CN111400083A CN111400083A CN202010186894.3A CN202010186894A CN111400083A CN 111400083 A CN111400083 A CN 111400083A CN 202010186894 A CN202010186894 A CN 202010186894A CN 111400083 A CN111400083 A CN 111400083A
- Authority
- CN
- China
- Prior art keywords
- data
- blocks
- erasure code
- storage
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003860 storage Methods 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000013500 data storage Methods 0.000 title claims abstract description 25
- 238000004590 computer program Methods 0.000 claims description 5
- 230000006870 function Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000008520 organization Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
- G06F11/1044—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices with specific ECC/EDC distribution
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a data storage method and a system, wherein the method comprises the following steps: dividing a memory merging cache region into a plurality of erasure code strips consisting of N data blocks and M check blocks; matching the data characteristics to be stored with the positions of the data blocks; storing the matched data into the corresponding data block position; when the N data blocks are full, M check blocks are calculated; and storing the N data blocks and the M check blocks into a storage node. The invention can store different data in the same coding mode environment without analyzing different coding formats, thereby greatly improving the read-write performance of the stored data and leading a user to efficiently and lowly access the data and recover the data.
Description
Technical Field
The present application relates to the field of data storage, and in particular, to a data storage method and system, and a storage medium.
Background
The rapid development of networks, broadband, software, intelligent devices and the like brings about the explosive growth of data, and the current intelligent society enters the era of taking data as production data. Therefore, people in the intelligent age have made higher demands on the reliability, read-write performance, safety and the like of stored data.
Early distributed storage systems mainly adopt a copy technology, which can prevent data loss, but as the copy technology needs to consume an equal number of additional storage spaces, the hardware cost rises sharply as the data storage capacity is larger and larger. In order to improve the reliability and security of storage systems and reduce costs, erasure coding techniques have been developed that achieve similar reliability at a much lower cost of storage space than replica techniques. In a distributed storage system, in order to implement erasure code storage, data received in each storage node needs to be merged and then distributed to different storage nodes through erasure codes.
In order to realize erasure code storage in the current storage system, there are two common storage modes.
The first method is as follows: the data is written into a multi-copy storage or other persistent storages, then the data is asynchronously read out, and then the read data is dispersed into different storage nodes through calculating erasure codes.
This approach has the following disadvantages:
1. since data needs to be stored first, and multiple copies of the data need to be stored in order to achieve data reliability similar to erasure codes, the space utilization rate of data storage is very low (1/copy number) before the data is not converted into erasure codes for storage;
2. data is stored by high-performance hardware such as NVDIM (nonvolatile memory management), SSD (solid state disk) and the like, and then is transferred to erasure code storage, so that the cost is increased;
3. because the data cannot be directly stored in an erasure code mode when the data is received, and the erasure code is calculated after the data is read out, the storage system additionally increases the reading and writing of the data, so that enough hard disks are needed, or high-performance SSD storage with high cost is needed, and the writing performance can be met.
The second method comprises the following steps: directly carrying out memory merging on data, and then writing in erasure codes for storage
In this way, under the condition of receiving a large file, an N + M data memory (N generally refers to an original data block, and M refers to a redundant data check block) can be filled quickly, and erasure codes are calculated and distributed to other storage nodes. And the problem of extra read-write overhead of the first mode can be solved.
However, in a scene of a large amount of small files, if a suitable merging strategy is not available, the erasure codes are calculated and distributed to other storage nodes only by filling the N + M data memory, so that the data are distributed in the storage nodes and the storage nodes are very discretely distributed, and a large amount of retrieval and inefficient access hit are required during data reading. Meanwhile, the merging strategy needs to be fast enough, so that the writing performance of the mass small files can be improved.
In view of the technical problems in the above technology, an existing improvement scheme, such as patent one of the inventions (CN102270161A), proposes a solution, which makes a data organization policy according to data characteristics, and can adapt to characteristics of data to the maximum extent and meet application requirements through a multi-level data organization method, so as to facilitate users to access the data in an optimal manner.
In addition, the invention patent II (CN110347344A) stores data by distinguishing the data stripe satisfaction condition and automatically matching different redundancy strategies, so that the high availability of the data and the high efficiency of IO are ensured.
Although the erasure code storage is improved in the first patent and the second patent, a data organization strategy needs to be formulated according to data characteristics or data strips need to be classified, and different data need to be encoded in different modes, so that when an actual large data storage engineering project is implemented, the situation or the type is complex, various complex situations are difficult to effectively solve, the complexity of implementation of the storage project is increased, and the reliability is reduced.
Disclosure of Invention
The application provides a data storage method, which comprises the following steps:
dividing a memory merging cache region into a plurality of erasure code strips consisting of N data blocks and M check blocks;
matching the data characteristics to be stored with the positions of the data blocks;
storing the matched data into the corresponding data block position;
when the N data blocks are full, M check blocks are calculated;
and storing the N data blocks and the M check blocks into a storage node.
Furthermore, when the matched data is stored in the corresponding data block position, the matched data is preferentially placed in the data block of the erasure code stripe cache area in which the data is written.
Further, when the stored data is the same data stream, the method includes the steps of:
(1) if the data length is larger than N data blocks, calculating an erasure code;
(2) if the data length is less than N data blocks but greater than one data block length, then each fully written data block is placed in the data block of the erasure code stripe buffer, and the rest of the data waits for the data stream to be one data block full:
if the cache region of the erasure code strip is not full, waiting for the erasure code strip to be written to full or overtime;
if the erasure code stripe buffer is full, the erasure code is calculated and the redundant data blocks are placed into a new erasure code stripe buffer.
(3) If the data is smaller than the length of one data block, waiting for the length of one data block to be written, if the data is still not full after overtime, placing the data block into an erasure code strip cache region according to the boundary of the data block, and waiting for the data block to be written.
The present invention also discloses a data storage device comprising:
a function dividing unit: the memory merging area is divided into a plurality of erasure code strips consisting of N data blocks and M check blocks;
a matching unit: matching the data characteristics to be stored with the positions of the data blocks;
a storage unit: the data storage module is used for storing the matched data into the corresponding data block position;
a checking unit: for performing erasure code checking on the stored data.
The present invention also discloses a storage medium, in which a computer program is stored, which, when running on a computer, causes the computer to execute the above-mentioned data storage method.
The data storage method and the data storage system provided by the embodiment of the application do not need to analyze formats of different codes, can store different data in the same coding mode environment, greatly improve the read-write performance of the stored data, and enable a user to efficiently access the data and recover the data:
1. under the scene of a large number of small files, an erasure code stripe (N + M) is constructed by all data blocks of a plurality of users, bucket names and prefixes in the system at the same time, so that the full stripe can be met more quickly, data can be landed as soon as possible, continuous data writing performance is higher for a hard disk, and reliability and writing performance are improved.
2. Different users, bucket names and data blocks of the prefixes are respectively and internally continuous, so that data can be continuously read when being read, and the access performance is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments will be briefly introduced below. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a schematic flow chart of a data storage method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of storage according to the storage data characteristics in the embodiment of the present invention;
fig. 3 is a schematic diagram illustrating a process of storing and verifying data according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Erasure Coding (EC) is a method of data protection that segments data into fragments, expands, encodes, and stores redundant data blocks in different locations, such as disks, storage nodes, or other geographic locations.
Referring to fig. 1, fig. 1 is a schematic flow chart of a data storage method according to an embodiment of the present invention, which includes the following steps:
s1: dividing a memory merging cache region into a plurality of erasure code strips consisting of N data blocks and M check blocks;
the embodiment of the invention aims to solve the problem that mass storage data is mainly stored by adopting a distributed storage system, and in order to improve the stability and reliability of the system, erasure codes are usually adopted for processing.
When the redundancy level is N + M, M check blocks are calculated from N source data blocks, and the N + M data blocks are respectively stored in N + M hard disks, so that faults of any M hard disks can be tolerated; when the hard disk fails, all source data can be calculated by randomly selecting N normal data blocks. If the N + M data blocks are dispersed on different storage nodes, the fault of the M nodes can be tolerated, and the fault tolerance of the system is greatly improved.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating storage according to storage data characteristics according to an embodiment of the present invention.
In this embodiment, the memory merge buffer of fig. 2 is divided into a data block 1 (i.e., N data blocks) with four data block lengths and a parity block 2 (i.e., M parity blocks) with two data block lengths.
S2: matching the data characteristics to be stored with the positions of the data blocks;
the stored data has many data characteristics which can be used for identification, and the data characteristics are matched with the data block positions of the erasure code strip cache region according to the characteristics in the system for receiving the data at the current moment, such as the number of user IDs, the number of bucket names, the number of file prefixes or the stream ID consisting of the user IDs, the bucket names and the file prefixes.
S3: storing the matched data into the corresponding data block position;
referring to fig. 2, in this embodiment, assuming that data of four users needs to be stored currently, the data may be stored after matching according to data characteristics of the four users, for example, assuming that the data is stored according to ID characteristics of the users, the data of a first user ID1 is stored in a first data block, and is sequentially stored according to ID numbers of the users, or is sequentially stored according to the number of buckets, and the like.
After receiving data write-in requests from a plurality of storage user IDs, a plurality of bucket names and file prefixes, the storage nodes put the data write-in requests into different data block positions according to different storage user IDs, bucket names and file prefixes and continuously put the data write-in requests. Thus, data with the same storage user ID, or bucket name, or file prefix can be stored in the data block continuously.
For data of users, especially files of the same user, the same bucket and the same naming prefix have internal aggregation. That is, the erasure codes are often read in a continuous time period when being read, and therefore, in the process that the erasure codes are distributed in a plurality of data nodes, if the continuity of data storage is ensured, in reading data, one-time reading operation can continuously read the data and place the data in a memory or a cache node, and the reading performance can be improved.
S4: when the N data blocks are full, M check blocks are calculated;
referring to fig. 3, fig. 3 is a schematic diagram illustrating a storage and verification process of data according to an embodiment of the present invention, where the stored data is the same data stream, the method includes:
(1) if the data length is larger than N data blocks, calculating an erasure code;
(2) if the data length is smaller than N data blocks but greater than one data block length, in order to improve the storage effect, first, each fully written data block in the data length is placed in a data block of an erasure code stripe buffer, and the rest of data waits for the data stream to be full of one data block size, which specifically includes:
if the cache region of the erasure code strip is not full, waiting for the erasure code strip to be written to full or overtime;
if the erasure code stripe buffer is full, the erasure code is calculated and the redundant data blocks are placed into a new erasure code stripe buffer.
(3) If the data is smaller than the length of one data block, waiting for the length of one data block to be written, if the data is still not full after overtime, placing the data block into an erasure code strip cache region according to the boundary of the data block, and waiting for the data block to be written.
S5: and finally, after the erasure code is calculated, storing the N data blocks and the M check blocks into the storage nodes, wherein the N data blocks and the M check blocks can be dispersed into different or same storage nodes.
The invention does not relate to the analysis and processing of the coding mode of the data to the storage of the data, thereby greatly reducing the complexity and the engineering quantity of the design of a storage system, and the optimized storage method improves the read-write performance of the stored data, so that a user can efficiently and lowly access the data and recover the data:
1. under the scene of a large number of small files, an erasure code stripe (N + M) is constructed by all data blocks of a plurality of users, bucket names and prefixes in the system at the same time, so that the full stripe can be met more quickly, data can be landed as soon as possible, continuous data writing performance is higher for a hard disk, and reliability and writing performance are improved.
2. Different users, bucket names and data blocks of the prefixes are respectively and internally continuous, so that data can be continuously read when being read, and the access performance is improved.
An embodiment of the present application further provides a data storage device, which includes a function dividing unit, a matching unit, a storage unit, and a checking unit, wherein:
the functional division unit is used for dividing the memory merging area into a plurality of erasure code strips formed by N data blocks and M check blocks, the matching unit is used for matching the data characteristics to be stored with the positions of the data blocks, the storage unit is used for storing the matched data into the corresponding positions of the data blocks, and the check unit is used for carrying out erasure code check on the stored data. For a detailed implementation, please refer to the above description of the method and not to detail.
An embodiment of the present application further provides a storage medium, where a computer program is stored in the storage medium, and when the computer program runs on a computer, the computer executes the data storage device according to any of the above embodiments.
It should be noted that, all or part of the steps in the methods of the above embodiments may be implemented by hardware related to instructions of a computer program, which may be stored in a computer-readable storage medium, which may include, but is not limited to: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.
The data storage method, system and storage medium provided by the embodiments of the present application are described in detail above, and the principles and embodiments of the present application are described herein using specific examples, and the description of the embodiments is only used to help understanding the present application. Meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (5)
1. A method of storing data, comprising the steps of:
dividing a memory merging cache region into a plurality of erasure code strips consisting of N data blocks and M check blocks;
matching the data characteristics to be stored with the positions of the data blocks;
storing the matched data into the corresponding data block position;
when the N data blocks are full, M check blocks are calculated;
and storing the N data blocks and the M check blocks into a storage node.
2. The data storage method according to claim 1, wherein the matched data is stored in the corresponding data block position, and is preferentially placed in the data block of the erasure code stripe buffer where the data has been written.
3. A data storage method according to claim 1 or 2, wherein when the stored data is the same data stream, the steps comprise:
(1) if the data length is larger than N data blocks, calculating an erasure code;
(2) if the data length is less than N data blocks but greater than one data block length, then each fully written data block is placed in the data block of the erasure code stripe buffer, and the rest of the data waits for the data stream to be one data block full:
if the cache region of the erasure code strip is not full, waiting for the erasure code strip to be written to full or overtime;
if the erasure code stripe buffer area is full, the erasure code is calculated, and redundant data blocks are placed into a new erasure code stripe buffer area;
(3) if the data is smaller than the length of one data block, waiting for the length of one data block to be written, if the data is still not full after overtime, placing the data block into an erasure code strip cache region according to the boundary of the data block, and waiting for the data block to be written.
4. A data storage device, comprising:
a function dividing unit: the memory merging area is divided into a plurality of erasure code strips consisting of N data blocks and M check blocks;
a matching unit: matching the data characteristics to be stored with the positions of the data blocks;
a storage unit: the data storage module is used for storing the matched data into the corresponding data block position;
a checking unit: for performing erasure code checking on the stored data.
5. A storage medium in which a computer program is stored, which, when run on a computer, causes the computer to execute a data storage method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010186894.3A CN111400083B (en) | 2020-03-17 | 2020-03-17 | Data storage method and system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010186894.3A CN111400083B (en) | 2020-03-17 | 2020-03-17 | Data storage method and system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111400083A true CN111400083A (en) | 2020-07-10 |
| CN111400083B CN111400083B (en) | 2024-02-23 |
Family
ID=71436371
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010186894.3A Active CN111400083B (en) | 2020-03-17 | 2020-03-17 | Data storage method and system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111400083B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112019788A (en) * | 2020-08-27 | 2020-12-01 | 杭州海康威视系统技术有限公司 | Data storage method, device, system and storage medium |
| CN112199054A (en) * | 2020-12-07 | 2021-01-08 | 上海七牛信息技术有限公司 | File storage method and system |
| CN113190384A (en) * | 2021-05-21 | 2021-07-30 | 重庆紫光华山智安科技有限公司 | Data recovery control method, device, equipment and medium based on erasure codes |
| CN114416437A (en) * | 2022-01-17 | 2022-04-29 | 杭州字节方舟科技有限公司 | Data storage method of self-adaptive redundancy level |
| WO2023279906A1 (en) * | 2021-07-09 | 2023-01-12 | 深圳大普微电子科技有限公司 | Data processing method and apparatus, device, and readable storage medium |
| CN116501267A (en) * | 2023-06-27 | 2023-07-28 | 苏州浪潮智能科技有限公司 | Independent redundant disk array card control method and device |
| CN117873378A (en) * | 2023-12-14 | 2024-04-12 | 天翼云科技有限公司 | Distributed storage system reading optimization scheme based on erasure codes |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105404469A (en) * | 2015-10-22 | 2016-03-16 | 浙江宇视科技有限公司 | Video data storage method and system |
| CN105930103A (en) * | 2016-05-10 | 2016-09-07 | 南京大学 | Distributed storage CEPH based erasure correction code overwriting method |
| CN107094186A (en) * | 2017-06-15 | 2017-08-25 | 深圳市云舒网络技术有限公司 | A kind of correcting and eleting codes optimization method of distributed memory system |
| WO2018001110A1 (en) * | 2016-06-29 | 2018-01-04 | 中兴通讯股份有限公司 | Method and device for reconstructing stored data based on erasure coding, and storage node |
| CN107807794A (en) * | 2017-10-31 | 2018-03-16 | 新华三技术有限公司 | A kind of date storage method and device |
| CN108170555A (en) * | 2017-12-21 | 2018-06-15 | 浙江大华技术股份有限公司 | A kind of data reconstruction method and equipment |
| CN108780386A (en) * | 2017-12-20 | 2018-11-09 | 华为技术有限公司 | A kind of methods, devices and systems of data storage |
| US10310736B1 (en) * | 2016-12-22 | 2019-06-04 | Veritas Technologies Llc | Systems and methods for storing data |
| CN110442535A (en) * | 2019-07-19 | 2019-11-12 | 华中科技大学 | Improve the method and system of distributed solid-state disk key assignments caching system reliability |
| CN110554839A (en) * | 2019-07-30 | 2019-12-10 | 华为技术有限公司 | distributed storage system access method, client and computer program product |
-
2020
- 2020-03-17 CN CN202010186894.3A patent/CN111400083B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105404469A (en) * | 2015-10-22 | 2016-03-16 | 浙江宇视科技有限公司 | Video data storage method and system |
| CN105930103A (en) * | 2016-05-10 | 2016-09-07 | 南京大学 | Distributed storage CEPH based erasure correction code overwriting method |
| WO2018001110A1 (en) * | 2016-06-29 | 2018-01-04 | 中兴通讯股份有限公司 | Method and device for reconstructing stored data based on erasure coding, and storage node |
| US10310736B1 (en) * | 2016-12-22 | 2019-06-04 | Veritas Technologies Llc | Systems and methods for storing data |
| CN107094186A (en) * | 2017-06-15 | 2017-08-25 | 深圳市云舒网络技术有限公司 | A kind of correcting and eleting codes optimization method of distributed memory system |
| CN107807794A (en) * | 2017-10-31 | 2018-03-16 | 新华三技术有限公司 | A kind of date storage method and device |
| CN108780386A (en) * | 2017-12-20 | 2018-11-09 | 华为技术有限公司 | A kind of methods, devices and systems of data storage |
| CN108170555A (en) * | 2017-12-21 | 2018-06-15 | 浙江大华技术股份有限公司 | A kind of data reconstruction method and equipment |
| CN110442535A (en) * | 2019-07-19 | 2019-11-12 | 华中科技大学 | Improve the method and system of distributed solid-state disk key assignments caching system reliability |
| CN110554839A (en) * | 2019-07-30 | 2019-12-10 | 华为技术有限公司 | distributed storage system access method, client and computer program product |
Non-Patent Citations (3)
| Title |
|---|
| XIA, JUNXU 等: "In-network block repairing for erasure coding storage systems", 《CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE》, 25 December 2019 (2019-12-25), pages 1 - 12 * |
| 曾赛峰;屈喜龙;: "云存储环境下分组校验纠删码冗余算法研究", 湖南工程学院学报(自然科学版), no. 04, 25 December 2016 (2016-12-25), pages 45 - 48 * |
| 杨小梅;黎斌;: "海量数据下的特定语义数据检索优化方法研究", 计算机仿真, no. 05, 15 May 2016 (2016-05-15), pages 428 - 432 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112019788A (en) * | 2020-08-27 | 2020-12-01 | 杭州海康威视系统技术有限公司 | Data storage method, device, system and storage medium |
| CN112019788B (en) * | 2020-08-27 | 2023-04-11 | 杭州海康威视系统技术有限公司 | Data storage method, device, system and storage medium |
| CN112199054A (en) * | 2020-12-07 | 2021-01-08 | 上海七牛信息技术有限公司 | File storage method and system |
| CN113190384A (en) * | 2021-05-21 | 2021-07-30 | 重庆紫光华山智安科技有限公司 | Data recovery control method, device, equipment and medium based on erasure codes |
| WO2023279906A1 (en) * | 2021-07-09 | 2023-01-12 | 深圳大普微电子科技有限公司 | Data processing method and apparatus, device, and readable storage medium |
| CN114416437A (en) * | 2022-01-17 | 2022-04-29 | 杭州字节方舟科技有限公司 | Data storage method of self-adaptive redundancy level |
| CN116501267A (en) * | 2023-06-27 | 2023-07-28 | 苏州浪潮智能科技有限公司 | Independent redundant disk array card control method and device |
| CN116501267B (en) * | 2023-06-27 | 2023-09-29 | 苏州浪潮智能科技有限公司 | Independent redundant disk array card control method and device |
| CN117873378A (en) * | 2023-12-14 | 2024-04-12 | 天翼云科技有限公司 | Distributed storage system reading optimization scheme based on erasure codes |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111400083B (en) | 2024-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111400083B (en) | Data storage method and system and storage medium | |
| US11243706B2 (en) | Fragment management method and fragment management apparatus | |
| CN110442535B (en) | Method and system for improving reliability of distributed solid-state disk key value cache system | |
| US11422703B2 (en) | Data updating technology | |
| WO2019001521A1 (en) | Data storage method, storage device, client and system | |
| CN102024034B (en) | Fragment processing method for high-definition media-oriented embedded file system | |
| CN110347344A (en) | It is a kind of that block storage method is automatically configured based on distributed memory system | |
| CN102681791B (en) | Energy saving storage method of cluster storage system | |
| US11656942B2 (en) | Methods for data writing and for data recovery, electronic devices, and program products | |
| CN111309245B (en) | Hierarchical storage writing method and device, reading method and device and system | |
| WO2021184996A1 (en) | Data storage method and apparatus for database | |
| CN103049508A (en) | Method and device for processing data | |
| CN113885809B (en) | Data management system and method | |
| EP4170499A1 (en) | Data storage method, storage system, storage device, and storage medium | |
| CN115904795A (en) | Data storage method and device in storage system | |
| CN101794246A (en) | Backup storage system, backup system, data backup method and recovery method | |
| US20190286374A1 (en) | Selectively improving raid operations latency | |
| CN103064762B (en) | Heavily delete restoration methods and the device of Backup Data | |
| CN107357677A (en) | A kind of data redundancy storage methods of GlusterFS based on correcting and eleting codes | |
| CN110968271B (en) | High-performance data storage method, system and device | |
| CN110058806B (en) | System and method for hybrid data reliability for object storage devices | |
| US10747610B2 (en) | Leveraging distributed metadata to achieve file specific data scrubbing | |
| CN111831230A (en) | Intelligent storage system for high-performance calculation and big data | |
| CN115599315B (en) | Data processing method, device, system, equipment and medium | |
| CN115687170A (en) | Data processing method, storage device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |