CN106020722A - Method, device and system for deduplication of repeated data of cloud storage system - Google Patents
Method, device and system for deduplication of repeated data of cloud storage system Download PDFInfo
- Publication number
- CN106020722A CN106020722A CN201610334354.9A CN201610334354A CN106020722A CN 106020722 A CN106020722 A CN 106020722A CN 201610334354 A CN201610334354 A CN 201610334354A CN 106020722 A CN106020722 A CN 106020722A
- Authority
- CN
- China
- Prior art keywords
- data
- duplicate removal
- storage
- storage object
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
- G06F3/0641—De-duplication techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for deduplication of repeated data of a cloud storage system. The method includes the steps of: S10. judging whether the type of a data storage object containing repeated data is file data or block data, if the type is file data, entering the Step S11, and if the type is block data, entering the Step S12; S11: determining to de-duplicate the data storage object according to a file level deduplication method; and S12: determining to de-duplicate the data storage object according to a block data level deduplication method. The method determines the corresponding deduplication method according to the type of the data storage object, and can improve the deduplication efficiency of the storage system and the overall utilization rate of a storage resource pool. In addition, the invention also discloses a device for deduplication of repeated data of a cloud storage system and the cloud storage system, and the effects are as mentioned above.
Description
Technical field
The present invention relates to field of computer technology, particularly relate to the repetition of a kind of cloud storage system
Data duplicate removal method, Apparatus and system.
Background technology
Currently, cloud computing is gradually approved by industry, and cloud storage system is the most gradually in social production
With sphere of life plays the most important effect.Cloud storage system also exist substantial amounts of heavy
Complex data, the efficiency that these existence repeating data leverage data storage, access, and
Cause a large amount of wastes of resource.Data to be stored need to repeat the duplicate removal work of data, for
Data to be stored to realize repeating the duplicate removal of data, on the one hand can effectively save depositing of user
Storage space, can save making of the hardware purchase cost of service provider, manpower energy consumption and machine room indirectly
With space etc.；Another aspect, repeats data deduplication and is not transmitted by the Internet or stored many
Part identical data, thus effectively reduces and takies memory space and the network bandwidth, Jin Erti
High access and recall precision.In being embodied as, the duplicate removal mode repeating data has multiple, closes
Suitable duplicate removal mode not only increases the utilization rate of memory resource pool and also improves storage efficiency.
Therefore, how counterweight complex data carries out duplicate removal, with improve memory resource pool utilization rate and
Storage efficiency is those skilled in the art's problem demanding prompt solutions.
Summary of the invention
It is an object of the invention to provide the repetition data duplicate removal method of a kind of cloud storage system, device
And system, for improving utilization rate and the storage efficiency of memory resource pool.
For solving above-mentioned technical problem, the present invention provides the repetition data of a kind of cloud storage system to go
Weighing method, including:
S10: judge the type of data storage object including repetition data be file data also
It is blocks of data, if file data then enters step S11, if blocks of data then enters step
Rapid S12；
S11: determine that described data storage object carries out duplicate removal according to file-level duplicate removal mode；
S12: determine that described data storage object carries out duplicate removal according to blocks of data level duplicate removal mode.
Preferably, the most also include:
Described data storage object is sent to the file-storage device in memory resource pool with
Described data storage object is carried out duplicate removal and storage.
Preferably, described file-storage device includes the NAS network storage equipment.
Preferably, the most also include:
Described data storage object is sent in the block storage device to memory resource pool with right
Described data storage object carries out duplicate removal and storage.
Preferably, described piece of storage device includes SAN storage device.
Preferably, also include after step S11 or step S12:
Described data storage object is sent to the object storage device in memory resource pool with
Described data storage object is carried out duplicate removal and storage.
Preferably, described object storage device includes Ceph object storage device.
Preferably, also included before step S10:
Obtain storage request；
Receive the data to be stored that described storage request is corresponding；
Judge whether described data to be stored include described repetition data, if it is, determine
Described data to be stored are described data storage object.
A kind of repetition data duplicate removal device of cloud storage system, including:
Data storage object type judging module, for judging that the data including repetition data are deposited
The type of storage object is file data or blocks of data；
Duplicate removal mode selects module, for judging in described data storage object type judging module
Go out the type of described data storage object when being file data, determine described data storage object by
Duplicate removal is carried out according to file-level duplicate removal mode, or, for sentencing in described data storage object type
Disconnected module is judged, when the type of described data storage object is blocks of data, to determine that described data are deposited
Storage object carries out duplicate removal according to blocks of data level duplicate removal mode.
A kind of cloud storage system, including the repetition data duplicate removal device of described cloud storage system.
The repetition data duplicate removal method of cloud storage system provided by the present invention, Apparatus and system,
After receiving data storage object, first determine whether that the type of data storage object is file data
Or blocks of data, if file data then determines that data storage object is according to file-level removing repeat
Formula carries out duplicate removal；Enter if it is determined that the type of data storage object is blocks of data, determine that data are deposited
Storage object carries out duplicate removal according to blocks of data level duplicate removal mode.The method is according to data storage object
Type determines corresponding duplicate removal mode, it is possible to increase storage system deduplicated efficiency and memory resource pool
Overall utilization rate.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present invention, will use required in embodiment below
Accompanying drawing do simple introduction, it should be apparent that, the accompanying drawing in describing below is only the present invention
Some embodiments, for those of ordinary skill in the art, do not paying creative work
On the premise of, it is also possible to other accompanying drawing is obtained according to these accompanying drawings.
The flow process of the repetition data duplicate removal method of a kind of cloud storage system that Fig. 1 provides for the present invention
Figure；
The structure of the repetition data duplicate removal device of a kind of cloud storage system that Fig. 2 provides for the present invention
Figure；
The structure chart of a kind of cloud storage system that Fig. 3 provides for the present invention.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present invention, to the technical side in the embodiment of the present invention
Case is clearly and completely described, it is clear that described embodiment is only the present invention one
Divide embodiment rather than whole embodiment.Based on the embodiment in the present invention, this area is common
Technical staff under not making creative work premise, the every other embodiment obtained,
Broadly fall into scope.
The core of the present invention is to provide repetition data duplicate removal method and the dress of a kind of cloud storage system
Put.
In order to make those skilled in the art be more fully understood that the present invention program, below in conjunction with the accompanying drawings
The present invention is described in further detail with detailed description of the invention.
The flow process of the repetition data duplicate removal method of a kind of cloud storage system that Fig. 1 provides for the present invention
Figure.The repetition data duplicate removal method of cloud storage system includes:
S10: judge the type of data storage object including repetition data be file data also
It is blocks of data, if file data then enters step S11, if blocks of data then enters step
Rapid S12；
S11: determine that data storage object carries out duplicate removal according to file-level duplicate removal mode；
S12: determine that data storage object carries out duplicate removal according to blocks of data level duplicate removal mode.
In being embodied as, the type of data storage object can be divided into file data type and block
The duplicate removal mode of data type, file data type and blocks of data type is different.If data are deposited
Storage object is file data type, if using blocks of data level duplicate removal mode, then reduces storage system
System deduplicated efficiency and the overall utilization rate of memory resource pool；Identical, if data storage object
For blocks of data type, if using file-level duplicate removal mode, then reduce storage system deduplicated efficiency
Overall utilization rate with memory resource pool.
File-level duplicate removal mode in the present invention refers to: if it find that two identical files,
One of them will the pointer of another file directed be replaced.Which the most simply will
Many parts of access request are linked to same number evidence, do not interfere with the reading performance of data, Er Qieyong
When file is opened at family, it is not required that carry out decompressing or data recombination.
Blocks of data level duplicate removal mode in the present invention refers to: divided by all of data storage object
Solution becomes data block, then by hashing algorithm, creates a cryptographic Hash for each piece, and and its
He compares at the cryptographic Hash of all data blocks, if the cryptographic Hash of two different pieces of information blocks is complete
Unanimously, one of them block will be deleted, and replaces with the pointer pointing to another block.
Which has higher compression ratio, and duplicate removal granularity is thinner, it is possible to preferably improve data deduplication
Efficiency.
The repetition data duplicate removal method of the cloud storage system that the present embodiment provides, when receiving data
After storage object, first determine whether that the type of data storage object is file data or blocks of data,
If file data then determines that data storage object carries out duplicate removal according to file-level duplicate removal mode；
Enter if it is determined that the type of data storage object is blocks of data, determine that data storage object is according to block
Data level duplicate removal mode carries out duplicate removal.The method determines according to the type of data storage object accordingly
Duplicate removal mode, it is possible to increase storage system deduplicated efficiency and the overall utilization rate of memory resource pool.
It is preferably carried out mode, on the basis of above-described embodiment, in step S11 as one
The most also include:
Data storage object is sent in the file-storage device to memory resource pool with logarithm
Duplicate removal and storage is carried out according to storage object.
Determining after data storage object is file data, then by data storage object according to literary composition
Part level duplicate removal mode carries out duplicate removal.Memory resource pool has multiple storage device, for example, it is possible to
It it is the NAS network storage equipment.
It is preferably carried out mode, on the basis of above-described embodiment, in step S12 as one
The most also include:
Data storage object is sent in the block storage device to memory resource pool with to data
Storage object carries out duplicate removal and storage.
Determining after data storage object is blocks of data, then by data storage object according to data
Level duplicate removal mode carries out duplicate removal.Memory resource pool has multiple storage device, for example, it may be
SAN storage device.
It is preferably carried out mode, on the basis of above-described embodiment, in step S11 as one
Or also include after step S12:
Data storage object is sent in the object storage device to memory resource pool with logarithm
Duplicate removal and storage is carried out according to storage object.
In being embodied as, when after the duplicate removal mode determining data storage object, either press
Data storage object can be sent extremely according to file-level duplicate removal mode or data level duplicate removal mode
During object storage is arranged, to complete duplicate removal and storage.Object storage device includes Ceph object
Storage device.
It is preferably carried out mode, on the basis of above-described embodiment, in step S10 as one
The most also include:
Obtain storage request；
Receive the data to be stored that storage request is corresponding；
Judge whether data to be stored include repetition data, if it is, determine number to be stored
According to for data storage object.
It is understood that on the basis of above-described embodiment, also include:
Any one step in recording step S10-S12 or operation letter corresponding to several step
Breath.
Start to go to after this step performed in step S10, by the operation information of this step
Record, such as, the execution time etc., in order to as follow-up issue handling and malfunction elimination
Foundation.For step S11 for step S12 identical.
The structure of the repetition data duplicate removal device of a kind of cloud storage system that Fig. 2 provides for the present invention
Figure.A kind of repetition data duplicate removal device of cloud storage system, including:
Data storage object type judging module 10, for judging to include the data of repetition data
The type of storage object is file data or blocks of data；
Duplicate removal mode selects module 11, for judging in data storage object type judging module
When the type of data storage object is file data, determine that data storage object goes according to file-level
Heavy prescription formula carries out duplicate removal, or, for judging number in data storage object type judging module
When being blocks of data according to the type of storage object, determine that data storage object is according to blocks of data level duplicate removal
Mode carries out duplicate removal.
In being embodied as, the type of data storage object can be divided into file data type and block
The duplicate removal mode of data type, file data type and blocks of data type is different.If data are deposited
Storage object is file data type, if using blocks of data level duplicate removal mode, then reduces storage system
System deduplicated efficiency and the overall utilization rate of memory resource pool；Identical, if data storage object
For blocks of data type, if using file-level duplicate removal mode, then reduce storage system deduplicated efficiency
Overall utilization rate with memory resource pool.
File-level duplicate removal mode in the present invention refers to: if it find that two identical files,
One of them will the pointer of another file directed be replaced.Which the most simply will
Many parts of access request are linked to same number evidence, do not interfere with the reading performance of data, Er Qieyong
When file is opened at family, it is not required that carry out decompressing or data recombination.
Blocks of data level duplicate removal mode in the present invention refers to: divided by all of data storage object
Solution becomes data block, then by hashing algorithm, creates a cryptographic Hash for each piece, and and its
He compares at the cryptographic Hash of all data blocks, if the cryptographic Hash of two different pieces of information blocks is complete
Unanimously, one of them block will be deleted, and replaces with the pointer pointing to another block.
Which has higher compression ratio, and duplicate removal granularity is thinner, it is possible to preferably improve data deduplication
Efficiency.
The repetition data duplicate removal device of the cloud storage system that the present embodiment provides, when data storage is right
After receiving data storage object as type judging module 10, first determine whether data storage object
Type is file data or blocks of data, if file data then duplicate removal mode selects module 11
Determine that data storage object carries out duplicate removal according to file-level duplicate removal mode；If it is determined that data storage
The type of object is that blocks of data enters, duplicate removal mode select module 11 determine data storage object by
Duplicate removal is carried out according to blocks of data level duplicate removal mode.The method determines according to the type of data storage object
Corresponding duplicate removal mode, it is possible to increase storage system deduplicated efficiency and the overall profit of memory resource pool
By rate.
The structure chart of a kind of cloud storage system that Fig. 3 provides for the present invention.A kind of cloud storage system,
Repetition data duplicate removal device 1 including the cloud storage system described in above-described embodiment.Concrete real
Shi Zhong, cloud storage system also includes memory resource pool 2 etc..The repetition data of cloud storage system are gone
Refitting is put the detailed description of the invention of 1 and is seen the description of above-described embodiment.
Above to the repetition data duplicate removal method of cloud storage system provided by the present invention, device and
System is described in detail.In description, each embodiment uses the mode gone forward one by one to describe, often
What individual embodiment stressed is all the difference with other embodiments, between each embodiment
Identical similar portion sees mutually.For device disclosed in embodiment, due to its with
Disclosed in embodiment, method is corresponding, so describe is fairly simple, relevant part sees method
Part illustrates.It should be pointed out that, for those skilled in the art,
Without departing from the principles of the invention, it is also possible to the present invention is carried out some improvement and modification,
These improve and modify in the protection domain also falling into the claims in the present invention.
Professional further appreciates that, describes in conjunction with the embodiments described herein
The unit of each example and algorithm steps, it is possible to electronic hardware, computer software or the two
Be implemented in combination in, in order to clearly demonstrate the interchangeability of hardware and software, in described above
In generally described composition and the step of each example according to function.These functions are actually
Perform with hardware or software mode, depend on application-specific and the design constraint of technical scheme
Condition.Each specifically should being used for can be used different methods to realize institute by professional and technical personnel
The function described, but this realization is it is not considered that beyond the scope of this invention.
The method described in conjunction with the embodiments described herein or the step of algorithm can be direct
Implement with hardware, the software module of processor execution, or the combination of the two.Software module
Random access memory (RAM), internal memory, read only memory (ROM), electrically programmable can be placed in
ROM, electrically erasable ROM, depositor, hard disk, moveable magnetic disc, CD-ROM,
Or in any other form of storage medium well known in technical field.
Claims (10)
1. the repetition data duplicate removal method of a cloud storage system, it is characterised in that including:
S10: judge the type of data storage object including repetition data be file data also
It is blocks of data, if file data then enters step S11, if blocks of data then enters step
Rapid S12；
S11: determine that described data storage object carries out duplicate removal according to file-level duplicate removal mode；
S12: determine that described data storage object carries out duplicate removal according to blocks of data level duplicate removal mode.
The repetition data duplicate removal method of cloud storage system the most according to claim 1, its
It is characterised by, the most also includes:
Described data storage object is sent to the file-storage device in memory resource pool with
Described data storage object is carried out duplicate removal and storage.
The repetition data duplicate removal method of cloud storage system the most according to claim 2, its
Being characterised by, described file-storage device includes the NAS network storage equipment.
The repetition data duplicate removal method of cloud storage system the most according to claim 1, its
It is characterised by, the most also includes:
Described data storage object is sent in the block storage device to memory resource pool with right
Described data storage object carries out duplicate removal and storage.
The repetition data duplicate removal method of cloud storage system the most according to claim 4, its
Being characterised by, described piece of storage device includes SAN storage device.
The repetition data duplicate removal method of cloud storage system the most according to claim 1, its
It is characterised by, also includes after step S11 or step S12:
Described data storage object is sent to the object storage device in memory resource pool with
Described data storage object is carried out duplicate removal and storage.
The repetition data duplicate removal method of cloud storage system the most according to claim 6, its
Being characterised by, described object storage device includes Ceph object storage device.
The repetition data duplicate removal method of cloud storage system the most according to claim 1, its
It is characterised by, also included before step S10:
Obtain storage request；
Receive the data to be stored that described storage request is corresponding；
Judge whether described data to be stored include described repetition data, if it is, determine
Described data to be stored are described data storage object.
9. the repetition data duplicate removal device of a cloud storage system, it is characterised in that including:
Data storage object type judging module, for judging that the data including repetition data are deposited
The type of storage object is file data or blocks of data；
Duplicate removal mode selects module, for judging in described data storage object type judging module
Go out the type of described data storage object when being file data, determine described data storage object by
Duplicate removal is carried out according to file-level duplicate removal mode, or, for sentencing in described data storage object type
Disconnected module is judged, when the type of described data storage object is blocks of data, to determine that described data are deposited
Storage object carries out duplicate removal according to blocks of data level duplicate removal mode.
10. a cloud storage system, it is characterised in that include that the cloud described in claim 9 is deposited
The repetition data duplicate removal device of storage system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610334354.9A CN106020722A (en) | 2016-05-19 | 2016-05-19 | Method, device and system for deduplication of repeated data of cloud storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610334354.9A CN106020722A (en) | 2016-05-19 | 2016-05-19 | Method, device and system for deduplication of repeated data of cloud storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106020722A true CN106020722A (en) | 2016-10-12 |
Family
ID=57095308
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610334354.9A Pending CN106020722A (en) | 2016-05-19 | 2016-05-19 | Method, device and system for deduplication of repeated data of cloud storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106020722A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107977168A (en) * | 2017-12-15 | 2018-05-01 | 安徽长泰信息安全服务有限公司 | A kind of data based on cloud storage disperse storage system |
| CN108345432A (en) * | 2017-01-25 | 2018-07-31 | 三星电子株式会社 | The algorithmic method of Efficient Compression for excess configuration memory system |
| CN108984103A (en) * | 2017-06-02 | 2018-12-11 | 伊姆西Ip控股有限责任公司 | Method and apparatus for duplicate removal |
| CN109743362A (en) * | 2018-12-17 | 2019-05-10 | 南京东大智能化系统有限公司 | A kind of date storage method applied to full format data structure |
| CN111404978A (en) * | 2019-09-06 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Data storage method and cloud storage system |
| CN112511612A (en) * | 2020-11-19 | 2021-03-16 | 中国联合网络通信集团有限公司 | Cloud storage data storage method, device, system, equipment and storage medium |
| CN116204136A (en) * | 2023-05-04 | 2023-06-02 | 山东浪潮科学研究院有限公司 | Data storage and query method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101916171A (en) * | 2010-07-16 | 2010-12-15 | 中国科学院计算技术研究所 | Concurrent hierarchy type replicated data eliminating method and system |
| CN102609215A (en) * | 2012-04-11 | 2012-07-25 | 成都市华为赛门铁克科技有限公司 | Data processing method and device |
| CN103714123A (en) * | 2013-12-06 | 2014-04-09 | 西安工程大学 | Methods for deleting duplicated data and controlling reassembly versions of cloud storage segmented objects of enterprise |
| CN105511812A (en) * | 2015-12-10 | 2016-04-20 | 浪潮(北京)电子信息产业有限公司 | Method and device for optimizing big data of memory system |
-
2016
- 2016-05-19 CN CN201610334354.9A patent/CN106020722A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101916171A (en) * | 2010-07-16 | 2010-12-15 | 中国科学院计算技术研究所 | Concurrent hierarchy type replicated data eliminating method and system |
| CN102609215A (en) * | 2012-04-11 | 2012-07-25 | 成都市华为赛门铁克科技有限公司 | Data processing method and device |
| CN103714123A (en) * | 2013-12-06 | 2014-04-09 | 西安工程大学 | Methods for deleting duplicated data and controlling reassembly versions of cloud storage segmented objects of enterprise |
| CN105511812A (en) * | 2015-12-10 | 2016-04-20 | 浪潮(北京)电子信息产业有限公司 | Method and device for optimizing big data of memory system |
Non-Patent Citations (1)
| Title |
|---|
| 刘洋: "《信息存储技术原理分析》", 31 December 2014 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108345432A (en) * | 2017-01-25 | 2018-07-31 | 三星电子株式会社 | The algorithmic method of Efficient Compression for excess configuration memory system |
| CN108345432B (en) * | 2017-01-25 | 2023-11-07 | 三星电子株式会社 | Algorithmic method for efficient compression of over-configured memory systems |
| CN108984103A (en) * | 2017-06-02 | 2018-12-11 | 伊姆西Ip控股有限责任公司 | Method and apparatus for duplicate removal |
| CN108984103B (en) * | 2017-06-02 | 2021-06-22 | 伊姆西Ip控股有限责任公司 | Method and apparatus for deduplication |
| US11461276B2 (en) | 2017-06-02 | 2022-10-04 | EMC IP Holding Company LLC | Method and device for deduplication |
| CN107977168A (en) * | 2017-12-15 | 2018-05-01 | 安徽长泰信息安全服务有限公司 | A kind of data based on cloud storage disperse storage system |
| CN107977168B (en) * | 2017-12-15 | 2021-01-01 | 安徽长泰信息安全服务有限公司 | Data dispersed storage system based on cloud storage |
| CN109743362A (en) * | 2018-12-17 | 2019-05-10 | 南京东大智能化系统有限公司 | A kind of date storage method applied to full format data structure |
| CN109743362B (en) * | 2018-12-17 | 2024-04-16 | 南京东大智能化系统有限公司 | Data storage method applied to full-format data structure |
| CN111404978B (en) * | 2019-09-06 | 2023-05-02 | 杭州海康威视系统技术有限公司 | Data storage method and cloud storage system |
| CN111404978A (en) * | 2019-09-06 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Data storage method and cloud storage system |
| CN112511612A (en) * | 2020-11-19 | 2021-03-16 | 中国联合网络通信集团有限公司 | Cloud storage data storage method, device, system, equipment and storage medium |
| CN116204136A (en) * | 2023-05-04 | 2023-06-02 | 山东浪潮科学研究院有限公司 | Data storage and query method, device, equipment and storage medium |
| CN116204136B (en) * | 2023-05-04 | 2023-08-15 | 山东浪潮科学研究院有限公司 | Data storage and query method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106020722A (en) | Method, device and system for deduplication of repeated data of cloud storage system | |
| Fu et al. | Aa-dedupe: An application-aware source deduplication approach for cloud backup services in the personal computing environment | |
| CN103778148B (en) | Life cycle management method and equipment for data file of Hadoop distributed file system | |
| US20190391744A1 (en) | Automated selection of functions to reduce storage capacity based on performance requirements | |
| Fu et al. | Application-aware local-global source deduplication for cloud backup services of personal storage | |
| CN103136243B (en) | File system duplicate removal method based on cloud storage and device | |
| US9298385B2 (en) | System, method and computer program product for deduplication aware quality of service over data tiering | |
| US20110184908A1 (en) | Selective data deduplication | |
| US20160259565A1 (en) | Dynamic three-tier data storage utilization | |
| CN106406759B (en) | Data storage method and device | |
| CN110727727B (en) | Statistical method and device for database | |
| CN107870981A (en) | Electronic installation, the method and storage medium of tables of data filing processing | |
| CN105511812A (en) | Method and device for optimizing big data of memory system | |
| CN103399797B (en) | Server resource allocation method and device | |
| CN103955530A (en) | Data reconstruction and optimization method of on-line repeating data deletion system | |
| CN112260694B (en) | Data compression method of simulation file | |
| CN103150260A (en) | Method and device for deleting repeating data | |
| Wang et al. | Exalt: Empowering Researchers to Evaluate {Large-Scale} Storage Systems | |
| CN105630810A (en) | Method for uploading mass small files in distributed storage system | |
| CN106569750A (en) | Data compression method and device | |
| CN104391961A (en) | Tens of millions of small file data read and write solution strategy | |
| EP2811410A1 (en) | Monitoring record management method and device | |
| US10241693B2 (en) | Dynamic two-tier data storage utilization | |
| US9424269B1 (en) | Systems and methods for deduplicating archive objects | |
| CN110851317A (en) | Method, device, equipment and storage medium for predicting IOPS performance data of storage equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161012 |