SG11201707075SA - Deduplication method and storage device - Google Patents

Deduplication method and storage device

Info

Publication number
SG11201707075SA
SG11201707075SA SG11201707075SA SG11201707075SA SG11201707075SA SG 11201707075S A SG11201707075S A SG 11201707075SA SG 11201707075S A SG11201707075S A SG 11201707075SA SG 11201707075S A SG11201707075S A SG 11201707075SA SG 11201707075S A SG11201707075S A SG 11201707075SA
Authority
SG
Singapore
Prior art keywords
storage device
deduplication method
deduplication
storage
Prior art date
Application number
SG11201707075SA
Inventor
Zongquan Zhang
Chengwei Zhang
Original Assignee
Huawei Tech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Tech Co Ltd filed Critical Huawei Tech Co Ltd
Publication of SG11201707075SA publication Critical patent/SG11201707075SA/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/174—Redundancy elimination performed by the file system
    • G06F16/1748—De-duplication implemented within the file system, e.g. based on file segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/14—Details of searching files based on file metadata
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G06F3/0641—De-duplication techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/3084—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction using adaptive string matching, e.g. the Lempel-Ziv method
    • H03M7/3091—Data deduplication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10—Providing a specific technical effect
    • G06F2212/1041—Resource optimization
    • G06F2212/1044—Space efficiency improvement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
SG11201707075SA 2015-12-29 2015-12-29 Deduplication method and storage device SG11201707075SA (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/099572 WO2017113123A1 (en) 2015-12-29 2015-12-29 Data deduplication method and storage device

Publications (1)

Publication Number Publication Date
SG11201707075SA true SG11201707075SA (en) 2017-09-28

Family

ID=59224199

Family Applications (1)

Application Number Title Priority Date Filing Date
SG11201707075SA SG11201707075SA (en) 2015-12-29 2015-12-29 Deduplication method and storage device

Country Status (7)

Country Link
US (1) US10613976B2 (en)
EP (1) EP3264285A4 (en)
JP (1) JP6537214B2 (en)
KR (1) KR102082765B1 (en)
CN (1) CN107430602B (en)
SG (1) SG11201707075SA (en)
WO (1) WO2017113123A1 (en)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11461269B2 (en) * 2017-07-21 2022-10-04 EMC IP Holding Company Metadata separated container format
US11113153B2 (en) * 2017-07-27 2021-09-07 EMC IP Holding Company LLC Method and system for sharing pre-calculated fingerprints and data chunks amongst storage systems on a cloud local area network
CN108415669A (en) * 2018-03-15 2018-08-17 深信服科技股份有限公司 The data duplicate removal method and device of storage system, computer installation and storage medium
US11055005B2 (en) 2018-10-12 2021-07-06 Netapp, Inc. Background deduplication using trusted fingerprints
US10852975B2 (en) * 2019-01-24 2020-12-01 EMC IP Holding Company LLC Efficient data deduplication caching
US11392551B2 (en) * 2019-02-04 2022-07-19 EMC IP Holding Company LLC Storage system utilizing content-based and address-based mappings for deduplicatable and non-deduplicatable types of data
CN112783417A (en) * 2019-11-01 2021-05-11 华为技术有限公司 Data reduction method and device, computing equipment and storage medium
CN114077569B (en) * 2020-08-18 2023-07-18 富泰华工业（深圳）有限公司 Method and device for compressing data, and method and device for decompressing data
CN111949624B (en) * 2020-09-11 2022-09-20 苏州浪潮智能科技有限公司 Pl overrun control method and device for data deduplication operation and readable storage medium
US11385817B2 (en) * 2020-09-22 2022-07-12 Vmware, Inc. Supporting deduplication in object storage using subset hashes
CN112463077B (en) * 2020-12-16 2021-11-12 北京云宽志业网络技术有限公司 Data block processing method, device, equipment and storage medium
CN112328641B (en) * 2021-01-05 2021-04-20 平安国际智慧城市科技股份有限公司 Multi-dimensional data aggregation method and device and computer equipment
JP7215804B2 (en) * 2021-05-14 2023-01-31 Ｎｅｃプラットフォームズ株式会社 Storage device, information processing system, information processing method, and program
US11874821B2 (en) 2021-12-22 2024-01-16 Ebay Inc. Block aggregation for shared streams

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4768009B2 (en) * 2005-03-11 2011-09-07 ロックソフト リミテッド How to store less redundant data using a data cluster
US7814129B2 (en) * 2005-03-11 2010-10-12 Ross Neil Williams Method and apparatus for storing data with reduced redundancy using data clusters
US20100281207A1 (en) * 2009-04-30 2010-11-04 Miller Steven C Flash-based data archive storage system
US20110055471A1 (en) * 2009-08-28 2011-03-03 Jonathan Thatcher Apparatus, system, and method for improved data deduplication
US9047301B2 (en) * 2010-04-19 2015-06-02 Greenbytes, Inc. Method for optimizing the memory usage and performance of data deduplication storage systems
US10394757B2 (en) * 2010-11-18 2019-08-27 Microsoft Technology Licensing, Llc Scalable chunk store for data deduplication
US9823981B2 (en) * 2011-03-11 2017-11-21 Microsoft Technology Licensing, Llc Backup and restore strategies for data deduplication
US9678863B2 (en) * 2012-06-12 2017-06-13 Sandisk Technologies, Llc Hybrid checkpointed memory
US10318495B2 (en) * 2012-09-24 2019-06-11 Sandisk Technologies Llc Snapshots for a non-volatile device
US8954392B2 (en) * 2012-12-28 2015-02-10 Futurewei Technologies, Inc. Efficient de-duping using deep packet inspection
US9141554B1 (en) * 2013-01-18 2015-09-22 Cisco Technology, Inc. Methods and apparatus for data processing using data compression, linked lists and de-duplication techniques
KR20140114515A (en) * 2013-03-15 2014-09-29 삼성전자주식회사 Nonvolatile memory device and deduplication method thereof
CN103514250B (en) * 2013-06-20 2017-04-26 易乐天 Method and system for deleting global repeating data and storage device
KR101532283B1 (en) * 2013-11-04 2015-06-30 인하대학교 산학협력단 A Unified De-duplication Method of Data and Parity Disks in SSD-based RAID Storage
CN104239518B (en) 2014-09-17 2017-09-29 华为技术有限公司 Data de-duplication method and device

Also Published As

Publication number Publication date
CN107430602B (en) 2020-05-08
EP3264285A1 (en) 2018-01-03
WO2017113123A1 (en) 2017-07-06
KR102082765B1 (en) 2020-02-28
JP2018514045A (en) 2018-05-31
US20180267896A1 (en) 2018-09-20
EP3264285A4 (en) 2018-05-30
CN107430602A (en) 2017-12-01
KR20170117572A (en) 2017-10-23
JP6537214B2 (en) 2019-07-03
US10613976B2 (en) 2020-04-07

Similar Documents

Publication Publication Date Title
ZA201904259B (en) Device and method
SG11201707075SA (en) Deduplication method and storage device
HK1222232A1 (en) Secure storing method and device
EP3217294A4 (en) File access method and apparatus and storage device
ZA201702900B (en) Device and method
GB201418282D0 (en) Storage device and assembly
EP3035197A4 (en) File migration method and apparatus and storage device
HK1209251A1 (en) Data interaction method and device thereof
IL235729A0 (en) Secure storage device and method
SG11201704922PA (en) Information push method and device
HK1213042A1 (en) Gait-based method and device for locating
IL257595A (en) Storage apparatus and storage method
TWI560718B (en) Data storage device and encoding method thereof
HK1224448A1 (en) Verification method and device
HK1224045A1 (en) Method and device for loading page
HUE053342T2 (en) Method and device for limit state determination
PL3186074T3 (en) Device and method for butt-splicing strip members
GB2525625B (en) Device and method
SG11201701644XA (en) Drawing device and drawing method
IL235083A0 (en) Liquid-atomization method and device
GB201416790D0 (en) Device and method
GB201519443D0 (en) Device and method
EP3128712A4 (en) Ndm file protection method and device
ZA201607883B (en) Card shooter device and card storage method
GB201806802D0 (en) Foot-measuring device and method