SG11201707075SA - Deduplication method and storage device - Google Patents

Deduplication method and storage device

Info

Publication number
SG11201707075SA
SG11201707075SA SG11201707075SA SG11201707075SA SG11201707075SA SG 11201707075S A SG11201707075S A SG 11201707075SA SG 11201707075S A SG11201707075S A SG 11201707075SA SG 11201707075S A SG11201707075S A SG 11201707075SA SG 11201707075S A SG11201707075S A SG 11201707075SA
Authority
SG
Singapore
Prior art keywords
storage device
deduplication method
deduplication
storage
Prior art date
Application number
SG11201707075SA
Other languages
English (en)
Inventor
Zongquan Zhang
Chengwei Zhang
Original Assignee
Huawei Tech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Tech Co Ltd filed Critical Huawei Tech Co Ltd
Publication of SG11201707075SA publication Critical patent/SG11201707075SA/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/174—Redundancy elimination performed by the file system
    • G06F16/1748—De-duplication implemented within the file system, e.g. based on file segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/14—Details of searching files based on file metadata
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G06F3/0641—De-duplication techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/3084—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction using adaptive string matching, e.g. the Lempel-Ziv method
    • H03M7/3091—Data deduplication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10—Providing a specific technical effect
    • G06F2212/1041—Resource optimization
    • G06F2212/1044—Space efficiency improvement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Library & Information Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
SG11201707075SA 2015-12-29 2015-12-29 Deduplication method and storage device SG11201707075SA (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/099572 WO2017113123A1 (fr) 2015-12-29 2015-12-29 Procédé de déduplication de données et dispositif de stockage

Publications (1)

Publication Number Publication Date
SG11201707075SA true SG11201707075SA (en) 2017-09-28

Family

ID=59224199

Family Applications (1)

Application Number Title Priority Date Filing Date
SG11201707075SA SG11201707075SA (en) 2015-12-29 2015-12-29 Deduplication method and storage device

Country Status (7)

Country Link
US (1) US10613976B2 (fr)
EP (1) EP3264285A4 (fr)
JP (1) JP6537214B2 (fr)
KR (1) KR102082765B1 (fr)
CN (1) CN107430602B (fr)
SG (1) SG11201707075SA (fr)
WO (1) WO2017113123A1 (fr)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11461269B2 (en) * 2017-07-21 2022-10-04 EMC IP Holding Company Metadata separated container format
US11113153B2 (en) * 2017-07-27 2021-09-07 EMC IP Holding Company LLC Method and system for sharing pre-calculated fingerprints and data chunks amongst storage systems on a cloud local area network
CN108415669A (zh) * 2018-03-15 2018-08-17 深信服科技股份有限公司 存储系统的数据去重方法及装置、计算机装置及存储介质
US11055005B2 (en) 2018-10-12 2021-07-06 Netapp, Inc. Background deduplication using trusted fingerprints
US10852975B2 (en) * 2019-01-24 2020-12-01 EMC IP Holding Company LLC Efficient data deduplication caching
US11392551B2 (en) * 2019-02-04 2022-07-19 EMC IP Holding Company LLC Storage system utilizing content-based and address-based mappings for deduplicatable and non-deduplicatable types of data
CN112783417A (zh) * 2019-11-01 2021-05-11 华为技术有限公司 数据缩减的方法、装置、计算设备和存储介质
CN114077569B (zh) * 2020-08-18 2023-07-18 富泰华工业（深圳）有限公司 压缩数据的方法及设备、解压缩数据的方法及设备
CN111949624B (zh) * 2020-09-11 2022-09-20 苏州浪潮智能科技有限公司 一种数据重删操作的pl超限控制方法、装置及可读存储介质
US11385817B2 (en) * 2020-09-22 2022-07-12 Vmware, Inc. Supporting deduplication in object storage using subset hashes
CN112463077B (zh) * 2020-12-16 2021-11-12 北京云宽志业网络技术有限公司 数据块处理方法、装置、设备及存储介质
CN112328641B (zh) * 2021-01-05 2021-04-20 平安国际智慧城市科技股份有限公司 多维度数据聚合方法、装置及计算机设备
JP7215804B2 (ja) * 2021-05-14 2023-01-31 Ｎｅｃプラットフォームズ株式会社 ストレージ装置、情報処理システム、情報処理方法、およびプログラム
CN113608701A (zh) * 2021-08-18 2021-11-05 合肥大唐存储科技有限公司 一种存储系统中数据管理方法和固态硬盘
US11874821B2 (en) 2021-12-22 2024-01-16 Ebay Inc. Block aggregation for shared streams

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7814129B2 (en) * 2005-03-11 2010-10-12 Ross Neil Williams Method and apparatus for storing data with reduced redundancy using data clusters
JP4768009B2 (ja) * 2005-03-11 2011-09-07 ロックソフト リミテッド データ・クラスタを使用する冗長性の少ないデータを格納する方法
US20100281207A1 (en) * 2009-04-30 2010-11-04 Miller Steven C Flash-based data archive storage system
US20110055471A1 (en) * 2009-08-28 2011-03-03 Jonathan Thatcher Apparatus, system, and method for improved data deduplication
WO2011133443A1 (fr) * 2010-04-19 2011-10-27 Greenbytes, Inc. Procédé d'optimisation de l'usage et de la performance de la mémoire de systèmes de stockage par déduplication de données
US10394757B2 (en) * 2010-11-18 2019-08-27 Microsoft Technology Licensing, Llc Scalable chunk store for data deduplication
US9823981B2 (en) 2011-03-11 2017-11-21 Microsoft Technology Licensing, Llc Backup and restore strategies for data deduplication
US9678863B2 (en) * 2012-06-12 2017-06-13 Sandisk Technologies, Llc Hybrid checkpointed memory
US10318495B2 (en) * 2012-09-24 2019-06-11 Sandisk Technologies Llc Snapshots for a non-volatile device
US8954392B2 (en) * 2012-12-28 2015-02-10 Futurewei Technologies, Inc. Efficient de-duping using deep packet inspection
US9141554B1 (en) * 2013-01-18 2015-09-22 Cisco Technology, Inc. Methods and apparatus for data processing using data compression, linked lists and de-duplication techniques
KR20140114515A (ko) * 2013-03-15 2014-09-29 삼성전자주식회사 불휘발성 메모리 장치 및 그것의 중복 데이터 제거 방법
CN103514250B (zh) * 2013-06-20 2017-04-26 易乐天 一种全局重复数据删除的方法和系统及存储装置
KR101532283B1 (ko) * 2013-11-04 2015-06-30 인하대학교 산학협력단 Ｓｓｄ 기반 ｒａｉｄ 스토리지에서 데이터 및 패리티 디스크의 복합적 중복제거 방법
CN104239518B (zh) * 2014-09-17 2017-09-29 华为技术有限公司 重复数据删除方法和装置

Also Published As

Publication number Publication date
CN107430602A (zh) 2017-12-01
JP6537214B2 (ja) 2019-07-03
WO2017113123A1 (fr) 2017-07-06
EP3264285A1 (fr) 2018-01-03
US10613976B2 (en) 2020-04-07
EP3264285A4 (fr) 2018-05-30
JP2018514045A (ja) 2018-05-31
US20180267896A1 (en) 2018-09-20
KR20170117572A (ko) 2017-10-23
CN107430602B (zh) 2020-05-08
KR102082765B1 (ko) 2020-02-28

Similar Documents

Publication Publication Date Title
ZA201904259B (en) Device and method
SG11201707075SA (en) Deduplication method and storage device
HK1222232A1 (zh) 安全存儲方法及設備
EP3217294A4 (fr) Procédé et appareil d'accès à un fichier et dispositif de mémorisation
EP3035197A4 (fr) Procédé et appareil de migration de fichier et dispositif de stockage
ZA201702900B (en) Device and method
GB201418282D0 (en) Storage device and assembly
HK1209251A1 (en) Data interaction method and device thereof
IL235729A0 (en) A device and method for securely storing information
SG11201704922PA (en) Information push method and device
HK1213042A1 (zh) 種基於步態的定位方法和裝置
IL257595A (en) Storage facility and storage method
TWI560718B (en) Data storage device and encoding method thereof
HK1224448A1 (zh) 種校驗方法和裝置
HK1224045A1 (zh) 種頁面加載方法及裝置
HUE053342T2 (hu) Eljárás és berendezés határszint meghatározáshoz
HUE042079T2 (hu) Eszköz és eljárás szalagelemek toldóillesztésére
GB2525625B (en) Device and method
SG11201701644XA (en) Drawing device and drawing method
IL235083A0 (en) Method and device for liquid atomization
GB201416790D0 (en) Device and method
GB201519443D0 (en) Device and method
EP3128712A4 (fr) Procédé et dispositif de protection de fichier ndm
ZA201607883B (en) Card shooter device and card storage method
GB201806802D0 (en) Foot-measuring device and method