IN2012KO01022A - - Google Patents

Info

Publication number
IN2012KO01022A
IN2012KO01022A IN1022KO2012A IN2012KO01022A IN 2012KO01022 A IN2012KO01022 A IN 2012KO01022A IN 1022KO2012 A IN1022KO2012 A IN 1022KO2012A IN 2012KO01022 A IN2012KO01022 A IN 2012KO01022A
Authority
IN
India
Prior art keywords
chunks
data
cdc
level
size
Prior art date
Application number
Other languages
English (en)
Inventor
Subhra CHAKRABORTY Rajat
Kishore DIDDI Bhanu
Original Assignee
Indian Inst Technology Kharagpur
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Indian Inst Technology Kharagpur filed Critical Indian Inst Technology Kharagpur
Priority to CN201280076874.4A priority Critical patent/CN104813310A/zh
Priority to IN1022KO2012 priority patent/IN2012KO01022A/en
Priority to US13/885,395 priority patent/US9311323B2/en
Priority to PCT/IB2012/055688 priority patent/WO2014037767A1/en
Publication of IN2012KO01022A publication Critical patent/IN2012KO01022A/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/174—Redundancy elimination performed by the file system
    • G06F16/1748—De-duplication implemented within the file system, e.g. based on file segments
    • G06F16/1752—De-duplication implemented within the file system, e.g. based on file segments based on file chunks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G06F3/0613—Improving I/O performance in relation to throughput
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G06F3/0641—De-duplication techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
IN1022KO2012 2012-09-05 2012-10-18 IN2012KO01022A (de)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN201280076874.4A CN104813310A (zh) 2012-09-05 2012-10-18 多级别内联数据去重
IN1022KO2012 IN2012KO01022A (de) 2012-09-05 2012-10-18
US13/885,395 US9311323B2 (en) 2012-09-05 2012-10-18 Multi-level inline data deduplication
PCT/IB2012/055688 WO2014037767A1 (en) 2012-09-05 2012-10-18 Multi-level inline data deduplication

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
IN1022KO2012 IN2012KO01022A (de) 2012-09-05 2012-10-18

Publications (1)

Publication Number Publication Date
IN2012KO01022A true IN2012KO01022A (de) 2015-06-05

Family

ID=50236597

Family Applications (1)

Application Number Title Priority Date Filing Date
IN1022KO2012 IN2012KO01022A (de) 2012-09-05 2012-10-18

Country Status (4)

Country Link
US (1) US9311323B2 (de)
CN (1) CN104813310A (de)
IN (1) IN2012KO01022A (de)
WO (1) WO2014037767A1 (de)

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9424285B1 (en) * 2012-12-12 2016-08-23 Netapp, Inc. Content-based sampling for deduplication estimation
US9465808B1 (en) * 2012-12-15 2016-10-11 Veritas Technologies Llc Deduplication featuring variable-size duplicate data detection and fixed-size data segment sharing
RU2639947C2 (ru) * 2014-02-14 2017-12-25 Хуавэй Текнолоджиз Ко., Лтд. Способ и сервер для поиска точки деления потока данных на основе сервера
CN105446964B (zh) * 2014-05-30 2019-04-26 国际商业机器公司 用于文件的重复数据删除的方法及装置
US9449012B2 (en) * 2014-05-30 2016-09-20 Apple Inc. Cloud library de-duplication
GB2542619A (en) * 2015-09-28 2017-03-29 Fujitsu Ltd A similarity module, a local computer, a server of a data hosting service and associated methods
US10997119B2 (en) * 2015-10-23 2021-05-04 Nutanix, Inc. Reduced size extent identification
CN105808169A (zh) * 2016-03-14 2016-07-27 联想(北京)有限公司 用于数据去重的方法、装置和系统
US10235396B2 (en) 2016-08-29 2019-03-19 International Business Machines Corporation Workload optimized data deduplication using ghost fingerprints
JP6841024B2 (ja) * 2016-12-09 2021-03-10 富士通株式会社 データ処理装置，データ処理プログラムおよびデータ処理方法
US10621144B2 (en) 2017-03-23 2020-04-14 International Business Machines Corporation Parallel deduplication using automatic chunk sizing
US10325021B2 (en) * 2017-06-19 2019-06-18 GM Global Technology Operations LLC Phrase extraction text analysis method and system
US10747729B2 (en) 2017-09-01 2020-08-18 Microsoft Technology Licensing, Llc Device specific chunked hash size tuning
US10372681B2 (en) 2017-09-12 2019-08-06 International Business Machines Corporation Tape drive memory deduplication
US10289335B2 (en) * 2017-09-12 2019-05-14 International Business Machines Corporation Tape drive library integrated memory deduplication
US10678778B1 (en) * 2017-10-19 2020-06-09 EMC IP Holding Company LLC Date deduplication acceleration
CN108427538B (zh) * 2018-03-15 2021-06-04 深信服科技股份有限公司 全闪存阵列的存储数据压缩方法、装置、及可读存储介质
US11079954B2 (en) * 2018-08-21 2021-08-03 Samsung Electronics Co., Ltd. Embedded reference counter and special data pattern auto-detect
US10248646B1 (en) 2018-08-22 2019-04-02 Cognigo Research Ltd. Token matching in large document corpora
CN111291770B (zh) * 2018-12-06 2023-07-25 华为技术有限公司 一种参数配置方法及装置
JP7295422B2 (ja) * 2019-09-10 2023-06-21 富士通株式会社 情報処理装置および情報処理プログラム
US11119995B2 (en) 2019-12-18 2021-09-14 Ndata, Inc. Systems and methods for sketch computation
US10938961B1 (en) 2019-12-18 2021-03-02 Ndata, Inc. Systems and methods for data deduplication by generating similarity metrics using sketch computation
US20230221864A1 (en) * 2022-01-10 2023-07-13 Vmware, Inc. Efficient inline block-level deduplication using a bloom filter and a small in-memory deduplication hash table

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6829355B2 (en) * 2001-03-05 2004-12-07 The United States Of America As Represented By The National Security Agency Device for and method of one-way cryptographic hashing
US7836387B1 (en) 2005-04-29 2010-11-16 Oracle America, Inc. System and method for protecting data across protection domain boundaries
US8527482B2 (en) * 2008-06-06 2013-09-03 Chrysalis Storage, Llc Method for reducing redundancy between two or more datasets
US8161255B2 (en) * 2009-01-06 2012-04-17 International Business Machines Corporation Optimized simultaneous storing of data into deduplicated and non-deduplicated storage pools
US8321648B2 (en) 2009-10-26 2012-11-27 Netapp, Inc Use of similarity hash to route data for improved deduplication in a storage server cluster
US9401967B2 (en) * 2010-06-09 2016-07-26 Brocade Communications Systems, Inc. Inline wire speed deduplication system
US20120053970A1 (en) * 2010-08-25 2012-03-01 International Business Machines Corporation Systems and methods for dynamic composition of business processes
EP2612246A4 (de) * 2010-08-31 2014-04-09 Nec Corp Speichersystem
US20120089579A1 (en) * 2010-10-08 2012-04-12 Sandeep Ranade Compression pipeline for storing data in a storage cloud
CN102082575A (zh) * 2010-12-14 2011-06-01 江苏格物信息科技有限公司 基于预分块及滑动窗口的重复数据消除方法
US20150134623A1 (en) * 2011-02-17 2015-05-14 Jitcomm Networks Pte Ltd Parallel data partitioning
CN102253820B (zh) * 2011-06-16 2013-03-20 华中科技大学 一种流式重复数据检测方法

Also Published As

Publication number Publication date
CN104813310A (zh) 2015-07-29
US20140114934A1 (en) 2014-04-24
WO2014037767A1 (en) 2014-03-13
US9311323B2 (en) 2016-04-12

Similar Documents

Publication Publication Date Title
IN2012KO01022A (de)
WO2015066719A3 (en) Use of solid state storage devices and the like in data deduplication
WO2014130800A3 (en) Deduplication storage system with efficient reference updating and space reclamation
EP3113043A4 (de) Verfahren, vorrichtung und host zur aktualisierung von metadaten, die in spalten eines verteilten dateisystems gespeichert sind
WO2014001568A3 (en) Method and apparatus for realizing a dynamically typed file or object system enabling a user to perform calculations over the fields associated with the files or objects in the system
WO2012125314A3 (en) Backup and restore strategies for data deduplication
GB201302917D0 (en) Hybrid backup and restore of very large file system using metadata image backup and traditional backup
CA2902821C (en) System for metadata management
WO2014150277A3 (en) Methods and systems for providing secure transactions
GB2525346A (en) Integrity checking and selective deduplication based on network parameters
WO2013019869A3 (en) Data fingerpringting for copy accuracy assurance
WO2012092212A3 (en) Using index partitioning and reconciliation for data deduplication
WO2014159781A3 (en) Caching content addressable data chunks for storage virtualization
GB2508325A (en) Scalable deduplication system with small blocks
WO2012083267A3 (en) Garbage collection and hotspots relief for a data deduplication chunk store
EP3923003A4 (de) Inselbildungsverfahren, vorrichtung und computerlesbares speichermedium
EP3742803A4 (de) Zellenneuauswahlverfahren und -vorrichtung sowie computerspeichermedium
WO2013187901A3 (en) Data deduplication management
EP3401798A4 (de) Sortierverfahren für grobe auswahl von push-informationen, vorrichtung und computerspeichermedium
WO2014089230A3 (en) Storing and retrieving data in a data file
WO2014174380A3 (en) Creating a universally deduplicatable archive volume
TW201612805A (en) Performance evaluation device, manipulating method and program therefor
WO2013068530A3 (en) Logically and end-user-specific physically storing an electronic file
WO2014190210A3 (en) Document management on a public document system
WO2014003707A3 (en) Hardware-based accelerator for managing copy-on-write