WO2012109145A3 - Pre-cache similarity-based delta compression for use in a data storage system - Google Patents

Pre-cache similarity-based delta compression for use in a data storage system Download PDF

Info

Publication number
WO2012109145A3
WO2012109145A3 PCT/US2012/023972 US2012023972W WO2012109145A3 WO 2012109145 A3 WO2012109145 A3 WO 2012109145A3 US 2012023972 W US2012023972 W US 2012023972W WO 2012109145 A3 WO2012109145 A3 WO 2012109145A3
Authority
WO
WIPO (PCT)
Prior art keywords
memory
data
delta
cache
locality
Prior art date
Application number
PCT/US2012/023972
Other languages
French (fr)
Other versions
WO2012109145A2 (en
Inventor
Qing Yang
Jin BEN
Original Assignee
Velobit, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Velobit, Inc. filed Critical Velobit, Inc.
Publication of WO2012109145A2 publication Critical patent/WO2012109145A2/en
Publication of WO2012109145A3 publication Critical patent/WO2012109145A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0875—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches with dedicated cache, e.g. instruction or stack
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608—Saving storage space on storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/3084—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction using adaptive string matching, e.g. the Lempel-Ziv method
    • H03M7/3091—Data deduplication
    • H03M7/3093—Data deduplication using fixed length segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0866—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches for peripheral storage systems, e.g. disk cache
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/40—Specific encoding of data in memory or cache
    • G06F2212/401—Compressed data

Abstract

A data storage caching architecture supports using native local memory such as host-based RAM, and if available, Solid State Disk (SSD) memory for storing pre-cache delta-compression based delta, reference, and independent data by exploiting content locality, temporal locality, and spatial locality of data accesses to primary (e.g. disk-based) storage. The architecture makes excellent use of the physical properties of the different types of memory available (fast r/w RAM, low cost fast read SSD, etc) by applying algorithms to determine what types of data to store in each type of memory. Algorithms include similarity detection, delta compression, least popularly used cache management, conservative insertion and promotion cache replacement, and the like.
PCT/US2012/023972 2011-02-11 2012-02-06 Pre-cache similarity-based delta compression for use in a data storage system WO2012109145A2 (en)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US201161441976P 2011-02-11 2011-02-11
US61/441,976 2011-02-11
US201161447208P 2011-02-28 2011-02-28
US61/447,208 2011-02-28
US201161497549P 2011-06-16 2011-06-16
US61/497,549 2011-06-16

Publications (2)

Publication Number Publication Date
WO2012109145A2 WO2012109145A2 (en) 2012-08-16
WO2012109145A3 true WO2012109145A3 (en) 2012-11-22

Family

ID=46639137

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2012/023972 WO2012109145A2 (en) 2011-02-11 2012-02-06 Pre-cache similarity-based delta compression for use in a data storage system

Country Status (1)

Country Link
WO (1) WO2012109145A2 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9413527B2 (en) 2009-04-30 2016-08-09 HGST Netherlands B.V. Optimizing signature computation and sampling for fast adaptive similarity detection based on algorithm-specific performance
US9176883B2 (en) 2009-04-30 2015-11-03 HGST Netherlands B.V. Storage of data reference blocks and deltas in different storage devices
US9582222B2 (en) 2009-04-30 2017-02-28 Western Digital Technologies, Inc. Pre-cache similarity-based delta compression for use in a data storage system
US10430391B2 (en) * 2012-09-28 2019-10-01 Oracle International Corporation Techniques for activity tracking, data classification, and in database archiving
WO2016185252A1 (en) * 2015-05-20 2016-11-24 Telefonaktiebolaget Lm Ericsson (Publ) Storing disposition notifications in a message store
EP3539218B1 (en) 2016-12-15 2021-04-07 Huawei Technologies Co., Ltd. Method and system of similarity-based deduplication
CN109783019B (en) * 2018-12-28 2022-08-19 上海威固信息技术股份有限公司 Intelligent data storage management method and device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030098987A1 (en) * 2001-11-28 2003-05-29 Scott Fiske Image data caching
US20070008330A1 (en) * 2005-07-07 2007-01-11 Via Technologies, Inc. Texture cache control using a data dependent slot selection scheme
US20070124552A1 (en) * 2005-11-25 2007-05-31 Inventec Corporation Memory control method for restoring data in a cache memory

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030098987A1 (en) * 2001-11-28 2003-05-29 Scott Fiske Image data caching
US20070008330A1 (en) * 2005-07-07 2007-01-11 Via Technologies, Inc. Texture cache control using a data dependent slot selection scheme
US20070124552A1 (en) * 2005-11-25 2007-05-31 Inventec Corporation Memory control method for restoring data in a cache memory

Also Published As

Publication number Publication date
WO2012109145A2 (en) 2012-08-16

Similar Documents

Publication Publication Date Title
WO2012109145A3 (en) Pre-cache similarity-based delta compression for use in a data storage system
CN106716334B (en) Efficient data movement within a file system volume
IN2012DN02977A (en)
KR101480659B1 (en) Two-level system main memory
CN105426127A (en) File storage method and apparatus for distributed cluster system
WO2012096503A3 (en) Storage device for adaptively determining a processing technique with respect to a host request based on partition data and an operating method for the storage device
US10810123B1 (en) Flush strategy for using DRAM as cache media system and method
US20140052899A1 (en) Memory address translation method for flash storage system
WO2012109679A3 (en) Apparatus, system, and method for application direct virtual memory management
JP2009020254A5 (en)
GB2506042A (en) Caching data in a storage system having multiple caches
WO2009006113A3 (en) Hierarchical cache tag architecture
WO2015026679A3 (en) Disconnected operation for systems utilizing cloud storage
JP2013521559A5 (en)
JP2017516234A5 (en)
WO2011005763A3 (en) Data transfer management
ATE422688T1 (en) STRATEGIES FOR READING INFORMATION FROM A MASS STORAGE MEDIUM USING A CACHE MEMORY
CN102521330A (en) Mirror distributed storage method under desktop virtual environment
US11734170B2 (en) Host-resident translation layer validity check
WO2012170235A3 (en) Storage architecture for backup application
WO2009158183A3 (en) Apparatus and method for cache utilization
WO2008042283A3 (en) Write-in-place within a write-anywhere filesystem
WO2012015766A3 (en) Cache memory that supports tagless addressing
CN103345449B (en) A kind of fingerprint forecasting method towards data de-duplication technology and system
KR20170129701A (en) Improved storage cache performance by using the compression rate of the data as the basis for cache insertion

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12744271

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205N DATED 18/10/2013)

122 Ep: pct application non-entry in european phase

Ref document number: 12744271

Country of ref document: EP

Kind code of ref document: A2