CN101467148A - 利用了数据段的相似度的高效数据存储 - Google Patents
利用了数据段的相似度的高效数据存储 Download PDFInfo
- Publication number
- CN101467148A CN101467148A CNA2007800217651A CN200780021765A CN101467148A CN 101467148 A CN101467148 A CN 101467148A CN A2007800217651 A CNA2007800217651 A CN A2007800217651A CN 200780021765 A CN200780021765 A CN 200780021765A CN 101467148 A CN101467148 A CN 101467148A
- Authority
- CN
- China
- Prior art keywords
- section
- new section
- previously stored
- increment
- segment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1748—De-duplication implemented within the file system, e.g. based on file segments
- G06F16/1756—De-duplication implemented within the file system, e.g. based on file segments based on delta files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Processing (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
Description
Claims (23)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201210299545.8A CN102999543B (zh) | 2006-04-11 | 2007-04-11 | 利用了数据段的相似度的高效数据存储 |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/402,631 US7562186B2 (en) | 2006-04-11 | 2006-04-11 | Efficient data storage using resemblance of data segments |
US11/402,631 | 2006-04-11 | ||
PCT/US2007/008989 WO2007120739A2 (en) | 2006-04-11 | 2007-04-11 | Efficient data storage using resemblance of data segments |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201210299545.8A Division CN102999543B (zh) | 2006-04-11 | 2007-04-11 | 利用了数据段的相似度的高效数据存储 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN101467148A true CN101467148A (zh) | 2009-06-24 |
CN101467148B CN101467148B (zh) | 2012-10-10 |
Family
ID=38576924
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN2007800217651A Active CN101467148B (zh) | 2006-04-11 | 2007-04-11 | 利用了数据段的相似度的高效数据存储 |
CN201210299545.8A Active CN102999543B (zh) | 2006-04-11 | 2007-04-11 | 利用了数据段的相似度的高效数据存储 |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201210299545.8A Active CN102999543B (zh) | 2006-04-11 | 2007-04-11 | 利用了数据段的相似度的高效数据存储 |
Country Status (4)
Country | Link |
---|---|
US (1) | US7562186B2 (zh) |
EP (1) | EP2013740B1 (zh) |
CN (2) | CN101467148B (zh) |
WO (1) | WO2007120739A2 (zh) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103220697A (zh) * | 2011-12-31 | 2013-07-24 | 国际商业机器公司 | 用于无线分析的在线和分布式优化方法和系统 |
CN103577276A (zh) * | 2012-07-18 | 2014-02-12 | 深圳市腾讯计算机系统有限公司 | 用户操作数据的备份系统及方法 |
WO2016008070A1 (zh) * | 2014-07-14 | 2016-01-21 | 华为技术有限公司 | 数据写入的方法及装置 |
CN105468533A (zh) * | 2014-09-10 | 2016-04-06 | 华为技术有限公司 | 数据写入方法、装置及存储器 |
Families Citing this family (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060030534A1 (en) | 2002-09-04 | 2006-02-09 | Gabriele Dorn | Treatment of neurological disorders by dsrna administration |
EP2325743B1 (en) | 2003-01-31 | 2012-12-19 | Good Technology Corporation | Asynchronous real-time retrieval of data |
US7844652B2 (en) * | 2006-04-11 | 2010-11-30 | Emc Corporation | Efficient computation of sketches |
US7949824B2 (en) * | 2006-04-11 | 2011-05-24 | Emc Corporation | Efficient data storage using two level delta resemblance |
US8416954B1 (en) | 2008-09-30 | 2013-04-09 | Emc Corporation | Systems and methods for accessing storage or network based replicas of encrypted volumes with no additional key management |
US8261068B1 (en) | 2008-09-30 | 2012-09-04 | Emc Corporation | Systems and methods for selective encryption of operating system metadata for host-based encryption of data at rest on a logical unit |
US7730316B1 (en) * | 2006-09-22 | 2010-06-01 | Fatlens, Inc. | Method for document fingerprinting |
US20080219495A1 (en) * | 2007-03-09 | 2008-09-11 | Microsoft Corporation | Image Comparison |
US8768895B2 (en) * | 2007-04-11 | 2014-07-01 | Emc Corporation | Subsegmenting for efficient storage, resemblance determination, and transmission |
US7925683B2 (en) * | 2008-12-18 | 2011-04-12 | Copiun, Inc. | Methods and apparatus for content-aware data de-duplication |
US8166314B1 (en) | 2008-12-30 | 2012-04-24 | Emc Corporation | Selective I/O to logical unit when encrypted, but key is not available or when encryption status is unknown |
WO2010097960A1 (en) * | 2009-02-25 | 2010-09-02 | Hitachi, Ltd. | Storage system and data processing method for the same |
US8255365B2 (en) * | 2009-06-08 | 2012-08-28 | Symantec Corporation | Source classification for performing deduplication in a backup operation |
US10230692B2 (en) | 2009-06-30 | 2019-03-12 | International Business Machines Corporation | Distributed storage processing module |
US8140821B1 (en) | 2009-12-18 | 2012-03-20 | Emc Corporation | Efficient read/write algorithms and associated mapping for block-level data reduction processes |
US8156306B1 (en) | 2009-12-18 | 2012-04-10 | Emc Corporation | Systems and methods for using thin provisioning to reclaim space identified by data reduction processes |
WO2011113042A2 (en) * | 2010-03-12 | 2011-09-15 | Copiun, Inc. | Distributed catalog, data store, and indexing |
EP2548122B1 (en) | 2010-03-16 | 2021-06-09 | BlackBerry Limited | Highly scalable and distributed data de-duplication |
CN103229161B (zh) | 2010-08-24 | 2016-01-20 | 科派恩股份有限公司 | 连续接入网关和去重数据缓存服务器 |
US10498356B2 (en) | 2011-09-13 | 2019-12-03 | Exagrid Systems, Inc. | Systems and methods for version chain clustering |
US11336295B2 (en) | 2011-09-13 | 2022-05-17 | Exagrid Systems, Inc. | Systems and methods for version chain clustering |
US10114831B2 (en) * | 2012-08-16 | 2018-10-30 | Exagrid Systems, Inc. | Delta version clustering and re-anchoring |
CN103260182A (zh) * | 2013-04-27 | 2013-08-21 | 苏州洁祥电子有限公司 | 车联网系统及其数据备份方法 |
US10387374B2 (en) | 2015-02-27 | 2019-08-20 | Exagrid Systems, Inc. | Scalable grid deduplication |
US10073855B2 (en) | 2015-05-21 | 2018-09-11 | Exagrid Systems, Inc. | Dynamic and optimized management of grid system resources |
US10303656B2 (en) | 2015-08-13 | 2019-05-28 | Exagrid Systems, Inc. | Parallelizing and deduplicating backup data |
US11150997B2 (en) | 2015-08-19 | 2021-10-19 | Exagrid Systems, Inc. | Adaptive bandwidth management of a replication process |
CN105743509B (zh) * | 2016-01-26 | 2019-05-24 | 华为技术有限公司 | 数据压缩装置及方法 |
CN106844479B (zh) * | 2016-12-23 | 2020-07-07 | 光锐恒宇(北京)科技有限公司 | 文件的压缩、解压方法及装置 |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5574906A (en) * | 1994-10-24 | 1996-11-12 | International Business Machines Corporation | System and method for reducing storage requirement in backup subsystems utilizing segmented compression and differencing |
FR2767939B1 (fr) * | 1997-09-04 | 2001-11-02 | Bull Sa | Procede d'allocation de memoire dans un systeme de traitement de l'information multiprocesseur |
GB2341249A (en) * | 1998-08-17 | 2000-03-08 | Connected Place Limited | A method of generating a difference file defining differences between an updated file and a base file |
US6343341B1 (en) * | 1999-08-20 | 2002-01-29 | Microsoft Corporation | Efficient access to variable-length data on a sequential access storage medium |
US6901414B2 (en) * | 2000-11-30 | 2005-05-31 | Storage Technology Corporation | Method and system of storing a main data file and deltas in a storage device for determining new data files from the main data file and the deltas |
JP4205350B2 (ja) * | 2002-02-28 | 2009-01-07 | 富士通株式会社 | 差分データ生成方法、プログラム、記録媒体及び装置 |
US6667700B1 (en) * | 2002-10-30 | 2003-12-23 | Nbt Technology, Inc. | Content-based segmentation scheme for data compression in storage and transmission including hierarchical segment representation |
JP4302970B2 (ja) * | 2002-12-16 | 2009-07-29 | 富士通株式会社 | 差分更新方法、プログラム及び装置 |
US6928526B1 (en) * | 2002-12-20 | 2005-08-09 | Datadomain, Inc. | Efficient data storage system |
US20060047855A1 (en) * | 2004-05-13 | 2006-03-02 | Microsoft Corporation | Efficient chunking algorithm |
US7523098B2 (en) * | 2004-09-15 | 2009-04-21 | International Business Machines Corporation | Systems and methods for efficient data searching, storage and reduction |
US7401192B2 (en) * | 2004-10-04 | 2008-07-15 | International Business Machines Corporation | Method of replicating a file using a base, delta, and reference file |
-
2006
- 2006-04-11 US US11/402,631 patent/US7562186B2/en active Active
-
2007
- 2007-04-11 CN CN2007800217651A patent/CN101467148B/zh active Active
- 2007-04-11 CN CN201210299545.8A patent/CN102999543B/zh active Active
- 2007-04-11 EP EP07755303.0A patent/EP2013740B1/en active Active
- 2007-04-11 WO PCT/US2007/008989 patent/WO2007120739A2/en active Application Filing
Cited By (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103220697A (zh) * | 2011-12-31 | 2013-07-24 | 国际商业机器公司 | 用于无线分析的在线和分布式优化方法和系统 |
CN103220697B (zh) * | 2011-12-31 | 2016-10-05 | 国际商业机器公司 | 用于无线分析的在线和分布式优化方法和系统 |
US9603033B2 (en) | 2011-12-31 | 2017-03-21 | International Business Machines Corporation | Online and distributed optimization framework for wireless analytics |
US9622091B2 (en) | 2011-12-31 | 2017-04-11 | International Business Machines Corporation | Online and distributed optimization framework for wireless analytics |
US9949149B2 (en) | 2011-12-31 | 2018-04-17 | International Business Machines Corporation | Online and distributed optimization framework for wireless analytics |
CN103577276A (zh) * | 2012-07-18 | 2014-02-12 | 深圳市腾讯计算机系统有限公司 | 用户操作数据的备份系统及方法 |
CN103577276B (zh) * | 2012-07-18 | 2017-11-17 | 深圳市腾讯计算机系统有限公司 | 用户操作数据的备份系统及方法 |
WO2016008070A1 (zh) * | 2014-07-14 | 2016-01-21 | 华为技术有限公司 | 数据写入的方法及装置 |
CN105518790A (zh) * | 2014-07-14 | 2016-04-20 | 华为技术有限公司 | 数据写入的方法及装置 |
CN105518790B (zh) * | 2014-07-14 | 2019-05-28 | 华为技术有限公司 | 数据写入的方法及装置 |
CN105468533A (zh) * | 2014-09-10 | 2016-04-06 | 华为技术有限公司 | 数据写入方法、装置及存储器 |
CN105468533B (zh) * | 2014-09-10 | 2019-02-19 | 华为技术有限公司 | 数据写入方法、装置及存储器 |
Also Published As
Publication number | Publication date |
---|---|
US20070239945A1 (en) | 2007-10-11 |
WO2007120739A3 (en) | 2008-11-06 |
EP2013740A2 (en) | 2009-01-14 |
EP2013740A4 (en) | 2011-07-27 |
EP2013740B1 (en) | 2014-11-12 |
CN101467148B (zh) | 2012-10-10 |
WO2007120739A2 (en) | 2007-10-25 |
CN102999543A (zh) | 2013-03-27 |
CN102999543B (zh) | 2016-02-10 |
US7562186B2 (en) | 2009-07-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN101467148B (zh) | 利用了数据段的相似度的高效数据存储 | |
US9678688B2 (en) | System and method for data deduplication for disk storage subsystems | |
US20190340165A1 (en) | Method of reducing redundancy between two or more datasets | |
US9823975B2 (en) | Efficient computation of sketches | |
US9292584B1 (en) | Efficient data communication based on lossless reduction of data by deriving data from prime data elements resident in a content-associative sieve | |
US8214607B2 (en) | Method and apparatus for detecting the presence of subblocks in a reduced-redundancy storage system | |
US8554745B2 (en) | Nearstore compression of data in a storage system | |
US8145863B2 (en) | Efficient data storage using two level delta resemblance | |
US9514179B2 (en) | Table boundary detection in data blocks for compression | |
CN105009067B (zh) | 管理对存储数据单元的操作 | |
KR20150121703A (ko) | 데이터를 저장 및 검색하기 위한 방법 및 시스템 | |
EP4150766A1 (en) | Exploiting locality of prime data for efficient retrieval of data that has been losslessly reduced using a prime data sieve | |
US20220100718A1 (en) | Systems, methods and devices for eliminating duplicates and value redundancy in computer memories | |
TW202311996A (zh) | 用於資料壓縮的系統以及方法 | |
CN115705161A (zh) | 用于划分和加密数据的系统、方法和装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
ASS | Succession or assignment of patent right |
Owner name: EMC COMPANY Free format text: FORMER OWNER: DATA FIELD HOLDING COMPANY Effective date: 20100524 |
|
C41 | Transfer of patent application or patent right or utility model | ||
COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: CALIFORNIA, U.S.A. TO: MASSACHUSETTS, U.S.A. |
|
TA01 | Transfer of patent application right |
Effective date of registration: 20100524 Address after: Massachusetts, USA Applicant after: EMC Corp. Address before: Massachusetts, USA Applicant before: Data domain Holdings Effective date of registration: 20100524 Address after: Massachusetts, USA Applicant after: Data domain Holdings Address before: California, USA Applicant before: Data field LLC |
|
C14 | Grant of patent or utility model | ||
GR01 | Patent grant |