CN102479245B - 数据区块的切分方法 - Google Patents
数据区块的切分方法 Download PDFInfo
- Publication number
- CN102479245B CN102479245B CN2010105895679A CN201010589567A CN102479245B CN 102479245 B CN102479245 B CN 102479245B CN 2010105895679 A CN2010105895679 A CN 2010105895679A CN 201010589567 A CN201010589567 A CN 201010589567A CN 102479245 B CN102479245 B CN 102479245B
- Authority
- CN
- China
- Prior art keywords
- block
- target data
- data block
- file
- moving window
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1748—De-duplication implemented within the file system, e.g. based on file segments
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Collating Specific Patterns (AREA)
Abstract
Description
Claims (5)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2010105895679A CN102479245B (zh) | 2010-11-30 | 2010-11-30 | 数据区块的切分方法 |
US13/070,052 US20120136842A1 (en) | 2010-11-30 | 2011-03-23 | Partitioning method of data blocks |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2010105895679A CN102479245B (zh) | 2010-11-30 | 2010-11-30 | 数据区块的切分方法 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN102479245A CN102479245A (zh) | 2012-05-30 |
CN102479245B true CN102479245B (zh) | 2013-07-17 |
Family
ID=46091893
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN2010105895679A Expired - Fee Related CN102479245B (zh) | 2010-11-30 | 2010-11-30 | 数据区块的切分方法 |
Country Status (2)
Country | Link |
---|---|
US (1) | US20120136842A1 (zh) |
CN (1) | CN102479245B (zh) |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102143039B (zh) * | 2010-06-29 | 2013-11-06 | 华为技术有限公司 | 数据压缩中数据分段方法及设备 |
EP3217298B1 (en) * | 2012-12-12 | 2018-08-29 | Huawei Technologies Co., Ltd. | Data processing method and apparatus in cluster system |
CN103078709B (zh) * | 2013-01-05 | 2016-04-13 | 中国科学院深圳先进技术研究院 | 数据冗余识别方法 |
US9306997B2 (en) | 2013-01-16 | 2016-04-05 | Cisco Technology, Inc. | Method for optimizing WAN traffic with deduplicated storage |
US9509736B2 (en) | 2013-01-16 | 2016-11-29 | Cisco Technology, Inc. | Method for optimizing WAN traffic |
US9300748B2 (en) * | 2013-01-16 | 2016-03-29 | Cisco Technology, Inc. | Method for optimizing WAN traffic with efficient indexing scheme |
CN104348571B (zh) * | 2013-07-23 | 2018-02-06 | 华为技术有限公司 | 数据分块方法及装置 |
EP3015999A4 (en) * | 2013-09-29 | 2016-08-17 | Huawei Tech Co Ltd | METHOD OF PROCESSING DATA, SYSTEM AND CLIENT |
US10410244B2 (en) | 2013-11-13 | 2019-09-10 | Bi Science (2009) Ltd | Behavioral content discovery |
MX358948B (es) | 2014-02-14 | 2018-09-07 | Huawei Tech Co Ltd | Metodo y servidor para buscar un punto de division de corriente de datos basado en servidor. |
CN105446964B (zh) * | 2014-05-30 | 2019-04-26 | 国际商业机器公司 | 用于文件的重复数据删除的方法及装置 |
US9760578B2 (en) * | 2014-07-23 | 2017-09-12 | International Business Machines Corporation | Lookup-based data block alignment for data deduplication |
CN112783417A (zh) | 2019-11-01 | 2021-05-11 | 华为技术有限公司 | 数据缩减的方法、装置、计算设备和存储介质 |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101706825A (zh) * | 2009-12-10 | 2010-05-12 | 华中科技大学 | 一种基于文件内容类型的重复数据删除方法 |
CN101814045A (zh) * | 2010-04-22 | 2010-08-25 | 华中科技大学 | 一种用于备份服务的数据组织方法 |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7281006B2 (en) * | 2003-10-23 | 2007-10-09 | International Business Machines Corporation | System and method for dividing data into predominantly fixed-sized chunks so that duplicate data chunks may be identified |
US8315984B2 (en) * | 2007-05-22 | 2012-11-20 | Netapp, Inc. | System and method for on-the-fly elimination of redundant data |
US8086799B2 (en) * | 2008-08-12 | 2011-12-27 | Netapp, Inc. | Scalable deduplication of stored data |
US9239843B2 (en) * | 2009-12-15 | 2016-01-19 | Symantec Corporation | Scalable de-duplication for storage systems |
US8442942B2 (en) * | 2010-03-25 | 2013-05-14 | Andrew C. Leppard | Combining hash-based duplication with sub-block differencing to deduplicate data |
US8397080B2 (en) * | 2010-07-29 | 2013-03-12 | Industrial Technology Research Institute | Scalable segment-based data de-duplication system and method for incremental backups |
-
2010
- 2010-11-30 CN CN2010105895679A patent/CN102479245B/zh not_active Expired - Fee Related
-
2011
- 2011-03-23 US US13/070,052 patent/US20120136842A1/en not_active Abandoned
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101706825A (zh) * | 2009-12-10 | 2010-05-12 | 华中科技大学 | 一种基于文件内容类型的重复数据删除方法 |
CN101814045A (zh) * | 2010-04-22 | 2010-08-25 | 华中科技大学 | 一种用于备份服务的数据组织方法 |
Non-Patent Citations (2)
Title |
---|
敖莉 等.重复数据删除技术.《软件学报》.2010,第21卷(第5期),第916-929页. |
重复数据删除技术;敖莉 等;《软件学报》;20100531;第21卷(第5期);第916-929页 * |
Also Published As
Publication number | Publication date |
---|---|
US20120136842A1 (en) | 2012-05-31 |
CN102479245A (zh) | 2012-05-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN102479245B (zh) | 数据区块的切分方法 | |
CN108319654B (zh) | 计算系统、冷热数据分离方法及装置、计算机可读存储介质 | |
US11580061B2 (en) | System and method for file archiving using machine learning | |
CN103870514A (zh) | 重复数据删除方法和装置 | |
CN110543446B (zh) | 一种基于快照的区块链直接归档方法 | |
CN102456059A (zh) | 重复数据删除的处理系统 | |
CN111095187B (zh) | 磁带驱动器存储器存储改进方法、设备和存储介质 | |
CN105446964A (zh) | 用于文件的重复数据删除的方法及装置 | |
CN104077380A (zh) | 一种重复数据删除方法、装置及系统 | |
US20200183604A1 (en) | Partitioning graph data for large scale graph processing | |
CN104050057A (zh) | 一种历史感知的数据去重碎片消除方法与系统 | |
CN114115734A (zh) | 一种数据重删方法、装置、设备及存储介质 | |
CN116795296B (zh) | 一种数据存储方法、存储设备及计算机可读存储介质 | |
CN116226681B (zh) | 一种文本相似性判定方法、装置、计算机设备和存储介质 | |
CN104484402A (zh) | 一种删除重复数据的方法及装置 | |
CN111625500B (zh) | 文件快照方法及装置、电子设备和存储介质 | |
CN109408496A (zh) | 一种减少数据冗余的方法及装置 | |
CN104298614A (zh) | 数据块在存储设备中存储方法和存储设备 | |
CN111984598A (zh) | 一种高性能元数据日志文件管理方法、系统、介质及终端 | |
JP2010191903A (ja) | 分散ファイルシステムのストライピング種別選択方法及びその分散ファイルシステム | |
CN105608089A (zh) | 一种文件存储方法及装置 | |
CN114063935B (zh) | 处理数据的方法以及装置 | |
CN114528258B (zh) | 文件异步处理方法、装置、服务器、介质、产品及系统 | |
CN116561120B (zh) | 一种用于时序数据库的数据文件快速合并方法及系统 | |
US20230385240A1 (en) | Optimizations for data deduplication operations |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
C41 | Transfer of patent application or patent right or utility model | ||
CB03 | Change of inventor or designer information |
Inventor after: Wu Zuyang Inventor before: Zhu Mingsheng Inventor before: Chen Zhifeng |
|
COR | Change of bibliographic data | ||
TR01 | Transfer of patent right |
Effective date of registration: 20161125 Address after: 844000 the Xinjiang Uygur Autonomous Region Kashi Economic Development Zone Deep Avenue headquarters economic zone far away wealth center, layer 03-02, No. 18 Patentee after: The youngest Xinjiang Network Technology Co.,Ltd. Address before: Tianhe District Tong East Road Guangzhou city Guangdong province 510665 B-101 No. 5, room B-118 Patentee before: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd. Effective date of registration: 20161125 Address after: Tianhe District Tong East Road Guangzhou city of Guangdong Province, No. 5, room B-118 B-101 Patentee after: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd. Address before: 300193 West Lake Road, Tianjin, No. 38, No. Patentee before: ITC, Inventec Tianjin Co. Patentee before: Yingda Co.,Ltd. |
|
CF01 | Termination of patent right due to non-payment of annual fee | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20130717 Termination date: 20171130 |