CN102479245A - 数据区块的切分方法 - Google Patents
数据区块的切分方法 Download PDFInfo
- Publication number
- CN102479245A CN102479245A CN2010105895679A CN201010589567A CN102479245A CN 102479245 A CN102479245 A CN 102479245A CN 2010105895679 A CN2010105895679 A CN 2010105895679A CN 201010589567 A CN201010589567 A CN 201010589567A CN 102479245 A CN102479245 A CN 102479245A
- Authority
- CN
- China
- Prior art keywords
- block
- target data
- data block
- file
- moving window
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1748—De-duplication implemented within the file system, e.g. based on file segments
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Collating Specific Patterns (AREA)
Abstract
Description
Claims (5)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2010105895679A CN102479245B (zh) | 2010-11-30 | 2010-11-30 | 数据区块的切分方法 |
US13/070,052 US20120136842A1 (en) | 2010-11-30 | 2011-03-23 | Partitioning method of data blocks |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2010105895679A CN102479245B (zh) | 2010-11-30 | 2010-11-30 | 数据区块的切分方法 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN102479245A true CN102479245A (zh) | 2012-05-30 |
CN102479245B CN102479245B (zh) | 2013-07-17 |
Family
ID=46091893
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN2010105895679A Expired - Fee Related CN102479245B (zh) | 2010-11-30 | 2010-11-30 | 数据区块的切分方法 |
Country Status (2)
Country | Link |
---|---|
US (1) | US20120136842A1 (zh) |
CN (1) | CN102479245B (zh) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103078709A (zh) * | 2013-01-05 | 2013-05-01 | 中国科学院深圳先进技术研究院 | 数据冗余识别方法 |
CN103547329A (zh) * | 2012-12-12 | 2014-01-29 | 华为技术有限公司 | 集群系统中数据处理方法及装置 |
CN105446964A (zh) * | 2014-05-30 | 2016-03-30 | 国际商业机器公司 | 用于文件的重复数据删除的方法及装置 |
Families Citing this family (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102143039B (zh) * | 2010-06-29 | 2013-11-06 | 华为技术有限公司 | 数据压缩中数据分段方法及设备 |
US9509736B2 (en) | 2013-01-16 | 2016-11-29 | Cisco Technology, Inc. | Method for optimizing WAN traffic |
US9300748B2 (en) * | 2013-01-16 | 2016-03-29 | Cisco Technology, Inc. | Method for optimizing WAN traffic with efficient indexing scheme |
US9306997B2 (en) | 2013-01-16 | 2016-04-05 | Cisco Technology, Inc. | Method for optimizing WAN traffic with deduplicated storage |
CN104348571B (zh) * | 2013-07-23 | 2018-02-06 | 华为技术有限公司 | 数据分块方法及装置 |
CN104823184B (zh) * | 2013-09-29 | 2016-11-09 | 华为技术有限公司 | 一种数据处理方法、系统及客户端 |
US10410244B2 (en) | 2013-11-13 | 2019-09-10 | Bi Science (2009) Ltd | Behavioral content discovery |
KR101912727B1 (ko) * | 2014-02-14 | 2018-10-29 | 후아웨이 테크놀러지 컴퍼니 리미티드 | 데이터 흐름 분할 포인트를 검색하기 위한 서버 기반 방법, 및 서버 |
US9760578B2 (en) * | 2014-07-23 | 2017-09-12 | International Business Machines Corporation | Lookup-based data block alignment for data deduplication |
CN112783417A (zh) * | 2019-11-01 | 2021-05-11 | 华为技术有限公司 | 数据缩减的方法、装置、计算设备和存储介质 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050091234A1 (en) * | 2003-10-23 | 2005-04-28 | International Business Machines Corporation | System and method for dividing data into predominantly fixed-sized chunks so that duplicate data chunks may be identified |
CN101706825A (zh) * | 2009-12-10 | 2010-05-12 | 华中科技大学 | 一种基于文件内容类型的重复数据删除方法 |
CN101814045A (zh) * | 2010-04-22 | 2010-08-25 | 华中科技大学 | 一种用于备份服务的数据组织方法 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8315984B2 (en) * | 2007-05-22 | 2012-11-20 | Netapp, Inc. | System and method for on-the-fly elimination of redundant data |
US8086799B2 (en) * | 2008-08-12 | 2011-12-27 | Netapp, Inc. | Scalable deduplication of stored data |
US9239843B2 (en) * | 2009-12-15 | 2016-01-19 | Symantec Corporation | Scalable de-duplication for storage systems |
US8442942B2 (en) * | 2010-03-25 | 2013-05-14 | Andrew C. Leppard | Combining hash-based duplication with sub-block differencing to deduplicate data |
US8397080B2 (en) * | 2010-07-29 | 2013-03-12 | Industrial Technology Research Institute | Scalable segment-based data de-duplication system and method for incremental backups |
-
2010
- 2010-11-30 CN CN2010105895679A patent/CN102479245B/zh not_active Expired - Fee Related
-
2011
- 2011-03-23 US US13/070,052 patent/US20120136842A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050091234A1 (en) * | 2003-10-23 | 2005-04-28 | International Business Machines Corporation | System and method for dividing data into predominantly fixed-sized chunks so that duplicate data chunks may be identified |
CN101706825A (zh) * | 2009-12-10 | 2010-05-12 | 华中科技大学 | 一种基于文件内容类型的重复数据删除方法 |
CN101814045A (zh) * | 2010-04-22 | 2010-08-25 | 华中科技大学 | 一种用于备份服务的数据组织方法 |
Non-Patent Citations (1)
Title |
---|
敖莉 等: "重复数据删除技术", 《软件学报》, vol. 21, no. 5, 31 May 2010 (2010-05-31), pages 916 - 929 * |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103547329A (zh) * | 2012-12-12 | 2014-01-29 | 华为技术有限公司 | 集群系统中数据处理方法及装置 |
WO2014089767A1 (zh) * | 2012-12-12 | 2014-06-19 | 华为技术有限公司 | 集群系统中数据处理方法及装置 |
US8892529B2 (en) | 2012-12-12 | 2014-11-18 | Huawei Technologies Co., Ltd. | Data processing method and apparatus in cluster system |
CN103547329B (zh) * | 2012-12-12 | 2016-11-02 | 华为技术有限公司 | 集群系统中数据处理方法及装置 |
CN106445413A (zh) * | 2012-12-12 | 2017-02-22 | 华为技术有限公司 | 集群系统中数据处理方法及装置 |
CN106445413B (zh) * | 2012-12-12 | 2019-10-25 | 华为技术有限公司 | 集群系统中数据处理方法及装置 |
CN103078709A (zh) * | 2013-01-05 | 2013-05-01 | 中国科学院深圳先进技术研究院 | 数据冗余识别方法 |
CN103078709B (zh) * | 2013-01-05 | 2016-04-13 | 中国科学院深圳先进技术研究院 | 数据冗余识别方法 |
CN105446964A (zh) * | 2014-05-30 | 2016-03-30 | 国际商业机器公司 | 用于文件的重复数据删除的方法及装置 |
CN105446964B (zh) * | 2014-05-30 | 2019-04-26 | 国际商业机器公司 | 用于文件的重复数据删除的方法及装置 |
US10769112B2 (en) | 2014-05-30 | 2020-09-08 | International Business Machines Corporation | Deduplication of file |
Also Published As
Publication number | Publication date |
---|---|
CN102479245B (zh) | 2013-07-17 |
US20120136842A1 (en) | 2012-05-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN102479245B (zh) | 数据区块的切分方法 | |
CN108319654B (zh) | 计算系统、冷热数据分离方法及装置、计算机可读存储介质 | |
CN108460045B (zh) | 一种快照的处理方法及分布式块存储系统 | |
US11580061B2 (en) | System and method for file archiving using machine learning | |
CN107729558B (zh) | 文件系统碎片整理的方法、系统、装置及计算机存储介质 | |
CN104281533B (zh) | 一种存储数据的方法及装置 | |
US8495022B1 (en) | Systems and methods for synthetic backups | |
CN104932841A (zh) | 一种云存储系统中节约型重复数据删除方法 | |
CN110543446B (zh) | 一种基于快照的区块链直接归档方法 | |
Zou et al. | The dilemma between deduplication and locality: Can both be achieved? | |
CN103870514A (zh) | 重复数据删除方法和装置 | |
CN104077380A (zh) | 一种重复数据删除方法、装置及系统 | |
CN105446964A (zh) | 用于文件的重复数据删除的方法及装置 | |
CN103744875A (zh) | 基于文件系统的数据快速迁移方法及系统 | |
CN104050057A (zh) | 一种历史感知的数据去重碎片消除方法与系统 | |
US20180011897A1 (en) | Data processing method having structure of cache index specified to transaction in mobile environment dbms | |
CN116795296B (zh) | 一种数据存储方法、存储设备及计算机可读存储介质 | |
CN116226681B (zh) | 一种文本相似性判定方法、装置、计算机设备和存储介质 | |
CN111984598A (zh) | 一种高性能元数据日志文件管理方法、系统、介质及终端 | |
CN102467557A (zh) | 重复数据删除的处理方法 | |
CN111625500B (zh) | 文件快照方法及装置、电子设备和存储介质 | |
CN107491363A (zh) | 一种基于Linux内核的存储卷的快照方法及装置 | |
CN115328696A (zh) | 一种数据库中的数据备份方法 | |
CN104298614A (zh) | 数据块在存储设备中存储方法和存储设备 | |
US20170315751A1 (en) | Columnar data storage on tape partition |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
C41 | Transfer of patent application or patent right or utility model | ||
CB03 | Change of inventor or designer information |
Inventor after: Wu Zuyang Inventor before: Zhu Mingsheng Inventor before: Chen Zhifeng |
|
COR | Change of bibliographic data | ||
TR01 | Transfer of patent right |
Effective date of registration: 20161125 Address after: 844000 the Xinjiang Uygur Autonomous Region Kashi Economic Development Zone Deep Avenue headquarters economic zone far away wealth center, layer 03-02, No. 18 Patentee after: The youngest Xinjiang Network Technology Co.,Ltd. Address before: Tianhe District Tong East Road Guangzhou city Guangdong province 510665 B-101 No. 5, room B-118 Patentee before: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd. Effective date of registration: 20161125 Address after: Tianhe District Tong East Road Guangzhou city of Guangdong Province, No. 5, room B-118 B-101 Patentee after: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd. Address before: 300193 West Lake Road, Tianjin, No. 38, No. Patentee before: ITC, Inventec Tianjin Co. Patentee before: Yingda Co.,Ltd. |
|
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20130717 Termination date: 20171130 |
|
CF01 | Termination of patent right due to non-payment of annual fee |