CN104145263A - 一种数据压缩方法及装置 - Google Patents

一种数据压缩方法及装置 Download PDF

Info

Publication number
CN104145263A
CN104145263A CN201280002718.3A CN201280002718A CN104145263A CN 104145263 A CN104145263 A CN 104145263A CN 201280002718 A CN201280002718 A CN 201280002718A CN 104145263 A CN104145263 A CN 104145263A
Authority
CN
China
Prior art keywords
burst
index
burst index
data
indexed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201280002718.3A
Other languages
English (en)
Other versions
CN104145263B (zh
Inventor
左少夫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Gaohang Intellectual Property Operation Co ltd
Hebei Yingda Industrial And Mining Machinery Parts Co ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of CN104145263A publication Critical patent/CN104145263A/zh
Application granted granted Critical
Publication of CN104145263B publication Critical patent/CN104145263B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2272—Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

本发明实施例提供了一种数据压缩方法及装置，该方法包括：依次计算数据分片序列中数据分片的分片索引，形成分片索引序列，为所述分片索引扩充后向索引描述符；判断已有的分片索引库中是否存在所述分片索引；若不存在，则根据所述后向索引描述符将存在数据相关性的分片索引串联形成分片索引参考序列；若存在，则进一步判断所述分片索引序列中是否存在所述分片索引的参考索引；若存在所述参考索引，则根据所述分片索引相对于所述参考索引的位移量，采用相对索引表示所述分片索引，否则不改变所述分片索引的表示方式。采用本发明，可提升数据压缩的效果和速率，降低分片索引的管理成本及存储成本。

Description

PCT国内申请，说明书已公开。

Claims (1)

  1. PCT国内申请，权利要求书已公开。
CN201280002718.3A 2012-12-11 2012-12-11 一种数据压缩方法及装置 Active CN104145263B (zh)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2012/086377 WO2014089760A1 (zh) 2012-12-11 2012-12-11 一种数据压缩方法及装置

Publications (2)

Publication Number Publication Date
CN104145263A true CN104145263A (zh) 2014-11-12
CN104145263B CN104145263B (zh) 2017-07-25

Family

ID=50933683

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201280002718.3A Active CN104145263B (zh) 2012-12-11 2012-12-11 一种数据压缩方法及装置

Country Status (2)

Country Link
CN (1) CN104145263B (zh)
WO (1) WO2014089760A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090254609A1 (en) * 2008-04-08 2009-10-08 Wideman Roderick B Methods and systems for improved throughput performance in a distributed data de-duplication environment
CN102317923A (zh) * 2009-02-25 2012-01-11 日本电气株式会社 存储系统
CN102467523A (zh) * 2010-11-03 2012-05-23 英业达股份有限公司 索引文件的建立方法与利用索引文件查询数据区块的方法
CN102609442A (zh) * 2010-12-28 2012-07-25 微软公司 用于数据去重复的自适应索引

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8799238B2 (en) * 2010-06-18 2014-08-05 Hewlett-Packard Development Company, L.P. Data deduplication

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090254609A1 (en) * 2008-04-08 2009-10-08 Wideman Roderick B Methods and systems for improved throughput performance in a distributed data de-duplication environment
CN102317923A (zh) * 2009-02-25 2012-01-11 日本电气株式会社 存储系统
CN102467523A (zh) * 2010-11-03 2012-05-23 英业达股份有限公司 索引文件的建立方法与利用索引文件查询数据区块的方法
CN102609442A (zh) * 2010-12-28 2012-07-25 微软公司 用于数据去重复的自适应索引

Also Published As

Publication number Publication date
CN104145263B (zh) 2017-07-25
WO2014089760A1 (zh) 2014-06-19

Similar Documents

Publication Publication Date Title
US9645736B2 (en) Processing time series data from multiple sensors
Lu et al. Frequency based chunking for data de-duplication
Pal et al. Detecting file fragmentation point using sequential hypothesis testing
US8396840B1 (en) System and method for targeted consistency improvement in a distributed storage system
CN103098035A (zh) 存储系统
US8468134B1 (en) System and method for measuring consistency within a distributed storage system
CN106201774B (zh) 一种nand flash存储芯片数据存储结构分析方法
CN102819592B (zh) 一种基于Lucene的桌面搜索系统及方法
CN107958079A (zh) 聚合文件删除方法、系统、装置及可读存储介质
CN103605704B (zh) 大量url数据任意字段索引及检索方法
WO2014067063A1 (zh) 重复数据检索方法及设备
CN103581331A (zh) 虚拟机在线迁移方法与系统
CN103631769A (zh) 一种判断文件内容与标题间一致性的方法及装置
CN106648991A (zh) 数据容灾系统中的重复数据删除方法
CN103780263B (zh) 数据压缩装置、数据压缩方法及记录介质
CN107577549A (zh) 一种存储重删功能的测试方法
CN109344163B (zh) 一种数据校验方法、装置和计算机可读介质
CN104778252A (zh) 索引的存储方法和装置
CN107590233B (zh) 一种文件管理方法及装置
CN104012055A (zh) 一种数据处理方法及装置
CN113687773A (zh) 数据压缩模型训练方法及装置、存储介质
CN102622302A (zh) 碎片数据类型的识别方法
CN104145263A (zh) 一种数据压缩方法及装置
CN107783904B (zh) 单元测试桩去重方法、装置、计算机可读存储介质及设备
CN110021349B (zh) 基因数据的编码方法

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20191212

Address after: 056000 Longcao road crossing, Yonghe Road, Yongnian District, Handan City, Hebei Province

Patentee after: HEBEI YINGDA INDUSTRIAL AND MINING MACHINERY PARTS CO.,LTD.

Address before: 510000 unit 2414-2416, building, No. five, No. 371, Tianhe District, Guangdong, China

Patentee before: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd.

Effective date of registration: 20191212

Address after: 510000 unit 2414-2416, building, No. five, No. 371, Tianhe District, Guangdong, China

Patentee after: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd.

Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen

Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd.

TR01 Transfer of patent right