CN103902408A - 使用布隆过滤器检测副本之间的偏差 - Google Patents
使用布隆过滤器检测副本之间的偏差 Download PDFInfo
- Publication number
- CN103902408A CN103902408A CN201310741125.5A CN201310741125A CN103902408A CN 103902408 A CN103902408 A CN 103902408A CN 201310741125 A CN201310741125 A CN 201310741125A CN 103902408 A CN103902408 A CN 103902408A
- Authority
- CN
- China
- Prior art keywords
- fragment
- bloom filter
- specific
- access code
- deviation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 claims abstract description 46
- 239000012634 fragment Substances 0.000 claims description 279
- 230000004044 response Effects 0.000 claims description 10
- 230000009471 action Effects 0.000 claims description 7
- 230000000246 remedial effect Effects 0.000 abstract description 2
- 238000003860 storage Methods 0.000 description 28
- 238000005516 engineering process Methods 0.000 description 24
- 230000008569 process Effects 0.000 description 16
- 238000004891 communication Methods 0.000 description 15
- 238000012795 verification Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 12
- 230000005540 biological transmission Effects 0.000 description 9
- 230000005055 memory storage Effects 0.000 description 8
- 230000011218 segmentation Effects 0.000 description 7
- 230000008859 change Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 238000005192 partition Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000005520 cutting process Methods 0.000 description 2
- 230000005611 electricity Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 210000000352 storage cell Anatomy 0.000 description 2
- GZPBVLUEICLBOA-UHFFFAOYSA-N 4-(dimethylamino)-3,5-dimethylphenol Chemical compound CN(C)C1=C(C)C=C(O)C=C1C GZPBVLUEICLBOA-UHFFFAOYSA-N 0.000 description 1
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000013011 mating Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/1658—Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
- G06F11/1662—Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit the resynchronized component or unit being a persistent storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/2053—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
- G06F11/2094—Redundant storage or storage space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/273—Asynchronous replication or reconciliation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/15—Correlation function computation including computation of convolution operations
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Physics (AREA)
- Algebra (AREA)
- Software Systems (AREA)
- Error Detection And Correction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims (23)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/728,133 US9075834B2 (en) | 2012-01-26 | 2012-12-27 | Detecting deviation between replicas using bloom filters |
US13/728,133 | 2012-12-27 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN103902408A true CN103902408A (zh) | 2014-07-02 |
CN103902408B CN103902408B (zh) | 2018-01-26 |
Family
ID=50993745
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201310741125.5A Expired - Fee Related CN103902408B (zh) | 2012-12-27 | 2013-12-27 | 用于检测复制内容区块之间的差异的方法和系统 |
Country Status (3)
Country | Link |
---|---|
US (1) | US9075834B2 (zh) |
CN (1) | CN103902408B (zh) |
HK (1) | HK1199507A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107516046A (zh) * | 2017-06-26 | 2017-12-26 | 江苏通付盾科技有限公司 | 数据保护方法及装置、电子设备、计算机存储介质 |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9052824B2 (en) | 2012-01-26 | 2015-06-09 | Upthere, Inc. | Content addressable stores based on sibling groups |
US9652511B2 (en) * | 2013-03-13 | 2017-05-16 | International Business Machines Corporation | Secure matching supporting fuzzy data |
CN104408163B (zh) * | 2014-12-05 | 2018-09-14 | 浪潮(北京)电子信息产业有限公司 | 一种数据分级存储方法和装置 |
US10019452B2 (en) | 2015-05-19 | 2018-07-10 | Morgan Stanley | Topology aware distributed storage system |
US10019456B2 (en) | 2016-06-29 | 2018-07-10 | Microsoft Technology Licensing, Llc | Recovering free space in nonvolatile storage with a computer storage system supporting shared objects |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080229103A1 (en) * | 2007-03-13 | 2008-09-18 | Board Of Trustees Of Michigan State University | Private entity authentication for pervasive computing environments |
CN101799783A (zh) * | 2009-01-19 | 2010-08-11 | 中国人民大学 | 一种数据存储处理方法、查找方法及其装置 |
CN101853260A (zh) * | 2009-04-01 | 2010-10-06 | 赛门铁克公司 | 检测电子邮件内容包含的系统和方法 |
CN102110171A (zh) * | 2011-03-22 | 2011-06-29 | 湖南大学 | 基于树形结构的布鲁姆过滤器的查询与更新方法 |
CN102799617A (zh) * | 2012-06-19 | 2012-11-28 | 华中科技大学 | 多层Bloom Filter的构建及查询优化方法 |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7535485B2 (en) | 2000-08-15 | 2009-05-19 | Polycom, Inc. | Delay reduction for transmission and processing of video data |
US8156429B2 (en) | 2004-10-22 | 2012-04-10 | Flash Networks. Ltd | Method and system for accelerating downloading of web pages |
JP5431148B2 (ja) | 2006-05-31 | 2014-03-05 | インターナショナル・ビジネス・マシーンズ・コーポレーション | ストレージ用論理データオブジェクトの変換方法およびシステム |
CN103546372A (zh) | 2006-11-27 | 2014-01-29 | 艾利森电话股份有限公司 | 用于提供覆盖网络的路由选择体系结构的方法和系统 |
GB2446199A (en) | 2006-12-01 | 2008-08-06 | David Irvine | Secure, decentralised and anonymous peer-to-peer network |
GB2446169A (en) | 2006-12-01 | 2008-08-06 | David Irvine | Granular accessibility to data in a distributed and/or corporate network |
DE602007010040D1 (de) | 2007-02-22 | 2010-12-02 | Ericsson Telefon Ab L M | Overlay-Netzwerk mit einheitlicher und fehlertoleranter verteilter Hash-Tabelle (DHT) |
US7788220B1 (en) | 2007-12-31 | 2010-08-31 | Emc Corporation | Storage of data with composite hashes in backup systems |
US8335889B2 (en) | 2008-09-11 | 2012-12-18 | Nec Laboratories America, Inc. | Content addressable storage systems and methods employing searchable blocks |
US9401967B2 (en) | 2010-06-09 | 2016-07-26 | Brocade Communications Systems, Inc. | Inline wire speed deduplication system |
US8631209B2 (en) | 2012-01-26 | 2014-01-14 | Upthere, Inc. | Reusable content addressable stores as building blocks for creating large scale storage infrastructures |
US9052824B2 (en) | 2012-01-26 | 2015-06-09 | Upthere, Inc. | Content addressable stores based on sibling groups |
-
2012
- 2012-12-27 US US13/728,133 patent/US9075834B2/en not_active Expired - Fee Related
-
2013
- 2013-12-27 CN CN201310741125.5A patent/CN103902408B/zh not_active Expired - Fee Related
-
2014
- 2014-12-25 HK HK14112938.4A patent/HK1199507A1/zh unknown
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080229103A1 (en) * | 2007-03-13 | 2008-09-18 | Board Of Trustees Of Michigan State University | Private entity authentication for pervasive computing environments |
CN101799783A (zh) * | 2009-01-19 | 2010-08-11 | 中国人民大学 | 一种数据存储处理方法、查找方法及其装置 |
CN101853260A (zh) * | 2009-04-01 | 2010-10-06 | 赛门铁克公司 | 检测电子邮件内容包含的系统和方法 |
CN102110171A (zh) * | 2011-03-22 | 2011-06-29 | 湖南大学 | 基于树形结构的布鲁姆过滤器的查询与更新方法 |
CN102799617A (zh) * | 2012-06-19 | 2012-11-28 | 华中科技大学 | 多层Bloom Filter的构建及查询优化方法 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107516046A (zh) * | 2017-06-26 | 2017-12-26 | 江苏通付盾科技有限公司 | 数据保护方法及装置、电子设备、计算机存储介质 |
CN107516046B (zh) * | 2017-06-26 | 2019-11-12 | 江苏通付盾科技有限公司 | 数据保护方法及装置、电子设备、计算机存储介质 |
Also Published As
Publication number | Publication date |
---|---|
CN103902408B (zh) | 2018-01-26 |
HK1199507A1 (zh) | 2015-07-03 |
US9075834B2 (en) | 2015-07-07 |
US20140188816A1 (en) | 2014-07-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN103902408A (zh) | 使用布隆过滤器检测副本之间的偏差 | |
US10073652B2 (en) | Performance optimized storage vaults in a dispersed storage network | |
US10719250B2 (en) | System and method for combining erasure-coded protection sets | |
US9430648B2 (en) | Method and apparatus for near field communication | |
EP4018346A1 (en) | Data breach detection | |
US8407186B1 (en) | Systems and methods for data-selection-specific data deduplication | |
US20160274967A1 (en) | Data recovery agent and search service for repairing bit rot | |
CN106445643B (zh) | 克隆、升级虚拟机的方法及设备 | |
CN105205082A (zh) | 用于处理hdfs中的文件存储的方法和系统 | |
JP2014521176A (ja) | インデックス構築方法、検索方法、検索デバイス、および検索システム | |
TWI705368B (zh) | 區塊鏈資料的壓縮處理方法和裝置、電腦設備以及電腦可讀儲存媒體 | |
CN105554081A (zh) | 一种文件差量的传输方法以及装置 | |
US20140337296A1 (en) | Techniques to recover files in a storage network | |
CN104079623A (zh) | 多级云存储同步控制方法及系统 | |
US10768840B2 (en) | Updating protection sets in a geographically distributed storage environment | |
CN111143113B (zh) | 复制元数据的方法、电子设备和计算机程序产品 | |
US9684668B1 (en) | Systems and methods for performing lookups on distributed deduplicated data systems | |
CN107766354A (zh) | 一种用于保证数据正确性的方法与设备 | |
US11200205B2 (en) | Displaying an alert and options when deleting a file that is associated with a sequence of files | |
JP6788002B2 (ja) | モバイル端末のためのデータ記憶方法及び装置 | |
US11586499B2 (en) | Method, device and computer program product for storing data | |
CN112084006B (zh) | 一种镜像包处理方法、装置及电子设备 | |
CN105376285A (zh) | 网络储存去重方法及服务器 | |
CN114172894A (zh) | 数据传输方法、装置、服务器和计算机设备 | |
CN112612865A (zh) | 一种基于Elasticsearch的文档存储方法和装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1199507 Country of ref document: HK |
|
TA01 | Transfer of patent application right |
Effective date of registration: 20171120 Address after: California, USA Applicant after: Western Digital Technologies, Inc. Address before: California, USA Applicant before: Upsell GmbH |
|
TA01 | Transfer of patent application right | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
REG | Reference to a national code |
Ref country code: HK Ref legal event code: WD Ref document number: 1199507 Country of ref document: HK |
|
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180126 |
|
CF01 | Termination of patent right due to non-payment of annual fee |