CA3095033C - Techniques for scheduled anti-entropy repair design - Google Patents

Techniques for scheduled anti-entropy repair design Download PDF

Info

Publication number
CA3095033C
CA3095033C CA3095033A CA3095033A CA3095033C CA 3095033 C CA3095033 C CA 3095033C CA 3095033 A CA3095033 A CA 3095033A CA 3095033 A CA3095033 A CA 3095033A CA 3095033 C CA3095033 C CA 3095033C
Authority
CA
Canada
Prior art keywords
repair
node
nodes
entropy
repair procedure
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CA3095033A
Other languages
English (en)
French (fr)
Other versions
CA3095033A1 (en
Inventor
Vinay Chella
Joseph Lynch
Ajay Upadhyay
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Netflix Inc
Original Assignee
Netflix Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Netflix Inc filed Critical Netflix Inc
Publication of CA3095033A1 publication Critical patent/CA3095033A1/en
Application granted granted Critical
Publication of CA3095033C publication Critical patent/CA3095033C/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
    • G06F11/0709—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0751—Error or fault detection not based on redundancy
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0793—Remedial or corrective actions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/273—Asynchronous replication or reconciliation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Hardware Design (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
CA3095033A 2018-03-27 2019-03-27 Techniques for scheduled anti-entropy repair design Active CA3095033C (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201862648907P 2018-03-27 2018-03-27
US62/648,907 2018-03-27
PCT/US2019/024417 WO2019191320A1 (en) 2018-03-27 2019-03-27 Techniques for scheduled anti-entropy repair design

Publications (2)

Publication Number Publication Date
CA3095033A1 CA3095033A1 (en) 2019-10-03
CA3095033C true CA3095033C (en) 2023-03-28

Family

ID=66103054

Family Applications (1)

Application Number Title Priority Date Filing Date
CA3095033A Active CA3095033C (en) 2018-03-27 2019-03-27 Techniques for scheduled anti-entropy repair design

Country Status (11)

Country Link
US (2) US11119845B2 (enExample)
EP (1) EP3776198A1 (enExample)
JP (1) JP7191493B2 (enExample)
KR (1) KR102509256B1 (enExample)
CN (1) CN112219189B (enExample)
AU (1) AU2019244116B2 (enExample)
BR (1) BR112020019697A2 (enExample)
CA (1) CA3095033C (enExample)
MX (1) MX2020010009A (enExample)
SG (1) SG11202009391UA (enExample)
WO (1) WO2019191320A1 (enExample)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12481703B2 (en) * 2020-03-30 2025-11-25 International Business Machines Corporation Shard hashing for database objects within a distributed database
DE102023102529B4 (de) 2023-02-02 2024-08-29 Deutsches Zentrum für Luft- und Raumfahrt e.V. Verfahren zur Übertragung eines Prüfvektors von einer Sendeeinheit an eine Empfangseinheit

Family Cites Families (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6098078A (en) * 1995-12-29 2000-08-01 Lucent Technologies Inc. Maintaining consistency of database replicas
GB9812221D0 (en) 1998-06-05 1998-08-05 British Telecomm Distributed database system
US6941287B1 (en) * 1999-04-30 2005-09-06 E. I. Du Pont De Nemours And Company Distributed hierarchical evolutionary modeling and visualization of empirical data
US7216251B2 (en) * 2000-02-19 2007-05-08 Powerquest Corporation Computer imaging recovery without a working partition or a secondary medium
JP2005352708A (ja) * 2004-06-10 2005-12-22 Hitachi Ltd オンライン同期スケジュール回復システムおよびその処理方法
US7685109B1 (en) 2005-12-29 2010-03-23 Amazon Technologies, Inc. Method and apparatus for data partitioning and replication in a searchable data service
US7783813B2 (en) * 2007-06-14 2010-08-24 International Business Machines Corporation Multi-node configuration of processor cards connected via processor fabrics
JP5452015B2 (ja) * 2008-12-15 2014-03-26 日本証券テクノロジー株式会社 ファイル整合性確認システム、ファイル整合性確認方法、プログラム
US7984161B2 (en) * 2009-06-16 2011-07-19 Microsoft Corporation Optimizing mobile device synchronization with multiple data sources
JP4913913B2 (ja) * 2010-04-28 2012-04-11 新日鉄ソリューションズ株式会社 情報処理システム、情報処理方法及びプログラム
US8234372B2 (en) * 2010-05-05 2012-07-31 Go Daddy Operating Company, LLC Writing a file to a cloud storage solution
CN102143194A (zh) * 2010-06-10 2011-08-03 华为技术有限公司 数据同步的方法、系统、中间数据节点及终止数据节点
JP5751029B2 (ja) * 2011-06-03 2015-07-22 株式会社リコー 認証装置、プログラムおよび記録媒体
US20140236902A1 (en) * 2011-09-29 2014-08-21 Beijing Qihoo Technology Company Limited User Data Backup Method and Apparatus
US9836514B2 (en) * 2011-11-07 2017-12-05 Empire Technology Development Llc Cache based key-value store mapping and replication
US9112809B2 (en) * 2012-11-21 2015-08-18 Telefonaktiebolaget L M Ericsson (Publ) Method and apparatus for controlling utilization in a horizontally scaled software application
US9002805B1 (en) * 2012-12-14 2015-04-07 Amazon Technologies, Inc. Conditional storage object deletion
KR101497669B1 (ko) * 2013-04-29 2015-03-11 서울대학교산학협력단 일정 수 이상의 파일 조각으로 원본 파일을 복구하는 파일 관리 장치 및 방법
US9304815B1 (en) * 2013-06-13 2016-04-05 Amazon Technologies, Inc. Dynamic replica failure detection and healing
EP2953025B1 (en) * 2013-10-18 2017-01-25 Hitachi Ltd. Target-driven independent data integrity and redundancy recovery in a shared-nothing distributed storage system
US9264494B2 (en) * 2013-10-21 2016-02-16 International Business Machines Corporation Automated data recovery from remote data object replicas
US9379951B2 (en) * 2014-01-10 2016-06-28 Instep Software, Llc Method and apparatus for detection of anomalies in integrated parameter systems
US9519532B2 (en) * 2014-01-20 2016-12-13 Lenovo Enterprise Solutions (Singapore) Pte. Ltd. Handling system interrupts with long-running recovery actions
US9600203B2 (en) 2014-03-11 2017-03-21 Amazon Technologies, Inc. Reducing data volume durability state for block-based storage
CN105446653B (zh) 2014-08-27 2018-12-14 阿里巴巴集团控股有限公司 一种数据合并方法和设备
US9515878B2 (en) * 2014-11-13 2016-12-06 Software AG USA Inc. Method, medium, and system for configuring a new node in a distributed memory network
US10355935B2 (en) * 2014-12-09 2019-07-16 Ciena Corporation Reduced link bandwidth update systems and methods for improved scalability, efficiency, and performance
US10528530B2 (en) * 2015-04-08 2020-01-07 Microsoft Technology Licensing, Llc File repair of file stored across multiple data stores
US10884869B2 (en) 2015-04-16 2021-01-05 Nuodb, Inc. Backup and restore in a distributed database utilizing consistent database snapshots
US9940169B2 (en) * 2015-07-23 2018-04-10 Pearson Education, Inc. Real-time partitioned processing streaming
KR20170022001A (ko) * 2015-08-19 2017-03-02 엘지전자 주식회사 데이터 송수신 방법 및 이를 위한 디바이스
JP6515799B2 (ja) * 2015-12-18 2019-05-22 京セラドキュメントソリューションズ株式会社 電子機器及びメモリー寿命警告プログラム
US10360095B2 (en) * 2016-03-31 2019-07-23 Change Healthcare Holdings, Llc Methods and apparatuses for improving failure recovery in a distributed system
JP6720744B2 (ja) * 2016-07-15 2020-07-08 富士通株式会社 情報処理システム、情報処理装置、及び制御プログラム
US10990467B2 (en) * 2016-12-15 2021-04-27 Nutanix, Inc. Accessing computing resource attributes of an external service provider
US20180225324A1 (en) * 2017-02-06 2018-08-09 Qualcomm Incorporated Providing Retry Schedules For File Repair Over Broadcast Networks

Also Published As

Publication number Publication date
US11119845B2 (en) 2021-09-14
CA3095033A1 (en) 2019-10-03
WO2019191320A1 (en) 2019-10-03
SG11202009391UA (en) 2020-10-29
JP7191493B2 (ja) 2022-12-19
JP2021532428A (ja) 2021-11-25
KR20200131899A (ko) 2020-11-24
KR102509256B1 (ko) 2023-03-14
AU2019244116B2 (en) 2021-10-07
MX2020010009A (es) 2021-01-15
US20190303234A1 (en) 2019-10-03
EP3776198A1 (en) 2021-02-17
US11636005B2 (en) 2023-04-25
US20210406116A1 (en) 2021-12-30
CN112219189B (zh) 2024-07-23
CN112219189A (zh) 2021-01-12
BR112020019697A2 (pt) 2021-01-05
AU2019244116A1 (en) 2020-10-15

Similar Documents

Publication Publication Date Title
US10169163B2 (en) Managing backup operations from a client system to a primary server and secondary server
US9805070B2 (en) Dynamic migration script management
US20180210754A1 (en) Adaptive self-maintenance scheduler
AU2020408143B2 (en) Watermark-based techniques for change-data-capture
US11675740B2 (en) Tiered cache filling
US11636005B2 (en) Techniques for scheduled anti-entropy repair design
CN112579550B (zh) 一种分布式文件系统的元数据信息同步方法及系统
CN106339176B (zh) 中间文件处理方法、客户端、服务器和系统
US10749921B2 (en) Techniques for warming up a node in a distributed data store
CN113742254B (zh) 内存碎片治理方法、装置和系统
CN111630500B (zh) 信息分布式存储系统、方法及程序
US20150088826A1 (en) Enhanced Performance for Data Duplication
US9933966B2 (en) Storage control device and computer system
US12487855B2 (en) Replacing stale clusters in a cluster pool
CN121858537A (zh) 一种分布式数据库的元数据管理方法、设备及介质
US20130246614A1 (en) Method of updating agent service
KR20160118402A (ko) Vdi 대용량 데이터 처리를 위한 액세스 패턴 기반 캐시 관리 방법

Legal Events

Date Code Title Description
EEER Examination request

Effective date: 20200923

MPN Maintenance fee for patent paid

Free format text: FEE DESCRIPTION TEXT: MF (PATENT, 6TH ANNIV.) - STANDARD

Year of fee payment: 6

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20241210

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20241210

MPN Maintenance fee for patent paid

Free format text: FEE DESCRIPTION TEXT: MF (PATENT, 7TH ANNIV.) - STANDARD

Year of fee payment: 7

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20260317

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20260317