AU2019244116B2 - Techniques for scheduled anti-entropy repair design - Google Patents

Techniques for scheduled anti-entropy repair design Download PDF

Info

Publication number
AU2019244116B2
AU2019244116B2 AU2019244116A AU2019244116A AU2019244116B2 AU 2019244116 B2 AU2019244116 B2 AU 2019244116B2 AU 2019244116 A AU2019244116 A AU 2019244116A AU 2019244116 A AU2019244116 A AU 2019244116A AU 2019244116 B2 AU2019244116 B2 AU 2019244116B2
Authority
AU
Australia
Prior art keywords
repair
node
nodes
entropy
repair procedure
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
AU2019244116A
Other languages
English (en)
Other versions
AU2019244116A1 (en
Inventor
Vinay Chella
Joseph Lynch
Ajay Upadhyay
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Netflix Inc
Original Assignee
Netflix Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Netflix Inc filed Critical Netflix Inc
Publication of AU2019244116A1 publication Critical patent/AU2019244116A1/en
Application granted granted Critical
Publication of AU2019244116B2 publication Critical patent/AU2019244116B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
    • G06F11/0709—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0751—Error or fault detection not based on redundancy
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0793—Remedial or corrective actions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/273—Asynchronous replication or reconciliation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Hardware Design (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
AU2019244116A 2018-03-27 2019-03-27 Techniques for scheduled anti-entropy repair design Active AU2019244116B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201862648907P 2018-03-27 2018-03-27
US62/648,907 2018-03-27
PCT/US2019/024417 WO2019191320A1 (en) 2018-03-27 2019-03-27 Techniques for scheduled anti-entropy repair design

Publications (2)

Publication Number Publication Date
AU2019244116A1 AU2019244116A1 (en) 2020-10-15
AU2019244116B2 true AU2019244116B2 (en) 2021-10-07

Family

ID=66103054

Family Applications (1)

Application Number Title Priority Date Filing Date
AU2019244116A Active AU2019244116B2 (en) 2018-03-27 2019-03-27 Techniques for scheduled anti-entropy repair design

Country Status (11)

Country Link
US (2) US11119845B2 (enExample)
EP (1) EP3776198A1 (enExample)
JP (1) JP7191493B2 (enExample)
KR (1) KR102509256B1 (enExample)
CN (1) CN112219189B (enExample)
AU (1) AU2019244116B2 (enExample)
BR (1) BR112020019697A2 (enExample)
CA (1) CA3095033C (enExample)
MX (1) MX2020010009A (enExample)
SG (1) SG11202009391UA (enExample)
WO (1) WO2019191320A1 (enExample)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12481703B2 (en) * 2020-03-30 2025-11-25 International Business Machines Corporation Shard hashing for database objects within a distributed database
DE102023102529B4 (de) 2023-02-02 2024-08-29 Deutsches Zentrum für Luft- und Raumfahrt e.V. Verfahren zur Übertragung eines Prüfvektors von einer Sendeeinheit an eine Empfangseinheit

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6098078A (en) * 1995-12-29 2000-08-01 Lucent Technologies Inc. Maintaining consistency of database replicas
US20030221095A1 (en) * 2000-02-19 2003-11-27 Powerquest Corporation Computer imaging recovery without a working partition or a secondary medium
US6941287B1 (en) * 1999-04-30 2005-09-06 E. I. Du Pont De Nemours And Company Distributed hierarchical evolutionary modeling and visualization of empirical data
US20100268986A1 (en) * 2007-06-14 2010-10-21 International Business Machines Corporation Multi-node configuration of processor cards connected via processor fabrics
US20130120449A1 (en) * 2010-04-28 2013-05-16 Noboru IHARA Information processing system, information processing method and program
US20150113324A1 (en) * 2013-10-21 2015-04-23 International Business Machines Corporation Automated Data Recovery from Remote Data Object Replicas
US20150199224A1 (en) * 2014-01-10 2015-07-16 Instep Software, Llc Method and Apparatus for Detection of Anomalies in Integrated Parameter Systems
US20150205661A1 (en) * 2014-01-20 2015-07-23 Lenovo Enterprise Solutions (Singapore) Pte. Ltd. Handling system interrupts with long-running recovery actions
US20160164739A1 (en) * 2014-12-09 2016-06-09 Ciena Corporation Reduced link bandwidth update systems and methods for improved scalability, efficiency, and performance
US20160292249A1 (en) * 2013-06-13 2016-10-06 Amazon Technologies, Inc. Dynamic replica failure detection and healing
US20170026441A1 (en) * 2015-07-23 2017-01-26 Christopher Moudy Real-Time Partitioned Processing Streaming
US20170180599A1 (en) * 2015-12-18 2017-06-22 Kyocera Document Solutions Inc. Electronic device and non-transitory computer-readable storage medium

Family Cites Families (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB9812221D0 (en) 1998-06-05 1998-08-05 British Telecomm Distributed database system
JP2005352708A (ja) * 2004-06-10 2005-12-22 Hitachi Ltd オンライン同期スケジュール回復システムおよびその処理方法
US7685109B1 (en) 2005-12-29 2010-03-23 Amazon Technologies, Inc. Method and apparatus for data partitioning and replication in a searchable data service
JP5452015B2 (ja) * 2008-12-15 2014-03-26 日本証券テクノロジー株式会社 ファイル整合性確認システム、ファイル整合性確認方法、プログラム
US7984161B2 (en) * 2009-06-16 2011-07-19 Microsoft Corporation Optimizing mobile device synchronization with multiple data sources
US8234372B2 (en) * 2010-05-05 2012-07-31 Go Daddy Operating Company, LLC Writing a file to a cloud storage solution
CN102143194A (zh) * 2010-06-10 2011-08-03 华为技术有限公司 数据同步的方法、系统、中间数据节点及终止数据节点
JP5751029B2 (ja) * 2011-06-03 2015-07-22 株式会社リコー 認証装置、プログラムおよび記録媒体
US20140236902A1 (en) * 2011-09-29 2014-08-21 Beijing Qihoo Technology Company Limited User Data Backup Method and Apparatus
US9836514B2 (en) * 2011-11-07 2017-12-05 Empire Technology Development Llc Cache based key-value store mapping and replication
US9112809B2 (en) * 2012-11-21 2015-08-18 Telefonaktiebolaget L M Ericsson (Publ) Method and apparatus for controlling utilization in a horizontally scaled software application
US9002805B1 (en) * 2012-12-14 2015-04-07 Amazon Technologies, Inc. Conditional storage object deletion
KR101497669B1 (ko) * 2013-04-29 2015-03-11 서울대학교산학협력단 일정 수 이상의 파일 조각으로 원본 파일을 복구하는 파일 관리 장치 및 방법
EP2953025B1 (en) * 2013-10-18 2017-01-25 Hitachi Ltd. Target-driven independent data integrity and redundancy recovery in a shared-nothing distributed storage system
US9600203B2 (en) 2014-03-11 2017-03-21 Amazon Technologies, Inc. Reducing data volume durability state for block-based storage
CN105446653B (zh) 2014-08-27 2018-12-14 阿里巴巴集团控股有限公司 一种数据合并方法和设备
US9515878B2 (en) * 2014-11-13 2016-12-06 Software AG USA Inc. Method, medium, and system for configuring a new node in a distributed memory network
US10528530B2 (en) * 2015-04-08 2020-01-07 Microsoft Technology Licensing, Llc File repair of file stored across multiple data stores
US10884869B2 (en) 2015-04-16 2021-01-05 Nuodb, Inc. Backup and restore in a distributed database utilizing consistent database snapshots
KR20170022001A (ko) * 2015-08-19 2017-03-02 엘지전자 주식회사 데이터 송수신 방법 및 이를 위한 디바이스
US10360095B2 (en) * 2016-03-31 2019-07-23 Change Healthcare Holdings, Llc Methods and apparatuses for improving failure recovery in a distributed system
JP6720744B2 (ja) * 2016-07-15 2020-07-08 富士通株式会社 情報処理システム、情報処理装置、及び制御プログラム
US10990467B2 (en) * 2016-12-15 2021-04-27 Nutanix, Inc. Accessing computing resource attributes of an external service provider
US20180225324A1 (en) * 2017-02-06 2018-08-09 Qualcomm Incorporated Providing Retry Schedules For File Repair Over Broadcast Networks

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6098078A (en) * 1995-12-29 2000-08-01 Lucent Technologies Inc. Maintaining consistency of database replicas
US6941287B1 (en) * 1999-04-30 2005-09-06 E. I. Du Pont De Nemours And Company Distributed hierarchical evolutionary modeling and visualization of empirical data
US20030221095A1 (en) * 2000-02-19 2003-11-27 Powerquest Corporation Computer imaging recovery without a working partition or a secondary medium
US20100268986A1 (en) * 2007-06-14 2010-10-21 International Business Machines Corporation Multi-node configuration of processor cards connected via processor fabrics
US20130120449A1 (en) * 2010-04-28 2013-05-16 Noboru IHARA Information processing system, information processing method and program
US20160292249A1 (en) * 2013-06-13 2016-10-06 Amazon Technologies, Inc. Dynamic replica failure detection and healing
US20150113324A1 (en) * 2013-10-21 2015-04-23 International Business Machines Corporation Automated Data Recovery from Remote Data Object Replicas
US20150199224A1 (en) * 2014-01-10 2015-07-16 Instep Software, Llc Method and Apparatus for Detection of Anomalies in Integrated Parameter Systems
US20150205661A1 (en) * 2014-01-20 2015-07-23 Lenovo Enterprise Solutions (Singapore) Pte. Ltd. Handling system interrupts with long-running recovery actions
US20160164739A1 (en) * 2014-12-09 2016-06-09 Ciena Corporation Reduced link bandwidth update systems and methods for improved scalability, efficiency, and performance
US20170026441A1 (en) * 2015-07-23 2017-01-26 Christopher Moudy Real-Time Partitioned Processing Streaming
US20170180599A1 (en) * 2015-12-18 2017-06-22 Kyocera Document Solutions Inc. Electronic device and non-transitory computer-readable storage medium

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
Apache Cassandra Project, "How is data maintained?", Apache Cassandra 2.2 Distribution, (2017-07-27), URL: https://docs.datastax.com/en/archived/cassandra/2.2/zip/cassandra22.zip, (2019-06-25) *
Apache Cassandra Project, "Manual repair: Anti-entropy repair", Apache Cassandra 2.2 Distribution, (2017-07-27), URL: https://docs.datastax.com/en/archived/cassandra/2.2/zip/cassandra22.zip, (2019-06-25) *
Apache Cassandra Project, "nodetool repair", Apache Cassandra 2.2 Distribution, (2017-07-27), URL: https://docs.datastax.com/en/archived/cassandra/2.2/zip/cassandra22.zip, (2019-06-25) *
Apache Cassandra Project, "When to run anti-entropy repair", Apache Cassandra 2.2 Distribution, (2017-07-27), URL: https://docs.datastax.com/en/archived/cassandra/2.2/zip/cassandra22.zip, (2019-06-25) *
Reaper Project, "Cassandra Backend", Reaper Online Documentation, (2018-03-13), URL: https://web.archive.org/web/2018-03-13200125/http://cassandra-reaper.io/docs/backends/cassandra/, (2019-06-25) *
Reaper Project, "Reaper Specific Settings", Reaper Online Documentation, (2018-03-16), URL: https://web.archive.org/web/2018-03-16064608/http://cassandra-reaper.io/docs/configuration/reaper_specific/, (2019-06-25) *
Reaper Project, "Reaper with multi DC clusters", Reaper Online Documentation, (2018-03-14), URL: https://web.archive.org/web/2018-03-14211841/http://cassandra-reaper.io/docs/usage/multi_dc/, (2019-06-25) *
Reaper Project, "Welcome to the Cassandra Reaper documentation+", Reaper Online Documentation, (2018-03-24), URL: https://web.archive.org/web/2018-03-24054004/http://cassandra-reaper.io/docs, (2019-06-25) *

Also Published As

Publication number Publication date
US11119845B2 (en) 2021-09-14
CA3095033A1 (en) 2019-10-03
WO2019191320A1 (en) 2019-10-03
SG11202009391UA (en) 2020-10-29
JP7191493B2 (ja) 2022-12-19
JP2021532428A (ja) 2021-11-25
KR20200131899A (ko) 2020-11-24
KR102509256B1 (ko) 2023-03-14
MX2020010009A (es) 2021-01-15
US20190303234A1 (en) 2019-10-03
EP3776198A1 (en) 2021-02-17
CA3095033C (en) 2023-03-28
US11636005B2 (en) 2023-04-25
US20210406116A1 (en) 2021-12-30
CN112219189B (zh) 2024-07-23
CN112219189A (zh) 2021-01-12
BR112020019697A2 (pt) 2021-01-05
AU2019244116A1 (en) 2020-10-15

Similar Documents

Publication Publication Date Title
US9984140B1 (en) Lease based leader election system
EP4078394B1 (en) Watermark-based techniques for change-data-capture
US9805070B2 (en) Dynamic migration script management
US9892183B2 (en) Computer system, computer system management method, and program
CN112955874A (zh) 在使用区块链的机器学习的去中心化模型构建中进行自修复的系统及方法
US9445162B2 (en) Interactive personal/internet protocol television reservation system, reservation plan management method and device
US11675740B2 (en) Tiered cache filling
WO2012120667A1 (ja) 計算機システム、データ複製スケジューリング方法及び計算機読み取り可能な非一時的記憶媒体
WO2019242455A1 (en) Method and apparatus for user request forwarding, reverse proxy and computer readable storage medium
US11636005B2 (en) Techniques for scheduled anti-entropy repair design
CN112579550B (zh) 一种分布式文件系统的元数据信息同步方法及系统
WO2017005116A1 (zh) 中间文件处理方法、客户端、服务器和系统
US10749921B2 (en) Techniques for warming up a node in a distributed data store
JP2014229088A (ja) データ処理システム、データ処理装置および記憶媒体
CN111630500B (zh) 信息分布式存储系统、方法及程序
US20150088826A1 (en) Enhanced Performance for Data Duplication
US8705537B1 (en) Eventually-consistent data stream consolidation
US20150347239A1 (en) Global backup lock manager
US20260122067A1 (en) Filtering updated alias information
JP5054495B2 (ja) 計算機システム、データ管理方法、データ管理プログラム及び処理装置
CN120450408A (zh) 一种生成线路编码的方法和装置
CN121858537A (zh) 一种分布式数据库的元数据管理方法、设备及介质

Legal Events

Date Code Title Description
FGA Letters patent sealed or granted (standard patent)