CN112219189B - 用于调度的反熵修复设计的技术 - Google Patents

用于调度的反熵修复设计的技术 Download PDF

Info

Publication number
CN112219189B
CN112219189B CN201980034671.0A CN201980034671A CN112219189B CN 112219189 B CN112219189 B CN 112219189B CN 201980034671 A CN201980034671 A CN 201980034671A CN 112219189 B CN112219189 B CN 112219189B
Authority
CN
China
Prior art keywords
repair
node
nodes
inverse entropy
procedure
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201980034671.0A
Other languages
English (en)
Chinese (zh)
Other versions
CN112219189A (zh
Inventor
维奈·切拉
约瑟夫·林奇
阿贾伊·乌帕德海耶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Netflix Inc
Original Assignee
Netflix Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Netflix Inc filed Critical Netflix Inc
Publication of CN112219189A publication Critical patent/CN112219189A/zh
Application granted granted Critical
Publication of CN112219189B publication Critical patent/CN112219189B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
    • G06F11/0709—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0751—Error or fault detection not based on redundancy
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0793—Remedial or corrective actions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/273—Asynchronous replication or reconciliation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Hardware Design (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
CN201980034671.0A 2018-03-27 2019-03-27 用于调度的反熵修复设计的技术 Active CN112219189B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201862648907P 2018-03-27 2018-03-27
US62/648,907 2018-03-27
PCT/US2019/024417 WO2019191320A1 (en) 2018-03-27 2019-03-27 Techniques for scheduled anti-entropy repair design

Publications (2)

Publication Number Publication Date
CN112219189A CN112219189A (zh) 2021-01-12
CN112219189B true CN112219189B (zh) 2024-07-23

Family

ID=66103054

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201980034671.0A Active CN112219189B (zh) 2018-03-27 2019-03-27 用于调度的反熵修复设计的技术

Country Status (11)

Country Link
US (2) US11119845B2 (enExample)
EP (1) EP3776198A1 (enExample)
JP (1) JP7191493B2 (enExample)
KR (1) KR102509256B1 (enExample)
CN (1) CN112219189B (enExample)
AU (1) AU2019244116B2 (enExample)
BR (1) BR112020019697A2 (enExample)
CA (1) CA3095033C (enExample)
MX (1) MX2020010009A (enExample)
SG (1) SG11202009391UA (enExample)
WO (1) WO2019191320A1 (enExample)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12481703B2 (en) * 2020-03-30 2025-11-25 International Business Machines Corporation Shard hashing for database objects within a distributed database
DE102023102529B4 (de) 2023-02-02 2024-08-29 Deutsches Zentrum für Luft- und Raumfahrt e.V. Verfahren zur Übertragung eines Prüfvektors von einer Sendeeinheit an eine Empfangseinheit

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6098078A (en) * 1995-12-29 2000-08-01 Lucent Technologies Inc. Maintaining consistency of database replicas

Family Cites Families (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB9812221D0 (en) 1998-06-05 1998-08-05 British Telecomm Distributed database system
US6941287B1 (en) * 1999-04-30 2005-09-06 E. I. Du Pont De Nemours And Company Distributed hierarchical evolutionary modeling and visualization of empirical data
US7216251B2 (en) * 2000-02-19 2007-05-08 Powerquest Corporation Computer imaging recovery without a working partition or a secondary medium
JP2005352708A (ja) * 2004-06-10 2005-12-22 Hitachi Ltd オンライン同期スケジュール回復システムおよびその処理方法
US7685109B1 (en) 2005-12-29 2010-03-23 Amazon Technologies, Inc. Method and apparatus for data partitioning and replication in a searchable data service
US7783813B2 (en) * 2007-06-14 2010-08-24 International Business Machines Corporation Multi-node configuration of processor cards connected via processor fabrics
JP5452015B2 (ja) * 2008-12-15 2014-03-26 日本証券テクノロジー株式会社 ファイル整合性確認システム、ファイル整合性確認方法、プログラム
US7984161B2 (en) * 2009-06-16 2011-07-19 Microsoft Corporation Optimizing mobile device synchronization with multiple data sources
JP4913913B2 (ja) * 2010-04-28 2012-04-11 新日鉄ソリューションズ株式会社 情報処理システム、情報処理方法及びプログラム
US8234372B2 (en) * 2010-05-05 2012-07-31 Go Daddy Operating Company, LLC Writing a file to a cloud storage solution
CN102143194A (zh) * 2010-06-10 2011-08-03 华为技术有限公司 数据同步的方法、系统、中间数据节点及终止数据节点
JP5751029B2 (ja) * 2011-06-03 2015-07-22 株式会社リコー 認証装置、プログラムおよび記録媒体
US20140236902A1 (en) * 2011-09-29 2014-08-21 Beijing Qihoo Technology Company Limited User Data Backup Method and Apparatus
US9836514B2 (en) * 2011-11-07 2017-12-05 Empire Technology Development Llc Cache based key-value store mapping and replication
US9112809B2 (en) * 2012-11-21 2015-08-18 Telefonaktiebolaget L M Ericsson (Publ) Method and apparatus for controlling utilization in a horizontally scaled software application
US9002805B1 (en) * 2012-12-14 2015-04-07 Amazon Technologies, Inc. Conditional storage object deletion
KR101497669B1 (ko) * 2013-04-29 2015-03-11 서울대학교산학협력단 일정 수 이상의 파일 조각으로 원본 파일을 복구하는 파일 관리 장치 및 방법
US9304815B1 (en) * 2013-06-13 2016-04-05 Amazon Technologies, Inc. Dynamic replica failure detection and healing
EP2953025B1 (en) * 2013-10-18 2017-01-25 Hitachi Ltd. Target-driven independent data integrity and redundancy recovery in a shared-nothing distributed storage system
US9264494B2 (en) * 2013-10-21 2016-02-16 International Business Machines Corporation Automated data recovery from remote data object replicas
US9379951B2 (en) * 2014-01-10 2016-06-28 Instep Software, Llc Method and apparatus for detection of anomalies in integrated parameter systems
US9519532B2 (en) * 2014-01-20 2016-12-13 Lenovo Enterprise Solutions (Singapore) Pte. Ltd. Handling system interrupts with long-running recovery actions
US9600203B2 (en) 2014-03-11 2017-03-21 Amazon Technologies, Inc. Reducing data volume durability state for block-based storage
CN105446653B (zh) 2014-08-27 2018-12-14 阿里巴巴集团控股有限公司 一种数据合并方法和设备
US9515878B2 (en) * 2014-11-13 2016-12-06 Software AG USA Inc. Method, medium, and system for configuring a new node in a distributed memory network
US10355935B2 (en) * 2014-12-09 2019-07-16 Ciena Corporation Reduced link bandwidth update systems and methods for improved scalability, efficiency, and performance
US10528530B2 (en) * 2015-04-08 2020-01-07 Microsoft Technology Licensing, Llc File repair of file stored across multiple data stores
US10884869B2 (en) 2015-04-16 2021-01-05 Nuodb, Inc. Backup and restore in a distributed database utilizing consistent database snapshots
US9940169B2 (en) * 2015-07-23 2018-04-10 Pearson Education, Inc. Real-time partitioned processing streaming
KR20170022001A (ko) * 2015-08-19 2017-03-02 엘지전자 주식회사 데이터 송수신 방법 및 이를 위한 디바이스
JP6515799B2 (ja) * 2015-12-18 2019-05-22 京セラドキュメントソリューションズ株式会社 電子機器及びメモリー寿命警告プログラム
US10360095B2 (en) * 2016-03-31 2019-07-23 Change Healthcare Holdings, Llc Methods and apparatuses for improving failure recovery in a distributed system
JP6720744B2 (ja) * 2016-07-15 2020-07-08 富士通株式会社 情報処理システム、情報処理装置、及び制御プログラム
US10990467B2 (en) * 2016-12-15 2021-04-27 Nutanix, Inc. Accessing computing resource attributes of an external service provider
US20180225324A1 (en) * 2017-02-06 2018-08-09 Qualcomm Incorporated Providing Retry Schedules For File Repair Over Broadcast Networks

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6098078A (en) * 1995-12-29 2000-08-01 Lucent Technologies Inc. Maintaining consistency of database replicas

Also Published As

Publication number Publication date
US11119845B2 (en) 2021-09-14
CA3095033A1 (en) 2019-10-03
WO2019191320A1 (en) 2019-10-03
SG11202009391UA (en) 2020-10-29
JP7191493B2 (ja) 2022-12-19
JP2021532428A (ja) 2021-11-25
KR20200131899A (ko) 2020-11-24
KR102509256B1 (ko) 2023-03-14
AU2019244116B2 (en) 2021-10-07
MX2020010009A (es) 2021-01-15
US20190303234A1 (en) 2019-10-03
EP3776198A1 (en) 2021-02-17
CA3095033C (en) 2023-03-28
US11636005B2 (en) 2023-04-25
US20210406116A1 (en) 2021-12-30
CN112219189A (zh) 2021-01-12
BR112020019697A2 (pt) 2021-01-05
AU2019244116A1 (en) 2020-10-15

Similar Documents

Publication Publication Date Title
TWI783355B (zh) 深度學習模型的分布式訓練方法以及裝置
US9430321B2 (en) Reconstructing data stored across archival data storage devices
US9031910B2 (en) System and method for maintaining a cluster setup
US20170123848A1 (en) Multi-task processing in a distributed storage network
US9424156B2 (en) Identifying a potential failure event for a data storage device
US9557938B2 (en) Data retrieval based on storage device activation schedules
JP2017518565A (ja) 散在ストレージ・ネットワークにおける多世代記憶されたデータの読取り
CN109558260B (zh) Kubernetes故障排除系统、方法、设备及介质
AU2023241318B1 (en) Watermark-based techniques for change-data-capture
CN112579550B (zh) 一种分布式文件系统的元数据信息同步方法及系统
US11500812B2 (en) Intermediate file processing method, client, server, and system
CN113821168A (zh) 一种共享存储迁移系统、方法及电子设备和存储介质
CN112219189B (zh) 用于调度的反熵修复设计的技术
JP2014229088A (ja) データ処理システム、データ処理装置および記憶媒体
CN103685359B (zh) 数据处理方法及装置
CN113946274B (zh) 数据处理方法、装置、设备及介质
CN111630500B (zh) 信息分布式存储系统、方法及程序
CN114676291B (zh) 一种数据库系统及数据库系统的控制方法
US20150088826A1 (en) Enhanced Performance for Data Duplication
JP6499038B2 (ja) ストレージ管理装置、方法、及びストレージシステム
US11770448B1 (en) Rotating offline storage units in a dispersed storage network
CN114820218A (zh) 内容操作方法、装置、服务器及存储介质
CN118093269A (zh) 数据修复方法、装置、设备及存储介质
CN118981498A (zh) 分布式数据库的实时数据处理方法、装置及存储介质
KR20160118402A (ko) Vdi 대용량 데이터 처리를 위한 액세스 패턴 기반 캐시 관리 방법

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant