SG11201805201XA - Recoverable stream processing - Google Patents

Recoverable stream processing

Info

Publication number
SG11201805201XA
SG11201805201XA SG11201805201XA SG11201805201XA SG11201805201XA SG 11201805201X A SG11201805201X A SG 11201805201XA SG 11201805201X A SG11201805201X A SG 11201805201XA SG 11201805201X A SG11201805201X A SG 11201805201XA SG 11201805201X A SG11201805201X A SG 11201805201XA
Authority
SG
Singapore
Prior art keywords
processing
data
international
stream
data units
Prior art date
Application number
SG11201805201XA
Inventor
Bryan Phil Douros
Craig W Stanfill
Joseph Skeffington Wholey
Original Assignee
Ab Initio Technology Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ab Initio Technology Llc filed Critical Ab Initio Technology Llc
Publication of SG11201805201XA publication Critical patent/SG11201805201XA/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/70—Media network packetisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1415—Saving, restoring, recovering or retrying at system level
    • G06F11/1438—Restarting or rejuvenating
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2053—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
    • G06F11/2094—Redundant storage or storage space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G06F11/2097—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements maintaining the standby controller/processing unit updated
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/84—Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) (19) World Intellectual Property -, Organization MD 1111101110101011111 0101111101110111011101111101fillOIMIOU III International Bureau 0.. .1 j ..... ..or::,„, (10) International Publication Number (43) International Publication Date WO 2017/123849 Al 20 July 2017 (20.07.2017) WIPO I PCT (51) International Patent Classification: (81) Designated States (unless otherwise indicated, for every GO6F 9 / 5 2 (2006.01) GO6F 11/14 (2006.01) kind of national protection available): AE, AG, AL, AM, AO, AT, AU, AZ, BA, BB, BG, BH, BN, BR, BW, BY, (21) International Application Number: BZ, CA, CH, CL, CN, CO, CR, CU, CZ, DE, DJ, DK, DM, PCT/US2017/013309 DO, DZ, EC, EE, EG, ES, FI, GB, GD, GE, GH, GM, GT, (22) International Filing Date: HN, HR, HU, ID, IL, IN, IR, IS, JP, KE, KG, KH, KN, 13 January 2017 (13.01.2017) KP, KR, KW, KZ, LA, LC, LK, LR, LS, LU, LY, MA, MD, ME, MG, MK, MN, MW, MX, MY, MZ, NA, NG, (25) Filing Language: English NI, NO, NZ, OM, PA, PE, PG, PH, PL, PT, QA, RO, RS, (26) Publication Language: English RU, RW, SA, SC, SD, SE, SG, SK, SL, SM, ST, SV, SY, TH, TJ, TM, TN, TR, TT, TZ, UA, UG, US, UZ, VC, VN, (30) Priority Data: ZA, ZM, ZW. 62/278,528 14 January 2016 (14.01.2016) US (84) Designated States (unless otherwise indicated, for every (71) Applicant: AB INITIO TECHNOLOGY LLC [US/US]; kind of regional protection available): ARIPO (BW, GH, 201 Spring Street, Lexington, Massachusetts 02421 (US). GM, KE, LR, LS, MW, MZ, NA, RW, SD, SL, ST, SZ, (72) Inventors: DOUROS, Bryan Phil; 92 Lakeview Road, TZ, UG, ZM, ZW), Eurasian (AM, AZ, BY, KG, KZ, RU, Framingham, Massachusetts 01701 (US). STANFILL, TJ, TM), European (AL, AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, FI, FR, GB, GR, HR, HU, IE, IS, IT, LT, LU, Craig W.; 43 Huckleberry Hill Road, Lincoln, Massachu- LV, MC, MK, MT, NL, NO, PL, PT, RO, RS, SE, SI, SK, setts 01773 (US). WHOLEY, Joseph Skeffington, III; 11 Hillcrest Road, Belmont, Massachusetts 02478 (US). SM, TR), OAPI (BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, KM, ML, MR, NE, SN, TD, TG). Agent: MASON, Elliott J., III; Occhiuti & Rohlicek LLP, (74) Published: 321 Summer Street, Boston, Massachusetts 02210 (US). — with international search report (Art. 21(3)) = (54) Title: RECOVERABLE STREAM PROCESSING = Processing of First Pulse 100 112C 112B 112A 132A M 2 122A ' G%////r, . HUH 122A, 121 1g 131 135 122A121 113A- 121 141 142 11 t:t 7r GC FIG. 7B M \" (57) : A computing system (100) includes nodes (152) executing data processing programs that each process at least one 1-1 ---- stream of data units. A data storage system (156, 157) stores shared data accessible by at least two of the programs. Processing at I r ----- least one stream using a first data processing program includes: processing a first stream of data units that includes multiple subsets 1-1 C of contiguous data units; initiating termination of processing within the first data processing program, between processing a first sub - N set of contiguous data units and processing a second subset of contiguous data units adjacent to the first subset of contiguous data 0 units within the first stream of data units; durably storing at least some changes to the shared data caused by processing the first sub - ,-t set of contiguous data units after determining that the termination of processing within the first data processing program has com- pleted; and resuming processing within the first data processing program.
SG11201805201XA 2016-01-14 2017-01-13 Recoverable stream processing SG11201805201XA (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201662278528P 2016-01-14 2016-01-14
PCT/US2017/013309 WO2017123849A1 (en) 2016-01-14 2017-01-13 Recoverable stream processing

Publications (1)

Publication Number Publication Date
SG11201805201XA true SG11201805201XA (en) 2018-07-30

Family

ID=58016792

Family Applications (1)

Application Number Title Priority Date Filing Date
SG11201805201XA SG11201805201XA (en) 2016-01-14 2017-01-13 Recoverable stream processing

Country Status (9)

Country Link
US (1) US10601890B2 (en)
EP (3) EP3614266B1 (en)
JP (1) JP6764938B2 (en)
KR (1) KR102121139B1 (en)
CN (1) CN108475218B (en)
AU (1) AU2017207404B2 (en)
CA (1) CA3009359C (en)
SG (1) SG11201805201XA (en)
WO (1) WO2017123849A1 (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10439917B2 (en) * 2016-11-15 2019-10-08 At&T Intellectual Property I, L.P. Recovering a replica in an operator in a data streaming processing system
WO2019089599A1 (en) * 2017-10-31 2019-05-09 Ab Initio Technology Llc Managing a computing cluster using durability level indicators
US10922199B2 (en) * 2018-07-04 2021-02-16 Vmware, Inc. Role management of compute nodes in distributed clusters
US10664397B2 (en) * 2018-07-31 2020-05-26 EMC IP Holding Company LLC Cache recovery method in a distributed storage system
US10489321B1 (en) 2018-07-31 2019-11-26 EMC IP Holding Company LLC Performance improvement for an active-active distributed non-ALUA system with address ownerships
US10733191B2 (en) 2018-09-28 2020-08-04 Microsoft Technology Licensing, Llc Static streaming job startup sequence
EP3871400A1 (en) * 2018-10-23 2021-09-01 tZERO IP, LLC Context based filtering within subsets of network nodes implementing a trading system
CN110018926B (en) * 2018-11-22 2023-08-01 创新先进技术有限公司 Fault recovery method, device, electronic equipment and computer readable storage medium
JP7331604B2 (en) * 2019-10-04 2023-08-23 富士通株式会社 Information processing system, information processing method, and information processing program
CN111541747B (en) * 2020-04-10 2023-05-12 深圳前海微众银行股份有限公司 Data check point setting method and device
US20220318099A1 (en) * 2021-03-31 2022-10-06 Nutanix, Inc. File analytics systems and methods including retrieving metadata from file system snapshots

Family Cites Families (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2281644A (en) * 1993-09-02 1995-03-08 Ibm Fault tolerant transaction-oriented data processing.
US5712971A (en) 1995-12-11 1998-01-27 Ab Initio Software Corporation Methods and systems for reconstructing the state of a computation
US5857204A (en) 1996-07-02 1999-01-05 Ab Initio Software Corporation Restoring the state of a set of files
US6584581B1 (en) * 1999-12-06 2003-06-24 Ab Initio Software Corporation Continuous flow checkpointing data processing
CA2355473A1 (en) * 2000-09-29 2002-03-29 Linghsiao Wang Buffer management for support of quality-of-service guarantees and data flow control in data switching
DE10161034A1 (en) * 2001-12-12 2003-07-03 Siemens Ag Method for combining source code, e.g. subroutine code, from a number of sources and locations in a program in a central project leader location, whereby combined program code is made available for transfer or read access
US7039663B1 (en) * 2002-04-19 2006-05-02 Network Appliance, Inc. System and method for checkpointing and restarting an asynchronous transfer of data between a source and destination snapshot
JP2004178788A (en) * 2002-11-15 2004-06-24 Matsushita Electric Ind Co Ltd Recording device and recording method
US7353218B2 (en) * 2003-08-14 2008-04-01 International Business Machines Corporation Methods and apparatus for clustering evolving data streams through online and offline components
CN100390738C (en) * 2005-10-18 2008-05-28 中国科学院计算技术研究所 Combination and optimization methods of access and storage based on analysis of data stream
US8069129B2 (en) 2007-04-10 2011-11-29 Ab Initio Technology Llc Editing and compiling business rules
US8225129B2 (en) * 2007-04-10 2012-07-17 International Business Machines Corporation Methods and apparatus for effective on-line backup selection for failure recovery in distributed stream processing systems
US7779298B2 (en) * 2007-06-11 2010-08-17 International Business Machines Corporation Distributed job manager recovery
JP4560074B2 (en) * 2007-09-26 2010-10-13 株式会社東芝 Virtual computer system and virtual computer restoration method in the same system
US8949801B2 (en) * 2009-05-13 2015-02-03 International Business Machines Corporation Failure recovery for stream processing applications
US8286191B2 (en) * 2009-05-14 2012-10-09 International Business Machines Corporation Dynamically composing data stream processing applications
US8205113B2 (en) * 2009-07-14 2012-06-19 Ab Initio Technology Llc Fault tolerant batch processing
JP4925143B2 (en) * 2009-08-12 2012-04-25 株式会社日立製作所 Stream data processing system, stream data processing method, and stream data processing program
US9021299B2 (en) * 2011-02-18 2015-04-28 Ab Initio Technology Llc Restarting processes
US9116759B2 (en) * 2011-02-18 2015-08-25 Ab Initio Technology Llc Restarting data processing systems
US8826072B2 (en) 2012-05-09 2014-09-02 Imec Method and system for real-time error mitigation
US8843524B2 (en) * 2012-09-25 2014-09-23 International Business Machines Corporation Handling out-of-sequence data in a streaming environment
US9298788B1 (en) * 2013-03-11 2016-03-29 DataTorrent, Inc. Checkpointing in distributed streaming platform for real-time applications
CA2926935C (en) 2013-10-21 2022-05-31 Ab Initio Technology Llc Checkpointing a collection of data units
US9641580B2 (en) * 2014-07-01 2017-05-02 Microsoft Technology Licensing, Llc Distributed stream processing in the cloud
WO2016064705A1 (en) 2014-10-20 2016-04-28 Ab Initio Technology Llc Recovery and fault-tolerance under computational indeterminism
US10339796B2 (en) * 2015-07-07 2019-07-02 Ilumi Sulutions, Inc. Wireless control device and methods thereof

Also Published As

Publication number Publication date
JP6764938B2 (en) 2020-10-07
AU2017207404B2 (en) 2020-12-10
CN108475218A (en) 2018-08-31
EP3403182A1 (en) 2018-11-21
CA3009359A1 (en) 2017-07-20
EP3614267A2 (en) 2020-02-26
EP3614267A3 (en) 2020-07-29
CN108475218B (en) 2022-03-22
KR20180101547A (en) 2018-09-12
EP3614266B1 (en) 2022-03-09
CA3009359C (en) 2020-09-15
US10601890B2 (en) 2020-03-24
EP3614266A3 (en) 2020-07-29
JP2019505916A (en) 2019-02-28
KR102121139B1 (en) 2020-06-17
EP3403182B1 (en) 2019-09-04
WO2017123849A1 (en) 2017-07-20
EP3614266A2 (en) 2020-02-26
AU2017207404A1 (en) 2018-07-12
EP3614267B1 (en) 2022-03-09
US20170208113A1 (en) 2017-07-20

Similar Documents

Publication Publication Date Title
SG11201805201XA (en) Recoverable stream processing
SG11201903098XA (en) Multiparty reconciliation systems and methods
SG11201908719QA (en) Biomarkers and car t cell therapies with enhanced efficacy
SG11201811424PA (en) Mirrored token vault
SG11201810588TA (en) Motorised scooter
SG11201805253SA (en) Compositions and methods related to multimodal therapeutic cell systems for cancer indications
SG11201807021UA (en) Heteroaryl inhibitors of pad4
SG11201803920TA (en) Compounds and compositions useful for treating disorders related to ntrk
SG11201807401RA (en) Therapeutic membrane vesicles
SG11201805266YA (en) Systems and methods for device push provisioning
SG11201906891RA (en) Methods for the administration of certain vmat2 inhibitors
SG11201906386XA (en) Combination therapy involving diaryl macrocyclic compounds
SG11201804699WA (en) Open-top microfluidic device with structural anchors
SG11201811283PA (en) System and method for determining safety score of driver
SG11201811492TA (en) Automation vending store
SG11201810466PA (en) Compositions and methods related to engineered fc constructs
SG11201407580YA (en) Composition comprising two antibodies engineered to have reduced and increased effector function
SG11201907023UA (en) Method of reducing neutropenia
SG11201807421TA (en) The use of glucocorticoid receptor modulators to potentiate checkpoint inhibitors
SG11201809097WA (en) Reconfigurable distributed processing
SG11201810927QA (en) Pd-l1-specific antibodies and methods of using the same
SG11201804841VA (en) Hardware integrity check
SG11201803928UA (en) Processing data using dynamic partitioning
SG11201901403VA (en) Tracing objects across different parties
SG11201900993QA (en) Track system and method for providing a track for an industrial cart