JP2019535058A5 - - Google Patents

Download PDF

Info

Publication number
JP2019535058A5
JP2019535058A5 JP2019512634A JP2019512634A JP2019535058A5 JP 2019535058 A5 JP2019535058 A5 JP 2019535058A5 JP 2019512634 A JP2019512634 A JP 2019512634A JP 2019512634 A JP2019512634 A JP 2019512634A JP 2019535058 A5 JP2019535058 A5 JP 2019535058A5
Authority
JP
Japan
Prior art keywords
computing device
microbatch
output
query
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2019512634A
Other languages
English (en)
Japanese (ja)
Other versions
JP7018435B2 (ja
JP2019535058A (ja
Filing date
Publication date
Application filed filed Critical
Priority claimed from PCT/US2017/051897 external-priority patent/WO2018053343A1/en
Publication of JP2019535058A publication Critical patent/JP2019535058A/ja
Publication of JP2019535058A5 publication Critical patent/JP2019535058A5/ja
Application granted granted Critical
Publication of JP7018435B2 publication Critical patent/JP7018435B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

JP2019512634A 2016-09-15 2017-09-15 スナップショットおよび状態のマイクロバッチ方式による管理 Active JP7018435B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
IN201641031479 2016-09-15
IN201641031479 2016-09-15
PCT/US2017/051897 WO2018053343A1 (en) 2016-09-15 2017-09-15 Managing snapshots and state with micro-batching

Publications (3)

Publication Number Publication Date
JP2019535058A JP2019535058A (ja) 2019-12-05
JP2019535058A5 true JP2019535058A5 (enExample) 2020-07-30
JP7018435B2 JP7018435B2 (ja) 2022-02-10

Family

ID=59955760

Family Applications (2)

Application Number Title Priority Date Filing Date
JP2019511926A Active JP7005600B2 (ja) 2016-09-15 2017-09-15 マイクロバッチストリーミングのための複合イベント処理
JP2019512634A Active JP7018435B2 (ja) 2016-09-15 2017-09-15 スナップショットおよび状態のマイクロバッチ方式による管理

Family Applications Before (1)

Application Number Title Priority Date Filing Date
JP2019511926A Active JP7005600B2 (ja) 2016-09-15 2017-09-15 マイクロバッチストリーミングのための複合イベント処理

Country Status (5)

Country Link
US (3) US11615088B2 (enExample)
EP (3) EP3584715B1 (enExample)
JP (2) JP7005600B2 (enExample)
CN (2) CN109716322B (enExample)
WO (2) WO2018053343A1 (enExample)

Families Citing this family (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10956422B2 (en) 2012-12-05 2021-03-23 Oracle International Corporation Integrating event processing with map-reduce
US11573965B2 (en) 2016-09-15 2023-02-07 Oracle International Corporation Data partitioning and parallelism in a distributed event processing system
US11615088B2 (en) 2016-09-15 2023-03-28 Oracle International Corporation Complex event processing for micro-batch streaming
WO2018169430A1 (en) 2017-03-17 2018-09-20 Oracle International Corporation Integrating logic in micro batch based event processing systems
WO2018169429A1 (en) 2017-03-17 2018-09-20 Oracle International Corporation Framework for the deployment of event-based applications
CN108228356B (zh) * 2017-12-29 2021-01-15 华中科技大学 一种流数据的分布式动态处理方法
US10572481B1 (en) * 2018-03-26 2020-02-25 Jeffrey M. Gunther System and method for integrating health information sources
WO2020128596A1 (en) * 2018-12-21 2020-06-25 Pratik Sharma Continuous event stream processor
US11461345B2 (en) * 2019-05-31 2022-10-04 Microsoft Technology Licensing, Llc Providing access to state information associated with operators in a data processing system
CN110929623A (zh) * 2019-11-15 2020-03-27 北京达佳互联信息技术有限公司 多媒体文件的识别方法、装置、服务器和存储介质
US20210373914A1 (en) * 2020-05-29 2021-12-02 Intuit Inc. Batch to stream processing in a feature management platform
CN112019605B (zh) * 2020-08-13 2023-05-09 上海哔哩哔哩科技有限公司 数据流的数据分发方法和系统
CN112506939B (zh) * 2020-12-08 2024-10-18 福建省海峡信息技术有限公司 一种基于Flink流式引擎的CEP规则更新方法
US20210109785A1 (en) * 2020-12-23 2021-04-15 Intel Corporation Methods, systems, articles of manufacture and apparatus to batch functions
US12210981B2 (en) * 2021-03-31 2025-01-28 International Business Machines Corporation Auto feature preparation for high performance online inferencing
CN113139658A (zh) * 2021-04-23 2021-07-20 上海中通吉网络技术有限公司 基于变更和告警事件流处理故障的管道理论方法
US11593367B1 (en) 2021-09-29 2023-02-28 Amazon Technologies, Inc. Selecting between hydration-based scanning and stateless scale-out scanning to improve query performance
US11695696B2 (en) 2021-11-23 2023-07-04 Capital One Services, Llc Prepopulation of caches
EP4437433A1 (en) 2021-11-23 2024-10-02 Capital One Services, LLC Stream listening cache updater
US11916787B2 (en) 2021-11-23 2024-02-27 Capital One Services, Llc Stream listening cache updater
US11765252B2 (en) 2021-11-23 2023-09-19 Capital One Services, Llc Prepopulation of call center cache
JP2025508662A (ja) 2022-01-21 2025-04-10 スリングショット エアロスペース，インコーポレイテッド ストリーミングバッファの数値伝播のための方法及びシステム
CN114490763B (zh) * 2022-01-24 2024-07-26 全知科技(杭州)有限责任公司 一种单机高性能批流一体数据处理系统
CN114625794A (zh) * 2022-03-10 2022-06-14 北京国电高科科技有限公司 一种卫星物联网Spark数据处理方法、系统、终端及存储介质
CN114490780B (zh) * 2022-03-31 2022-07-22 北京数变科技有限公司 一种数据流的调度方法、装置、电子设备、介质及产品
US12411706B2 (en) * 2022-08-10 2025-09-09 Capital One Financial Corporation Reconciliation systems and methods for unbounded streams
US12111825B2 (en) * 2022-11-10 2024-10-08 Bank Of America Corporation Event-driven batch processing system with granular operational access
US20240202198A1 (en) * 2022-12-16 2024-06-20 Microsoft Technology Licensing, Llc Continuous query using anchored paths
US20240331703A1 (en) * 2023-03-30 2024-10-03 Comcast Cable Communications, Llc Systems and methods for processing voice queries of different duration
US20250036627A1 (en) * 2023-07-25 2025-01-30 Sap Se Deepest-only scheduling
US12450231B1 (en) 2024-05-13 2025-10-21 Zoom Communications, Inc. Global directed acyclic graphs in distributed computing systems

Family Cites Families (119)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2335514B (en) 1998-03-18 2003-01-22 Ibm A method and component for serialisation of images
US6633867B1 (en) 2000-04-05 2003-10-14 International Business Machines Corporation System and method for providing a session query within the context of a dynamic search result set
US7139844B2 (en) 2000-08-04 2006-11-21 Goldman Sachs & Co. Method and system for processing financial data objects carried on broadcast data streams and delivering information to subscribing clients
US7139977B1 (en) 2001-01-24 2006-11-21 Oracle International Corporation System and method for producing a virtual online book
JP2003208323A (ja) * 2002-01-11 2003-07-25 Hitachi Ltd バッチジョブの実行方法、システムおよびプログラム
US7840550B2 (en) 2002-08-13 2010-11-23 International Business Machines Corporation System and method for monitoring database queries
US20040060054A1 (en) 2002-09-20 2004-03-25 International Business Machines Corporation Composition service for autonomic computing
JP2004280283A (ja) 2003-03-13 2004-10-07 Hitachi Ltd 分散ファイルシステム、分散ファイルシステムサーバ及び分散ファイルシステムへのアクセス方法
US6986019B1 (en) 2003-04-21 2006-01-10 Maxtor Corporation Method and apparatus for detection and management of data streams
US7546284B1 (en) 2003-06-11 2009-06-09 Blue Titan Software, Inc. Virtual message persistence service
US7047252B2 (en) 2003-12-02 2006-05-16 Oracle International Corporation Complex computation across heterogenous computer systems
US7194451B2 (en) 2004-02-26 2007-03-20 Microsoft Corporation Database monitoring system
US20060167869A1 (en) 2005-01-27 2006-07-27 Kevin Jones Multi-path simultaneous Xpath evaluation over data streams
US8126870B2 (en) 2005-03-28 2012-02-28 Sybase, Inc. System and methodology for parallel query optimization using semantic-based partitioning
JP4687253B2 (ja) 2005-06-03 2011-05-25 株式会社日立製作所 ストリームデータ処理システムのクエリ処理方法
US20070168154A1 (en) 2005-12-23 2007-07-19 Ericson Richard E User interface for statistical data analysis
GB0606776D0 (en) 2006-04-03 2006-05-10 Novartis Pharma Ag Predictive biomarkers for chronic allograft nephropathy
US7974984B2 (en) 2006-04-19 2011-07-05 Mobile Content Networks, Inc. Method and system for managing single and multiple taxonomies
US20080016095A1 (en) 2006-07-13 2008-01-17 Nec Laboratories America, Inc. Multi-Query Optimization of Window-Based Stream Queries
US8077059B2 (en) 2006-07-21 2011-12-13 Eric John Davies Database adapter for relational datasets
US8099452B2 (en) 2006-09-05 2012-01-17 Microsoft Corporation Event stream conditioning
US7921416B2 (en) 2006-10-20 2011-04-05 Yahoo! Inc. Formal language and translator for parallel processing of data
US10152687B2 (en) 2006-12-01 2018-12-11 Goldman Sachs & Co. LLC Application directory
US7899904B2 (en) 2007-04-30 2011-03-01 Lsi Corporation Hardware processing of regular expressions
US20080301135A1 (en) 2007-05-29 2008-12-04 Bea Systems, Inc. Event processing query language using pattern matching
US20090070786A1 (en) 2007-09-11 2009-03-12 Bea Systems, Inc. Xml-based event processing networks for event server
US9811849B2 (en) 2007-09-28 2017-11-07 Great-Circle Technologies, Inc. Contextual execution of automated workflows
US7996388B2 (en) 2007-10-17 2011-08-09 Oracle International Corporation Adding new continuous queries to a data stream management system operating on existing queries
US8073826B2 (en) 2007-10-18 2011-12-06 Oracle International Corporation Support for user defined functions in a data stream management system
US9275353B2 (en) 2007-11-09 2016-03-01 Oracle America, Inc. Event-processing operators
US7870167B2 (en) 2007-11-09 2011-01-11 Oracle America, Inc. Implementing event processors
WO2009108782A1 (en) 2008-02-26 2009-09-03 Ab Initio Software Llc Graphic representations of data relationships
US7818370B2 (en) 2008-05-20 2010-10-19 Bea Systems, Inc. Event server using clustering
US8291006B2 (en) 2008-05-30 2012-10-16 International Business Machines Corporation Method for generating a distributed stream processing application
US20100030896A1 (en) * 2008-06-19 2010-02-04 Microsoft Corporation Estimating latencies for query optimization in distributed stream processing
US7945565B2 (en) 2008-11-20 2011-05-17 Yahoo! Inc. Method and system for generating a hyperlink-click graph
US8725707B2 (en) * 2009-03-26 2014-05-13 Hewlett-Packard Development Company, L.P. Data continuous SQL process
US8713038B2 (en) 2009-04-02 2014-04-29 Pivotal Software, Inc. Integrating map-reduce into a distributed relational database
US8180801B2 (en) 2009-07-16 2012-05-15 Sap Ag Unified window support for event stream data management
US20110035253A1 (en) 2009-08-07 2011-02-10 onFucus Healthcare Systems and Methods for Optimizing Enterprise Performance Relationships to Other Applications
JP5395565B2 (ja) 2009-08-12 2014-01-22 株式会社日立製作所 ストリームデータ処理方法及び装置
JP4992945B2 (ja) 2009-09-10 2012-08-08 株式会社日立製作所 ストリームデータ生成方法、ストリームデータ生成装置及びストリームデータ生成プログラム
US20110084967A1 (en) 2009-10-09 2011-04-14 International Business Machines Corporation Visualization of Datasets
US8195648B2 (en) 2009-10-21 2012-06-05 Microsoft Corporation Partitioned query execution in event processing systems
US9977702B2 (en) 2009-11-23 2018-05-22 International Business Machines Corporation Event processing networks
US8959106B2 (en) 2009-12-28 2015-02-17 Oracle International Corporation Class loading using java data cartridges
US9805101B2 (en) 2010-02-26 2017-10-31 Ebay Inc. Parallel data stream processing system
JP5331737B2 (ja) 2010-03-15 2013-10-30 株式会社日立製作所 ストリームデータ処理障害回復方法および装置
US8484243B2 (en) 2010-05-05 2013-07-09 Cisco Technology, Inc. Order-independent stream query processing
US8595234B2 (en) 2010-05-17 2013-11-26 Wal-Mart Stores, Inc. Processing data feeds
US8886631B2 (en) 2010-06-04 2014-11-11 Yale University Query execution systems and methods
US8935232B2 (en) 2010-06-04 2015-01-13 Yale University Query execution systems and methods
JP5605109B2 (ja) 2010-09-14 2014-10-15 日本電気株式会社 プログラム生成装置、方法及びプログラム並びに遠隔管理システム
US8260803B2 (en) 2010-09-23 2012-09-04 Hewlett-Packard Development Company, L.P. System and method for data stream processing
US9189280B2 (en) 2010-11-18 2015-11-17 Oracle International Corporation Tracking large numbers of moving objects in an event processing system
US8510284B2 (en) 2010-12-20 2013-08-13 Microsoft Corporation Large-scale event evaluation using realtime processors
US9201754B2 (en) 2011-01-19 2015-12-01 Red Hat, Inc. Recording application consumption details
US20130073586A1 (en) 2011-05-02 2013-03-21 Amadeus S.A.S. Database system using batch-oriented computation
US8880493B2 (en) 2011-09-28 2014-11-04 Hewlett-Packard Development Company, L.P. Multi-streams analytics
US9424150B2 (en) 2011-12-06 2016-08-23 Sap Se Fault tolerance based query execution
US9652502B2 (en) 2012-03-30 2017-05-16 Khalifa University Of Science, Technology And Research Method and system for continuous query processing
US10931735B2 (en) 2012-06-28 2021-02-23 Netflix, Inc. Application discovery
US9607045B2 (en) 2012-07-12 2017-03-28 Microsoft Technology Licensing, Llc Progressive query computation using streaming architectures
US9009213B2 (en) 2012-08-27 2015-04-14 Sap Se Distributing pre-rendering processing tasks
US8892484B2 (en) 2012-09-28 2014-11-18 Sphere Of Influence, Inc. System and method for predicting events
US9563663B2 (en) 2012-09-28 2017-02-07 Oracle International Corporation Fast path evaluation of Boolean predicates
US9361308B2 (en) 2012-09-28 2016-06-07 Oracle International Corporation State initialization algorithm for continuous queries over archived relations
JP5433760B2 (ja) 2012-10-18 2014-03-05 株式会社日立製作所 会議分析システム
US9183271B2 (en) * 2012-12-04 2015-11-10 Pivotal Software, Inc. Big-fast data connector between in-memory database system and data warehouse system
US10956422B2 (en) 2012-12-05 2021-03-23 Oracle International Corporation Integrating event processing with map-reduce
US20140172506A1 (en) 2012-12-17 2014-06-19 Microsoft Corporation Customer segmentation
KR20140090769A (ko) 2013-01-10 2014-07-18 한국전자통신연구원 연속질의 언어 기반의 네트워크 제어 시스템
US9098587B2 (en) 2013-01-15 2015-08-04 Oracle International Corporation Variable duration non-event pattern matching
US10298444B2 (en) 2013-01-15 2019-05-21 Oracle International Corporation Variable duration windows on continuous data streams
US10318533B2 (en) 2013-02-15 2019-06-11 Telefonaktiebolaget Lm Ericsson (Publ) Optimized query execution in a distributed data stream processing environment
US9674249B1 (en) 2013-03-11 2017-06-06 DataTorrent, Inc. Distributed streaming platform for real-time applications
US9298788B1 (en) 2013-03-11 2016-03-29 DataTorrent, Inc. Checkpointing in distributed streaming platform for real-time applications
US20140324530A1 (en) 2013-04-30 2014-10-30 Liveops, Inc. Method and system for detecting patters in data streams
US9418113B2 (en) * 2013-05-30 2016-08-16 Oracle International Corporation Value based windows on relations in continuous data streams
US10091282B2 (en) 2013-06-12 2018-10-02 Sap Se Metadata-driven dynamic load balancing in multi-tenant systems
EP3049920A1 (de) 2013-09-27 2016-08-03 Rudolf Markus Petri Verfahren und einrichtung zur automatisierten erzeugung und bereitstellung wenigstens einer softwareanwendung
US10663609B2 (en) 2013-09-30 2020-05-26 Saudi Arabian Oil Company Combining multiple geophysical attributes using extended quantization
US9313134B2 (en) 2013-10-15 2016-04-12 Cisco Technology, Inc. Leveraging hardware accelerators for scalable distributed stream processing in a network environment
EP2866144B1 (en) 2013-10-28 2020-03-25 Software AG Self-correcting complex event processing system and corresponding method for error correction
US9405854B2 (en) 2013-12-16 2016-08-02 Sybase, Inc. Event stream processing partitioning
US9697262B2 (en) 2013-12-17 2017-07-04 Microsoft Technology Licensing, Llc Analytical data processing engine
WO2015094269A1 (en) 2013-12-19 2015-06-25 Hewlett-Packard Development Company, L.P. Hybrid flows containing a continuous flow
US9817851B2 (en) * 2014-01-09 2017-11-14 Business Objects Software Ltd. Dyanmic data-driven generation and modification of input schemas for data analysis
US9791485B2 (en) 2014-03-10 2017-10-17 Silver Spring Networks, Inc. Determining electric grid topology via a zero crossing technique
US8918371B1 (en) 2014-05-27 2014-12-23 Flexera Software Llc Systems and methods for event log compensation
US9244978B2 (en) 2014-06-11 2016-01-26 Oracle International Corporation Custom partitioning of a data stream
US9712645B2 (en) 2014-06-26 2017-07-18 Oracle International Corporation Embedded event processing
US10235436B2 (en) 2014-08-29 2019-03-19 Microsoft Technology Licensing, Llc Event stream transformations
US10025802B2 (en) 2014-09-19 2018-07-17 Amazon Technologies, Inc. Automated configuration of log-coordinated storage groups
US10120907B2 (en) 2014-09-24 2018-11-06 Oracle International Corporation Scaling event processing using distributed flows and map-reduce operations
US9886486B2 (en) 2014-09-24 2018-02-06 Oracle International Corporation Enriching events with dynamically typed big data for event processing
US9613110B2 (en) 2014-12-12 2017-04-04 Sap Se Fast serialization for data transfer
US9894147B1 (en) 2014-12-23 2018-02-13 EMC IP Holding Company LLC Application plugin framework for big-data clusters
KR102054568B1 (ko) 2015-02-11 2020-01-22 아브 이니티오 테크놀로지 엘엘시 필터링 데이터 계통 다이어그램
US10095547B1 (en) 2015-03-13 2018-10-09 Twitter, Inc. Stream processing at scale
US20160306827A1 (en) 2015-04-15 2016-10-20 International Business Machines Corporation Synchronizing data rules and corresponding metadata to implement data governance
US20160328432A1 (en) 2015-05-06 2016-11-10 Squigglee LLC System and method for management of time series data sets
CA3128629A1 (en) * 2015-06-05 2016-07-28 C3.Ai, Inc. Systems and methods for data processing and enterprise ai applications
WO2017018901A1 (en) 2015-07-24 2017-02-02 Oracle International Corporation Visually exploring and analyzing event streams
US10657134B2 (en) * 2015-08-05 2020-05-19 Ab Initio Technology Llc Selecting queries for execution on a stream of real-time data
US10198298B2 (en) 2015-09-16 2019-02-05 Salesforce.Com, Inc. Handling multiple task sequences in a stream processing framework
US10303695B2 (en) 2015-10-21 2019-05-28 Oracle International Corporation Query decomposition for scalability of continuous query processing
US9672082B2 (en) 2015-10-21 2017-06-06 Oracle International Corporation Guaranteeing the event order for multi-stage processing in distributed systems
US10255141B2 (en) 2015-10-22 2019-04-09 Oracle International Corporation Event batching, output sequencing, and log based state storage in continuous query processing
US10795935B2 (en) 2016-02-05 2020-10-06 Sas Institute Inc. Automated generation of job flow definitions
US10437635B2 (en) 2016-02-10 2019-10-08 Salesforce.Com, Inc. Throttling events in entity lifecycle management
US10402256B2 (en) * 2016-05-03 2019-09-03 Gopro, Inc. Systems and methods for micro-batch processing of data
US11573965B2 (en) 2016-09-15 2023-02-07 Oracle International Corporation Data partitioning and parallelism in a distributed event processing system
US11615088B2 (en) 2016-09-15 2023-03-28 Oracle International Corporation Complex event processing for micro-batch streaming
US10891326B2 (en) 2017-01-05 2021-01-12 International Business Machines Corporation Representation of a data analysis using a flow graph
US10554577B2 (en) 2017-03-14 2020-02-04 International Business Machines Corporation Adaptive resource scheduling for data stream processing
WO2018169430A1 (en) 2017-03-17 2018-09-20 Oracle International Corporation Integrating logic in micro batch based event processing systems
WO2018169429A1 (en) 2017-03-17 2018-09-20 Oracle International Corporation Framework for the deployment of event-based applications
US10552161B2 (en) 2017-06-21 2020-02-04 International Business Machines Corporation Cluster graphical processing unit (GPU) resource sharing efficiency by directed acyclic graph (DAG) generation

Similar Documents

Publication Publication Date Title
JP2019535058A5 (enExample)
JP2019535056A5 (enExample)
Zaharia et al. Discretized streams: an efficient and {Fault-Tolerant} model for stream processing on large clusters
JP2019533854A5 (enExample)
US8381016B2 (en) Fault tolerance for map/reduce computing
CN110457397A (zh) 一种数据同步的方法及装置
WO2018236925A1 (en) Distributed quantum computing system
CN103701906B (zh) 分布式实时计算系统及其数据处理方法
CN111444158B (zh) 长短期用户画像生成方法、装置、设备及可读存储介质
JP2020027245A5 (ja) 情報処理方法、情報処理装置およびプログラム
Demirbaga et al. SmartMonit: Real-time big data monitoring system
WO2017015059A1 (en) Efficient cache warm up based on user requests
Zhelev et al. Big data processing in the cloud-Challenges and platforms
CN109376137B (zh) 一种文件处理方法及装置
Kim et al. Towards the design of a system and a workflow model for medical big data processing in the hybrid cloud
JP2017191387A (ja) データ処理プログラム、データ処理方法およびデータ処理装置
Wang et al. Turbo: Dynamic and decentralized global analytics via machine learning
Koschel et al. Efficiency experiments on Hadoop and Giraph with PageRank
Thanekar et al. A study on mapreduce: Challenges and trends
CN109587224B (zh) 数据处理方法、装置、电子设备及计算机可读介质
Cheng et al. Improved algorithms for intermediate dataset storage in a cloud-based dataflow
Acosta et al. SHEPHERD: A Shipping-Based Query Processor to Enhance SPARQL Endpoint Performance.
Che Fauzi et al. Managing fragmented database replication for Mygrants using binary vote assignment on cloud quorum
US10467079B2 (en) Information processing device, information processing method, and non-transitory computer-readable storage medium
JP6139399B2 (ja) 分散クエリ処理装置、クエリ処理方法及びクエリ処理プログラム