WO2009012403A3 - Implementation of stream algebra over class instances - Google Patents

Implementation of stream algebra over class instances Download PDF

Info

Publication number
WO2009012403A3
WO2009012403A3 PCT/US2008/070359 US2008070359W WO2009012403A3 WO 2009012403 A3 WO2009012403 A3 WO 2009012403A3 US 2008070359 W US2008070359 W US 2008070359W WO 2009012403 A3 WO2009012403 A3 WO 2009012403A3
Authority
WO
WIPO (PCT)
Prior art keywords
operators
implementation
query
events
stream
Prior art date
Application number
PCT/US2008/070359
Other languages
French (fr)
Other versions
WO2009012403A2 (en
Inventor
Gueorgui B Chkodrov
Paul F Ringseth
Tihomir T Tarnavski
Andy Shen
Roger Shane Barga
Jonathan Goldstein
Original Assignee
Microsoft Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Corp filed Critical Microsoft Corp
Publication of WO2009012403A2 publication Critical patent/WO2009012403A2/en
Publication of WO2009012403A3 publication Critical patent/WO2009012403A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24568—Data stream processing; Continuous queries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24534—Query rewriting; Transformation
    • G06F16/24542—Plan optimisation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00—Data processing: database and file management or data structures
    • Y10S707/99931—Database or file accessing
    • Y10S707/99933—Query processing, i.e. searching
    • Y10S707/99934—Query formulation, input preparation, or translation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Operations Research (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Stored Programmes (AREA)
  • Devices For Executing Special Programs (AREA)

Abstract

Creating and executing a distributed stream processing operator graph based on a query. The operator graph includes movable stream algebra operators for processing events received from high volume data streams. The operators are partially compiled and distributed to computing devices for completion of the compilation and subsequent execution. During execution, the operators maintain minimal state information associated with received events via an expiration time assigned to each of the event instances. Additional events are generated and aggregated by the operators for communication to a service responsible for the query.
PCT/US2008/070359 2007-07-18 2008-07-17 Implementation of stream algebra over class instances WO2009012403A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11/779,574 2007-07-18
US11/779,574 US7676461B2 (en) 2007-07-18 2007-07-18 Implementation of stream algebra over class instances

Publications (2)

Publication Number Publication Date
WO2009012403A2 WO2009012403A2 (en) 2009-01-22
WO2009012403A3 true WO2009012403A3 (en) 2009-03-19

Family

ID=40260384

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2008/070359 WO2009012403A2 (en) 2007-07-18 2008-07-17 Implementation of stream algebra over class instances

Country Status (2)

Country Link
US (3) US7676461B2 (en)
WO (1) WO2009012403A2 (en)

Families Citing this family (77)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7676461B2 (en) * 2007-07-18 2010-03-09 Microsoft Corporation Implementation of stream algebra over class instances
JP5104489B2 (en) * 2008-04-03 2012-12-19 日本電気株式会社 Distributed event detection system, distributed event detection method, and distributed event detection program
US7962650B2 (en) * 2008-04-10 2011-06-14 International Business Machines Corporation Dynamic component placement in an event-driven component-oriented network data processing system
US8498956B2 (en) * 2008-08-29 2013-07-30 Oracle International Corporation Techniques for matching a certain class of regular expression-based patterns in data streams
US8214408B2 (en) * 2008-09-29 2012-07-03 Teradata Us, Inc. Method, database system and computer program for joining temporal database tables
US8805875B1 (en) * 2008-10-04 2014-08-12 Reflex Systems Llc Systems and methods for information retrieval
US20100088325A1 (en) 2008-10-07 2010-04-08 Microsoft Corporation Streaming Queries
US8935293B2 (en) 2009-03-02 2015-01-13 Oracle International Corporation Framework for dynamically generating tuple and page classes
US8145859B2 (en) * 2009-03-02 2012-03-27 Oracle International Corporation Method and system for spilling from a queue to a persistent store
US8875111B2 (en) * 2009-04-23 2014-10-28 Microsoft Corporation Intermediate language representation and modification
US8387076B2 (en) * 2009-07-21 2013-02-26 Oracle International Corporation Standardized database connectivity support for an event processing server
US8321450B2 (en) * 2009-07-21 2012-11-27 Oracle International Corporation Standardized database connectivity support for an event processing server in an embedded context
US8386466B2 (en) * 2009-08-03 2013-02-26 Oracle International Corporation Log visualization tool for a data stream processing server
US8527458B2 (en) * 2009-08-03 2013-09-03 Oracle International Corporation Logging framework for a data stream processing server
US9081873B1 (en) 2009-10-05 2015-07-14 Stratacloud, Inc. Method and system for information retrieval in response to a query
US9158816B2 (en) 2009-10-21 2015-10-13 Microsoft Technology Licensing, Llc Event processing with XML query based on reusable XML query template
US9430494B2 (en) * 2009-12-28 2016-08-30 Oracle International Corporation Spatial data cartridge for event processing systems
US8959106B2 (en) * 2009-12-28 2015-02-17 Oracle International Corporation Class loading using java data cartridges
US9305057B2 (en) * 2009-12-28 2016-04-05 Oracle International Corporation Extensible indexing framework using data cartridges
US20110265069A1 (en) 2010-04-21 2011-10-27 Salesforce.Com Methods and systems for execution of tenant code in an on-demand service environment including utilization of shared resources and inline governor limit enforcement
US9552280B2 (en) 2010-05-13 2017-01-24 Microsoft Technology Licensing, Llc Visual analysis and debugging of complex event flows
US8725592B2 (en) 2010-11-18 2014-05-13 Wal-Mart Stores, Inc. Method, system, and medium for recommending gift products based on textual information of a selected user
US20110314019A1 (en) * 2010-06-18 2011-12-22 Universidad Politecnica De Madrid Parallel processing of continuous queries on data streams
US8713049B2 (en) 2010-09-17 2014-04-29 Oracle International Corporation Support for a parameterized query/view in complex event processing
US8752072B2 (en) 2010-09-29 2014-06-10 Microsoft Corporation Altering event lifetimes
US9189280B2 (en) 2010-11-18 2015-11-17 Oracle International Corporation Tracking large numbers of moving objects in an event processing system
WO2012068561A2 (en) * 2010-11-18 2012-05-24 Wal-Mart Stores, Inc. Processing data feeds
US9659063B2 (en) * 2010-12-17 2017-05-23 Software Ag Systems and/or methods for event stream deviation detection
EP2469420B1 (en) 2010-12-22 2019-11-27 Software AG CEP engine and method for processing CEP queries
US8788484B2 (en) * 2010-12-27 2014-07-22 Software Ag Systems and/or methods for user feedback driven dynamic query rewriting in complex event processing environments
US8990416B2 (en) 2011-05-06 2015-03-24 Oracle International Corporation Support for a new insert stream (ISTREAM) operation in complex event processing (CEP)
US9329975B2 (en) 2011-07-07 2016-05-03 Oracle International Corporation Continuous query language (CQL) debugger in complex event processing (CEP)
US20130125099A1 (en) * 2011-11-14 2013-05-16 Microsoft Corporation Modular compilation using partial compilers
US9563663B2 (en) 2012-09-28 2017-02-07 Oracle International Corporation Fast path evaluation of Boolean predicates
US9953059B2 (en) 2012-09-28 2018-04-24 Oracle International Corporation Generation of archiver queries for continuous queries over archived relations
US10956422B2 (en) 2012-12-05 2021-03-23 Oracle International Corporation Integrating event processing with map-reduce
US9122725B2 (en) * 2012-12-06 2015-09-01 International Business Machines Corporation Tuple routing in a streaming application
US10298444B2 (en) * 2013-01-15 2019-05-21 Oracle International Corporation Variable duration windows on continuous data streams
US9098587B2 (en) 2013-01-15 2015-08-04 Oracle International Corporation Variable duration non-event pattern matching
US9047249B2 (en) 2013-02-19 2015-06-02 Oracle International Corporation Handling faults in a continuous event processing (CEP) system
US9390135B2 (en) 2013-02-19 2016-07-12 Oracle International Corporation Executing continuous event processing (CEP) queries in parallel
US9251205B2 (en) * 2013-02-21 2016-02-02 International Business Machines Corporation Streaming delay patterns in a streaming environment
US9298788B1 (en) * 2013-03-11 2016-03-29 DataTorrent, Inc. Checkpointing in distributed streaming platform for real-time applications
US9280549B2 (en) 2013-03-14 2016-03-08 International Business Machines Corporation Compressing tuples in a streaming application
US9002846B2 (en) 2013-03-15 2015-04-07 International Business Machines Corporation Ending tuple processing in a stream-based computing application
EP2801909B1 (en) 2013-05-07 2017-07-05 Software AG Monitoring system and method for monitoring the operation of distributed computing components
US9542291B2 (en) 2013-05-24 2017-01-10 Software Ag Self-monitoring event-based system and method
US9418113B2 (en) 2013-05-30 2016-08-16 Oracle International Corporation Value based windows on relations in continuous data streams
EP2866144B1 (en) 2013-10-28 2020-03-25 Software AG Self-correcting complex event processing system and corresponding method for error correction
US9934279B2 (en) 2013-12-05 2018-04-03 Oracle International Corporation Pattern matching across multiple input data streams
US20150248461A1 (en) * 2014-02-28 2015-09-03 Alcatel Lucent Streaming query deployment optimization
US9244978B2 (en) 2014-06-11 2016-01-26 Oracle International Corporation Custom partitioning of a data stream
US9712645B2 (en) 2014-06-26 2017-07-18 Oracle International Corporation Embedded event processing
US9292267B2 (en) * 2014-06-27 2016-03-22 International Business Machines Corporation Compiling nested relational algebras with multiple intermediate representations
US10776711B2 (en) 2014-09-08 2020-09-15 Pivotal Software, Inc. Compute intensive stream processing
US10120907B2 (en) 2014-09-24 2018-11-06 Oracle International Corporation Scaling event processing using distributed flows and map-reduce operations
US9886486B2 (en) 2014-09-24 2018-02-06 Oracle International Corporation Enriching events with dynamically typed big data for event processing
US9734038B2 (en) 2014-09-30 2017-08-15 International Business Machines Corporation Path-specific break points for stream computing
US10241761B2 (en) * 2014-12-29 2019-03-26 Nvidia Corporation System and method for compiler support for compile time customization of code
US9886485B2 (en) * 2015-03-11 2018-02-06 International Business Machines Corporation Reducing redundant operations in a streaming environment
US10255336B2 (en) 2015-05-07 2019-04-09 Datometry, Inc. Method and system for transparent interoperability between applications and data management systems
WO2017018901A1 (en) 2015-07-24 2017-02-02 Oracle International Corporation Visually exploring and analyzing event streams
US10915531B2 (en) * 2015-08-14 2021-02-09 California Institute Of Technology Algebraic query language (AQL) database management system
US10594779B2 (en) 2015-08-27 2020-03-17 Datometry, Inc. Method and system for workload management for data management systems
US10067703B2 (en) 2015-12-15 2018-09-04 International Business Machines Corporation Monitoring states of processing elements
US9792259B2 (en) 2015-12-17 2017-10-17 Software Ag Systems and/or methods for interactive exploration of dependencies in streaming data
WO2017135837A1 (en) 2016-02-01 2017-08-10 Oracle International Corporation Pattern based automated test data generation
WO2017135838A1 (en) 2016-02-01 2017-08-10 Oracle International Corporation Level of detail control for geostreaming
US10983966B2 (en) * 2016-04-22 2021-04-20 International Business Machines Corporation Database algebra and compiler with environments
CN107678790B (en) * 2016-07-29 2020-05-08 华为技术有限公司 Flow calculation method, device and system
CN106445645B (en) * 2016-09-06 2019-11-26 北京百度网讯科技有限公司 Method and apparatus for executing distributed computing task
US10623281B1 (en) * 2017-04-18 2020-04-14 EMC IP Holding Company LLC Dynamically scheduled checkpoints in distributed data streaming system
US10740457B2 (en) * 2018-06-20 2020-08-11 International Business Machines Corporation System for preventing malicious operator placement in streaming applications
US11475001B1 (en) 2018-12-19 2022-10-18 Datometry, Inc. Quantifying complexity of a database query
US11294869B1 (en) 2018-12-19 2022-04-05 Datometry, Inc. Expressing complexity of migration to a database candidate
US11403282B1 (en) 2018-12-20 2022-08-02 Datometry, Inc. Unbatching database queries for migration to a different database
WO2023183990A1 (en) * 2022-03-31 2023-10-05 ZeroFlucs Labs Pty Ltd System and method for measuring outcomes from events

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020032676A1 (en) * 1994-01-31 2002-03-14 David Reiner Method and apparatus for data access in multiprocessor digital data processing systems
US6477562B2 (en) * 1998-12-16 2002-11-05 Clearwater Networks, Inc. Prioritized instruction scheduling for multi-streaming processors
US20040205110A1 (en) * 2002-09-18 2004-10-14 Netezza Corporation Asymmetric data streaming architecture having autonomous and asynchronous job processing unit
US20040260829A1 (en) * 2001-04-13 2004-12-23 Husak David J. Manipulating data streams in data stream processors

Family Cites Families (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US2028324A (en) * 1934-06-27 1936-01-21 Hugh P Griffin Shoe polish
US5999908A (en) * 1992-08-06 1999-12-07 Abelow; Daniel H. Customer-based product design module
US5549115A (en) * 1994-09-28 1996-08-27 Heartstream, Inc. Method and apparatus for gathering event data using a removable data storage medium and clock
GB9725347D0 (en) 1997-11-28 1998-01-28 Ncr Int Inc Database relationship analysis and strategy implementation tool
US6763353B2 (en) * 1998-12-07 2004-07-13 Vitria Technology, Inc. Real time business process analysis method and apparatus
US6629106B1 (en) * 1999-02-26 2003-09-30 Computing Services Support Solutions, Inc. Event monitoring and correlation system
US6496831B1 (en) * 1999-03-25 2002-12-17 Lucent Technologies Inc. Real-time event processing system for telecommunications and other applications
US6477565B1 (en) * 1999-06-01 2002-11-05 Yodlee.Com, Inc. Method and apparatus for restructuring of personalized data for transmission from a data network to connected and portable network appliances
US7020618B1 (en) * 1999-10-25 2006-03-28 Ward Richard E Method and system for customer service process management
EP1143362A2 (en) * 1999-12-14 2001-10-10 Citicorp Development Center, Inc. System and method for managing financial transaction information
US6507840B1 (en) * 1999-12-21 2003-01-14 Lucent Technologies Inc. Histogram-based approximation of set-valued query-answers
US7590644B2 (en) * 1999-12-21 2009-09-15 International Business Machine Corporation Method and apparatus of streaming data transformation using code generator and translator
US6775658B1 (en) * 1999-12-21 2004-08-10 Mci, Inc. Notification by business rule trigger control
US20040220791A1 (en) * 2000-01-03 2004-11-04 Interactual Technologies, Inc. A California Corpor Personalization services for entities from multiple sources
EP1122901A3 (en) * 2000-02-03 2006-07-12 Matsushita Electric Industrial Co., Ltd. Digital broadcasting system and event message transmission method
US6941557B1 (en) * 2000-05-23 2005-09-06 Verizon Laboratories Inc. System and method for providing a global real-time advanced correlation environment architecture
US20020062237A1 (en) * 2000-08-04 2002-05-23 Scott Matsumoto Transactional monitoring system and method
US7111010B2 (en) * 2000-09-25 2006-09-19 Hon Hai Precision Industry, Ltd. Method and system for managing event attributes
US7103556B2 (en) * 2000-11-02 2006-09-05 Jpmorgan Chase Bank, N.A. System and method for aggregate portfolio client support
US7107224B1 (en) * 2000-11-03 2006-09-12 Mydecide, Inc. Value driven integrated build-to-buy decision analysis system and method
US20020099578A1 (en) * 2001-01-22 2002-07-25 Eicher Daryl E. Performance-based supply chain management system and method with automatic alert threshold determination
US6954757B2 (en) * 2001-02-02 2005-10-11 Hewlett-Packard Development Company, L.P. Framework, architecture, method and system for reducing latency of business operations of an enterprise
US7013289B2 (en) 2001-02-21 2006-03-14 Michel Horn Global electronic commerce system
US7065566B2 (en) * 2001-03-30 2006-06-20 Tonic Software, Inc. System and method for business systems transactions and infrastructure management
US7865867B2 (en) * 2002-03-08 2011-01-04 Agile Software Corporation System and method for managing and monitoring multiple workflows
JP4047053B2 (en) 2002-04-16 2008-02-13 富士通株式会社 Retrieval apparatus and method using sequence pattern including repetition
US7610211B2 (en) * 2002-06-21 2009-10-27 Hewlett-Packard Development Company, L.P. Investigating business processes
US7437675B2 (en) * 2003-02-03 2008-10-14 Hewlett-Packard Development Company, L.P. System and method for monitoring event based systems
JP2004240766A (en) 2003-02-06 2004-08-26 Toshiba Corp System and method for generating pattern detection processing program
CA2418568C (en) * 2003-02-10 2011-10-11 Watchfire Corporation Method and system for classifying content and prioritizing web site content issues
US7487148B2 (en) * 2003-02-28 2009-02-03 Eaton Corporation System and method for analyzing data
US7693810B2 (en) * 2003-03-04 2010-04-06 Mantas, Inc. Method and system for advanced scenario based alert generation and processing
US7383255B2 (en) * 2003-06-23 2008-06-03 Microsoft Corporation Common query runtime system and application programming interface
US20070043856A1 (en) * 2005-02-07 2007-02-22 Metavize, Inc. Methods and systems for low-latency event pipelining
US20060253831A1 (en) 2005-05-03 2006-11-09 Microsoft Corporation Validation architecture
US7627544B2 (en) * 2005-05-20 2009-12-01 Microsoft Corporation Recognizing event patterns from event streams
US7512829B2 (en) * 2005-06-09 2009-03-31 Microsoft Corporation Real time event stream processor to ensure up-to-date and accurate result
US7747610B2 (en) * 2006-11-10 2010-06-29 Sybase, Inc. Database system and methodology for processing path based queries
US20080120283A1 (en) * 2006-11-17 2008-05-22 Oracle International Corporation Processing XML data stream(s) using continuous queries in a data stream management system
WO2008148130A2 (en) * 2007-05-31 2008-12-04 Agent Logic, Inc. Distributed system for monitoring information events
US7676461B2 (en) * 2007-07-18 2010-03-09 Microsoft Corporation Implementation of stream algebra over class instances
US7996388B2 (en) * 2007-10-17 2011-08-09 Oracle International Corporation Adding new continuous queries to a data stream management system operating on existing queries
US8447859B2 (en) 2007-12-28 2013-05-21 International Business Machines Corporation Adaptive business resiliency computer system for information technology environments
KR101130734B1 (en) * 2010-08-12 2012-03-28 연세대학교 산학협력단 Method for generating context hierachyand, system for generating context hierachyand

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020032676A1 (en) * 1994-01-31 2002-03-14 David Reiner Method and apparatus for data access in multiprocessor digital data processing systems
US6477562B2 (en) * 1998-12-16 2002-11-05 Clearwater Networks, Inc. Prioritized instruction scheduling for multi-streaming processors
US20040260829A1 (en) * 2001-04-13 2004-12-23 Husak David J. Manipulating data streams in data stream processors
US20040205110A1 (en) * 2002-09-18 2004-10-14 Netezza Corporation Asymmetric data streaming architecture having autonomous and asynchronous job processing unit

Also Published As

Publication number Publication date
US8775482B2 (en) 2014-07-08
US20090024622A1 (en) 2009-01-22
US20130014094A1 (en) 2013-01-10
WO2009012403A2 (en) 2009-01-22
US8296331B2 (en) 2012-10-23
US7676461B2 (en) 2010-03-09
US20100131543A1 (en) 2010-05-27

Similar Documents

Publication Publication Date Title
WO2009012403A3 (en) Implementation of stream algebra over class instances
WO2006138403A3 (en) Packet processor and filter apparatus and methods
WO2007075638A3 (en) System and method for monitoring system performance levels across a network
WO2008150986A3 (en) Dynamically updating software applications on a device
WO2012103143A3 (en) Method and apparatus for compiling regular expressions
WO2011109252A3 (en) Compressing source code written in a scripting language
WO2009076095A3 (en) Delivery of streams to repair errored media streams in periods of insufficient resources
WO2008008725A3 (en) System and method for generating use statistics for location-based applications
WO2014022743A3 (en) Mobile build, quality and deployment manager
WO2008013738A3 (en) Virtual user authentication system and method
TW200943849A (en) Off-line task list architecture
WO2009073156A3 (en) Enhanced data exchange and functionality control system and method
WO2009002751A3 (en) Running add-on components in virtual environments
WO2007135555A3 (en) Method and system for providing a graphical workbench environment with intelligent plug-ins
TW200731133A (en) Firmware filters and patches
WO2011041172A3 (en) A rules-based fan interaction engine, system and method
WO2008101022A3 (en) Methods and apparatus to reach through to business logic services
TW200703080A (en) Method for detecting hand gesture on touch pad
WO2013087038A8 (en) System and method for communicating using short-header frames
WO2013167092A3 (en) Method, device and system for remote management of terminal peripheral
WO2009148769A3 (en) Virtual media device
WO2010100535A3 (en) System and method for junket business
WO2010014144A3 (en) Mobile banking with short message service
GB0413060D0 (en) A method for selecting plug-in code modules in a computing device
WO2007075479A3 (en) Service provisioning

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08796259

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 08796259

Country of ref document: EP

Kind code of ref document: A2