KR101014492B1 - Continuous query language based continuous query processor and method for relational data model based streaming data - Google Patents

Abstract

The present invention relates to a continuum language-based continuum processor and method for processing relational data model-based streaming data, and provides a new grammar element for processing queries for streaming data in SQL, a query language standard for relational data models. By adding the standard SQL query language, which is designed to extend the standard SQL query language to process streaming data based on the relational data model while maintaining the functions and features of the standard SQL query language, It has the advantage of being able to query and process streaming data while using SQL query language invariably.

Sequential, sequential processor, streaming data, SQL

Description

Continuous-language-based continuity processor and method for processing relational data model-based streaming data {CONTINUOUS QUERY LANGUAGE BASED CONTINUOUS QUERY PROCESSOR AND METHOD FOR RELATIONAL DATA MODEL BASED STREAMING DATA}

The present invention relates to a continuous-language processor and method of a continuous language, and more particularly, to a relational data model based on a relational data model based on a relational data model. The present invention relates to a continuous language-based continuous processor and method for processing streaming data.

Structured Query Language (SQL) is a computer language designed for data retrieval and management, database schema creation and modification, and database object access control management in relational database management systems. SQL is a standard interactive programming language for getting and updating information from a database. Many database programs adopt SQL as a standard.

In today's Internet environment, applications that require streaming data processing functions are increasing, and interest in them is also increasing. In general, streaming data is continuously generated continuously, and the amount of data changes over time. Occurs. Examples of such streaming data include data such as network traffic, sensors, RFID, call logs, finance, web logs, online games, and the like.

Common requirements for streaming data processing include filtering, aggregation, (de) multiplexing, frequency operation, join, and window operations on streaming data. Most existing streaming data processing applications We have developed features to meet the above requirements. However, in recent years, as data producers diversify, and data production volume and speed also increase, it has been recognized that it is difficult to meet the requirements for rapidly changing development environments with existing development methods. In addition to these recent perceptions, studies are being actively conducted to solve these problems in academia and industry.

Among systems studies that process streaming data in a standardized way, STREAM [Ref. A. Arasu, S. Babu, J. Widom. CQL: A Language for Continuous Queries over Streams and Relations. LNCS2921], Telegraph CQ [Reference: Samuel Madden et al. Continuously Adaptive Continuous Queries over Streams, ACM SIGMOD 2002]. They are characterized in that the query for streaming data is SQL, a relational database query language.

The SQL query language is a representative standard query language for relational database queries and is used by users and systems. Providing a continuous query function for streaming data using the standard and popularized SQL query language has a great advantage in that it provides user-friendly query methods and system compatibility. The systems in the previous example offer the advantage that they are basically based on the syntax of the SQL query language. However, all of the example systems use a method of altering, extending, or restricting the syntax of SQL to extend it to a continuum language. Changes in the grammatical structure of the SQL query language place a burden on the user to learn a new grammar and cause incompatibility with the existing query language processing system. Therefore, if possible, a continuous query for streaming data based on relational data model should be provided without changing the basic structure of SQL syntax, which is a standard query language.

Continuous query is a query processing method for streaming data that is continuously and continuously input. The continuous query can be compared based on the difference between the query processing method of a conventional relational database and the query processing method of a continuous system. The most distinctive difference between the query processing methods is that a continuum processor is one compared to a traditional relational database query processor performing one (1) query processing for one (1) query request and then returning the query result. As long as the query request of (1) is not canceled, the query processing is executed continuously, and the result is continuously produced. In addition, the query target data set of the traditional relational database query processor becomes a limited set at the time of query processing, whereas the query target data set of the continuous query processor changes continuously during query execution. In addition, in the method of returning a query result, the traditional relational database query processor returns a result only when a request for a query result is made, whereas a continuous query processor automatically returns a result whenever a query result is produced. .

The current standard SQL query language does not have the ability to define a continuous query for streaming data. Therefore, in order to provide a continuous query function for streaming data based on relational data model, it is necessary to expand the grammar structure that can provide continuous query function without affecting the basic grammar structure of standard SQL query language. .

Accordingly, the present invention is to solve the problem of lack of continuity of the streaming data of the standard SQL query language, search query for the relational data model-based streaming data without modifying the syntax structure of the standard SQL query language The present invention is to provide a continuous processor and method for performing continuous query based on the continuous language.

Continuous-language-based continuous-processing processing system and method for processing relational data model-based streaming data according to the present invention is a query language and processing method for a relational data model-based streaming data continuously and continuously inputted with different amounts of data over time. By providing, it provides a function to collectively search the streaming data based on the relational data model. In addition, the search function for streaming data can be further extended without modifying the standard SQL syntax, so that the functions of the existing standard SQL can be used without modification.

Continuous language-based continuous processing system for processing relational data model-based streaming data of the present invention for achieving the above object comprises an input queue for receiving the relational data model-based streaming data; A window for representing the input queue as a data set for a continuous query; A continuous query processor configured to receive a continuous query and construct a continuous execution environment, and execute the query continuously; And an output queue for outputting the continuous data produced by the continuous processor as relational data model based streaming data.

In this case, the processor of the continuous query receives a continuous query, parses the syntax, checks whether the continuous query is written correctly according to the language syntax of the continuous query, and, in the case of a correctly prepared continuous query, constructs a parse tree of the continuous query. module; A module for verifying whether a window definition is correct using a window definition syntax of the serial parsing tree, and generating a window for an input queue; A module for verifying whether the search definition is correct by using the definition syntax of the search query of the parsing tree of the continuous query, and generating an output queue for the search result; When the window creation and the output queue creation are correct, the module includes a window, an output queue, and a continuous query.

In addition, the method of processing a continuous language-based continuous query for processing relational data model-based streaming data of the present invention comprises the steps of: (a) inputting the continuous query to the processor of the continuous query through the query registration user interface; (b) performing a syntax analysis on the input continuous query by the continuous language processor; (c) generating a window using the parsed syntax; (c) creating an output queue using the parsed syntax; (d) registering the created window, output queue, and continuous query to configure a continuous execution environment; And (e) continuously executing the query for the streaming data and generating a result.

Hereinafter, a processor and a method of a continuous-language-based continuous query for processing relational data model-based streaming data will be described in detail with reference to the accompanying drawings.

1 is a part (110 ~ 200) of the continuous language syntax rules to extend the existing SQL query language to perform a continuous query on the streaming data based on the relational data model.

In the standard SQL query language, a window clause (120) has been added to define a window for a streaming data set based on a relational data model. The window clause can define a name for an input queue, which is where streaming data is input. The window expression clause 130 defining the queue name, the window name, and the window property can be defined. In the window expression clause, the queue name can be defined as the unique ID value of the queue in the queue name clause 140, and the window name can be defined as the window unique ID value in the window name clause 150. In the window expression clause, the window property allows the size and movement of the window to be defined in the window property clause 160 as a time value or a count value, and the window time values in the window property clause are days and hours. ), Minutes (Min), and seconds (Sec) values can be defined in the window time value clause 170, and the window count value of the window property clause can be defined in the window count value clause 180 as a positive integer. It was made. In order to use the unique property value required for the query, a unique property use clause 190 has been added so that it can be defined by using property keyword and unique ID value. The SQL query clause 200 can be defined as a standard SQL statement, which allows to use the functions of the standard SQL statement invariably.

FIG. 2 shows examples 210 to 240 using the continuous language syntax of FIG.

One embodiment 210 creates a window win1 having a time size value of 5 seconds and a moving value of 5 seconds for an input queue queue1 in which a relational data model based streaming data set is stored, and the total of streaming data sets from the window win1. Example of retrieving the number of elements. Another embodiment 220 creates a window win1 with a time size value of 5 minutes and a move value of 1 minute for input queue queue1 in which a relational data model based streaming data set is stored, and 1 minute for the same input queue queue1. In this example, win2 having a time size value of 5 and a moving value of 5 minutes is generated, and the total number of elements of the streaming data set is retrieved from each window. Another embodiment 230 creates a window win1 having a size of 10 streaming data elements and a moving value of 10 streaming data elements for an input queue queue1 in which a relational data model based streaming data set is stored, and from the window win1. This example retrieves the total number of elements in the streaming data set. Another embodiment 240 creates a window win1 having 10 streaming data element size values and 5 streaming data element moving values for an input queue queue1 in which a relational data model based streaming data set is stored, and the same input queue queue1. For example, a window win2 having a size of five streaming data elements and a moving value of ten streaming data elements is generated, and the total number of elements of the streaming data set is retrieved from each window.

Figure 3 illustrates an example of a processor of the present invention, a language-based continuum.

As shown in FIG. 3, the continuous-language-based continuous processing system of the present invention includes a user interface 310, an input queue application 320, an input queue 330, a window 340, and a continuous processor 350. ), An output queue 360, and an output queue application program 370.

The user interface 310 is responsible for providing system usage functions such as input queue creation / deletion / search, registration / deletion / search of continuous query, and result queue search to the user of the continuous processing system. The input queue application 320 is responsible for producing relational data model based streaming data and inputting it to the input queue 330. The input queue 330 stores the relational data model-based streaming data received from the input queue application 320. The continuous query processor 350 analyzes and verifies the continuous query received through the user interface 310 to generate a window 340 associated with the input queue 330 and to generate an output queue 360 for the search result of the continuous query. It also plays a role of delivering a query result to the output queue 360 by continuously performing a query according to the execution plan of the continuous query. The window 340 serves to manage the size value and the moving value of the time or number period for the continuous query for the input queue 330 by using. The output queue 360 is responsible for storing the continuous streaming resultant data produced from the continuous processing processor 350. The output queue application 370 receives the continuous streaming result data from the output queue 360 and is responsible for using the result.

4 is a flowchart illustrating a process of registering the continuous query by the processor 350 of the continuous query.

The continuous query is input through the user interface 310 (410). Parsing of the input continuous query is performed, and if it is not correct syntax, the registration process is terminated (420). If the syntax of the input continuous query is correct, the syntax tree of the continuous query is generated (430). The syntax of the window definition is verified in the syntax tree of the continuous query. If the syntax is not correct, the registration process is terminated (440). If the window definition syntax is correct, the window is created (441). If the search query is correct in the syntax tree of the continuous query, the registration process is terminated if it is not the correct search query (450). If the correct search query, an output queue for the search result is created (451). It is verified whether both the window and the output queue for the continuous query are correctly created, otherwise, the registration process is terminated (460). Registration of the correctly created window, output queue, and continuous query is performed (470).

5 is a flowchart illustrating a process of performing the continuous query by the processor 350 of the continuous query.

The input queue application program 320 produces streaming data and inputs it to the input queue 330 (S501). Streaming data input (S501) process is repeatedly performed until the end of the system.

The continuous language processor 350 performs window sliding (S502) with respect to the input queue 330 using the size value and the movement value of the window 340, and inputs the input based on the time value and the movement value of the sliding window. After querying the queue 330 (S503), if there is a query result, the query result is returned to the output queue 360 (S504). The query process (S502 ~ S504) is repeated until the continuous query is deleted.

The output queue application program 370 receives the continuous streaming result data from the output queue 360 (S505). This process (S505) is repeated until the continuous query is deleted.

Although the present invention has been described in more detail with reference to some embodiments, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention.

1 is a continuous language syntax rule for streaming data retrieval based on a relational data model according to the present invention.

2 illustrates an embodiment of a continuum language construct of the present invention.

3 shows an example of a streaming data processing system including a continuous processor of the present invention.

4 is a flowchart illustrating a process of registering a continuous query by the continuous processor of the present invention.

5 is a flowchart of a process of performing a continuous query by the continuous processor of the present invention.

<Explanation of symbols for the main parts of the drawings>

310: user interface

320: input queue application

330: input queue

340: Windows

350: continuous processor

360: output queue

370: Output queue application

410: input of continuous quality

420: parsing

430: sequence syntax tree

440: Windows Verification

441: Create Window

450: Validation of search quality

451: Create output queue

460: Verification of continuous quality

470: Registering windows / output queues / continuities

S501: Streaming data input

S502: sliding window

S503: Query Execution

S504: Return query results

S505: streaming data output

Claims (7)

An input queue for independently storing and managing streaming data based on a relational data model transmitted from an application for the input queue; An output queue for independently storing and managing streaming data based on a relational data model that is a result of a continuous query; A window for storing and managing the size and movement information of the sequential subset relative to the streaming data set stored in the input queue based on a time value or a number value; And A continuous query processor that performs a search query on the streaming data set stored in the input queue based on the window information, and repeatedly stores a result in the output queue; Continuous-language-based continuous processing system for the relational data model-based streaming data processing comprising a. The system of claim 1, wherein the continuous treatment system is A user interface for registering, deleting, and managing change (input queue, output queue, window, serial query); Continuous-language-based continuous processing system for processing relational data model-based streaming data, further comprising a. The process of claim 1, wherein the continuous processor is a. receiving a continuous query through a user interface; b. performing and verifying parsing on the continuous query entered in the step; c. generating a syntax tree of the continuum if the syntax of the continuum verified in the above step is correct; d. verifying that the window definition syntax is correct in the syntax tree of the continuous query generated in the step; e. generating a window by the window definition syntax when the window definition syntax verified in the above step is correct; f. verifying that the search query is correct in the syntax tree of the continuous query generated in the step; g. generating an output queue for the search query result if the search query verified in the step is correct; h. verifying that the window and output queue for the continuous query have been correctly created in the step; And i. performing registration of the window, the output queue, and the input continuous query generated in the step; Continuous-language-based continuous processing system for the relational data model-based streaming data processing comprising a. The method of claim 3, wherein the step e, Determining whether the window definition criterion is a time value or a count value; Extracting a window size value and a window movement value defined in units of days, hours, minutes, and seconds when the window definition criterion determined in the step is a time value; Extracting a window size value and a movement value defined in positive integer units when the window definition criterion determined in the step is a number value; Comparing the window size value and the window shift value extracted in the step; Defining a property of a window as a general window when the window size value and the window movement value are the same as a result of the comparison in the above step; Defining a property of a window as an overlapping window when the window size value is greater than the window moving value as a result of the comparison in the above step; And Defining a property of a window as a jump window when the window size value is smaller than the window moving value as a result of the comparison in the above step; Continuous-language-based continuous processing system for the relational data model-based streaming data processing comprising a. a. the continuous processor moving the window for the input queue based on the size value and the moving value of the window; b. performing a query on the input queue based on the window moved in the step; c. inputting the result of the query performed in the step to an output queue; d. asynchronously delivering the results of the query entered in the output queue to the output queue application program; And sequentially repeating steps (a, b, c, d); Continuous-language-based continuous processing system for the relational data model-based streaming data processing comprising a. 4. The method of claim 3, wherein the language syntax rules for the continuous query are: In the standard SQL query language Continuous-language-based continuity processing for relational data model-based streaming data processing, further comprising a window clause that defines the input queue name where the streaming data is stored, and the size and movement values of the window as time or number. system. The method of claim 6, wherein the language syntax rules for the continuous query, Relational data model that includes a time clause representing the size and slide values of time in days, hours, minutes, and seconds in the window clause. Continuous-language-based continuous-processing system for processing streaming data.

Images