KR20130071773A - System and method of processing realtime data based on query language - Google Patents

Abstract

PURPOSE: A query language based real-time data processing apparatus and a processing method thereof are provided to perform dynamic data inputted in real-time dynamically and to perform data processing for a query with restricted items effectively. CONSTITUTION: A data management part (110) receives and stores data from a data gathering device. A query receiving part (120) receives a query from a user terminal, and an analysis part (130) extracts response from the data by analyzing the query. If the query is a query having a condition, a push processing part (150) stores the condition and receives response appropriate for the condition from the analysis part and provides the condition in a push type to the user terminal. A conversion part (140) converts the response to the type which is requested by the user terminal. The conversion part converts the response into Java script object notation (JSON) or extensible markup language (XML). [Reference numerals] (110) Data management part; (120) Query receiving part; (130) Analysis part; (140) Conversion part; (150) Push processing part

Description

Query language-based real-time data processing device and processing method {SYSTEM AND METHOD OF PROCESSING REALTIME DATA BASED ON QUERY LANGUAGE}

The present invention relates to a query language based real-time data processing apparatus and processing method.

Developers using a user terminal in a service that selects and outputs a result of a user terminal query from a resource use a query language for a database. In general, a structured query language, SQL is often used. When using SQL, various query languages are used to extract and filter data from distributed repositories due to the difficulty of resource expansion and high resource cost due to the static database structure. Such query languages include Yahoo Query Language (YQL) and Facebook Query Language (FQL).

However, these query languages are also designed for the processing of static data, so it is not easy to process the data in real time and the processing of the conditional query.

The technical problem to be achieved by the present invention is to efficiently process dynamic data flowing in real time, and to effectively perform data processing for a conditional query.

A data processing device according to an embodiment of the present invention includes a data management unit for receiving and storing data from a data collection device, a query receiving unit for receiving a query from a user terminal, and analyzing the query to extract a response from the data. An analysis unit, and a push processing unit for storing the condition when the query is a query having a condition, and receives a response corresponding to the condition from the analysis unit to provide to the user terminal in the form of a push.

The apparatus may further include a converter configured to convert the response into a form requested by the user terminal.

The conversion unit may convert the response into a JavaScript object notation (JSON) or an extensible markup language (XML).

The data may include any one of data and sensor data flowing in real time.

The data manager may store the data in a structure including a data source description, data resource metadata, and data source property.

A query language corresponding to the query may follow a grammar system of a structured query language (SQL).

The query language may be in the form of supporting at least one of a show data source, a describe data source, and a select statement.

A data processing method according to another embodiment of the present invention is a data processing method in which a data processing device provides data from a data collection device to a user terminal, receiving and storing data from the data collection device, querying from the user terminal. receiving a query, and analyzing the query to extract a response from the data, wherein extracting a response from the data comprises: storing the condition if the query is a query having a condition; And extracting a response meeting the condition.

The method may further include providing the response to the user terminal in a push form.

The method may further include converting the response into a form requested by the user terminal.

The data may include any one of data and sensor data flowing in real time.

Receiving and storing the data further includes storing the data in a structure including a data source description, data source metadata, and data source property. It may include.

A query language corresponding to the query follows a grammar system of a structured query language (SQL) and includes a show data source, a describe data source, and a select. It may be in the form of supporting at least one of the statements.

According to another embodiment of the present invention, a recording medium receives and stores data from a data collection device, receives a query from a user terminal, and stores the condition if the query is a query having a condition. And extracting a response meeting the condition, and receiving the response meeting the condition and providing the user terminal in the form of a push. .

The method may further include converting the response into a form requested by the user terminal.

The data may include any one of data and sensor data flowing in real time.

According to an embodiment of the present invention, it is possible to efficiently process dynamic data flowing in real time, and to effectively perform data processing for a query having limitations.

1 is a block diagram of an environment in which a data processing apparatus according to an embodiment of the present invention operates.
2 is a block diagram of a data processing apparatus according to an embodiment of the present invention.
3 is a diagram illustrating a structure of a data management unit of a data processing apparatus according to an embodiment of the present invention.
4 is a diagram illustrating a query language processed by a data processing apparatus according to an exemplary embodiment of the present invention.
5 is a flowchart of a data processing method according to an embodiment of the present invention.
6 is a block diagram of a data processing apparatus according to another embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding other components unless specifically stated otherwise. Also, the terms " part, "" module," and " module ", etc. in the specification mean a unit for processing at least one function or operation and may be implemented by hardware or software or a combination of hardware and software have.

A data processing apparatus and a data processing method according to an embodiment of the present invention will be described in detail with reference to the drawings.

1 is a block diagram of an environment in which a data processing apparatus according to an embodiment of the present invention operates.

Referring to FIG. 1, the data processing apparatus 100 receives and stores data from the data collection apparatus 200, extracts a response according to a query of the user terminal 300, and extracts a response from the stored data. A server provided to 300 is an open application program interface (API) device.

The data collection device 200 collects data that is a resource of a response to a query of the user terminal 300. The data may be data flowing in real time. In addition, the data collection device 200 may be a device for collecting sensor data, and for example, as data on the weather, collects data on temperature, humidity, wind speed, rainfall, snowfall, and weather for a specific region at regular intervals. can do. The data collection device 200 is connected to the data processing device 100 through a network.

The user terminal 300 is a device in which a user connects to and communicates with the data processing apparatus 100 through a network. For example, the user terminal 300 may be a computer, a personal digital assistant (PDA), a mobile communication terminal, and a television. Various communication devices such as TV) can be used.

A data processing apparatus according to an embodiment of the present invention will now be described in detail with reference to FIGS. 2 to 4.

2 is a block diagram of a data processing apparatus according to an embodiment of the present invention, and FIG. 3 is a diagram illustrating a structure of a data management unit of the data processing apparatus according to an embodiment of the present invention, and FIG. FIG. 7 is a diagram illustrating a query language processed by a data processing apparatus, according to an exemplary embodiment. Referring to FIG.

Referring to FIG. 2, the data processing apparatus 100 may include a data manager 110, a query receiver 120, an analyzer 130, a converter 140, and a push processor 150.

The data manager 110 collects data from the data collection device 200 and stores and manages the data in a storage unit (not shown) so as to be suitable for data processing. The data collection device 200 may be a device for collecting various types of sensor data, and the collected data may have various formats according to the characteristics of the sensor. Therefore, the form of data supported by the data processing apparatus 100 may not be limited, and the data manager 110 may add, delete, and modify the form of data that may be supported.

The data manager 110 may store the data structure as shown in FIG. 3. Referring to FIG. 3, the data structure includes a data source description, data source metadata, and data source property. The data resource description includes the name of the data resource, an accessible address (url) and a user identifier (ID) and password required for data processing. Data resource metadata includes natural language-based descriptions and ownership. Data resource attributes include the name and type of the attribute that the data resource supports.

The query receiving unit 120 receives a query from the user terminal 300 and transmits the query to the analysis unit 130. The query of the user terminal 300 includes a general query and a condition query. A condition query is a query that includes a condition of a response to the query. For example, the condition query may be "notify me when the temperature of Hannam-dong rises 3 degrees".

The analyzing unit 130 analyzes the query received by the query receiving unit 120 and maps the corresponding response from the data stored in the storage unit. In this case, the query of the user terminal 300 may be expressed according to a specific query language, and the query language is as shown in FIG. 4.

Referring to FIG. 4, the query language follows a grammar system of a structured query language (SQL) and may support three kinds of sentences. The first is a show data source, which visualizes the types of data tables supported by the data processing apparatus 100. That is, the result obtained through the data manager 110 may be visualized to the user terminal 300. The second is a describe data source, which can obtain some kind and form of information from a specific data resource through the data management unit 110 for metadata about a specific data resource recognized through the show data resource. Provide information about the presence. The third is a select statement, which is designed to obtain data of the data collection device 200 satisfying a given condition from a specific data resource. It is intended to specify the target data resource similar to an SQL statement and to extract data that meets the condition below the where clause based on the specified property. The where clause can be expressed by combining certain conditions based on comparison operators, logical operators, and parentheses. A string can be compared with double quotes.

If the query received by the query receiver 120 is a conditional query, the interpreter 130 extracts a response corresponding to the condition. For example, if the condition is a duration, the corresponding time period is specified to extract the result set.

The conversion unit 140 converts the response selected by the analysis unit 130 into a data format requested by the user terminal 300. The format converted by the conversion unit 140 may be, for example, a JavaScript object notation (JSON) or an extensible markup language (XML).

If the query received by the query receiving unit 120 is a condition query, the push processing unit 150 stores the corresponding condition and transmits the stored condition to the analyzing unit 130 so as to meet the condition received by the analyzing unit 130. Let's extract the response. In addition, the push processing unit 150 receives the response extracted by the analysis unit 130 to meet the conditions and delivers it to the user terminal 300 in a push form.

As described above, according to the present invention, the dynamic data flowing in real time can be efficiently processed, and the data processing can be effectively performed by appropriately extracting the response corresponding to the condition and providing it as a push for the conditional query. .

A data processing method according to another embodiment of the present invention will now be described in detail with reference to FIG. 5.

5 is a flowchart of a data processing method according to another embodiment of the present invention.

Referring to FIG. 5, the data collection device 200 collects data and transmits the data to the data processing device 100 (S510). In this case, the data collected by the data collection device 200 may be data or sensor data flowing in real time.

The data processing apparatus 100 stores the received data (S520). In this case, as illustrated in FIG. 3, the data may be stored in a data structure including a data source description, data resource metadata, and data source property.

Subsequently, the user terminal 300 transmits a query to the data processing apparatus 100 (S530). In this case, the query includes a general query and a condition query, and the query may be received in a format as shown in FIG. 4.

The data processing apparatus 100 analyzes a query received from the user terminal 300 (S540). Subsequently, the data processing apparatus 100 extracts a response to the analyzed query from the stored data (S550).

The data processing apparatus 100 converts the extracted response into a format requested by the user terminal 300, for example, a JSON format or an XML format (S560), and provides the same to the user terminal 300 (S570). In this case, when the query of the user terminal 300 is a condition query, the data processing apparatus 100 stores the condition, extracts a response according to the condition, and provides the user terminal 300 in a push form.

The data processing method according to the present invention can be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The medium may be a transmission medium such as an optical or metal line, a wave guide, or the like, including a carrier wave for transmitting a signal designating a program command, a data structure, or the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

In addition, at least some of the functions of the data processing apparatus and the data processing method according to the embodiment of the present invention described above may be implemented in hardware or software coupled to the hardware. Hereinafter, an embodiment in which each of a data processing apparatus and a data processing method are coupled to a computer system will be described in detail with reference to FIG. 6.

FIG. 6 is a schematic diagram of a data processing apparatus 1000 according to another exemplary embodiment of the present invention, wherein the data managing unit 110, the query receiving unit 120, the analyzing unit 130, the converting unit 140, and the push unit are described above. Represents a system that can be used to perform at least some of the functions of processor 150

Referring to FIG. 6, the data processing apparatus 1000 may include a processor 1110, a memory 1120, at least a storage device 1130, an input / output (I / O) interface 1140, and a network interface 1150. It includes.

The processor 1110 may be implemented with a central processing unit (CPU), other chipsets, microprocessors, or the like. The memory 1120 may be a medium such as RAM, such as dynamic random access memory (DRAM), rambus DRAM (RDRAM), synchronous DRAM (SDRAM), static RAM (SRAM), or the like. It can be implemented as. The storage device 1130 may include a hard disk, a compact disk read only memory (CD-ROM), a CD rewritable (CD-RW), a digital video disk ROM (DVD-ROM), a DVD-RAM, and a DVD-RW disk. The optical disk may be implemented as a permanent or volatile storage device such as an optical disk such as a blu-ray disk, a flash memory, or various types of RAM. The I / O interface 1140 also allows the processor 1110 and / or the memory 1120 to access the storage device 1130. The network interface 1150 allows the processor 1110 and / or the memory 1120 to access the network.

In this case, the processor 1110 may store a program command for implementing at least some of the functions of the data manager 110, the query receiver 120, the interpreter 130, the converter 140, and the push processor 150. 1120, the functions of the data management unit 110, the query receiving unit 120, the analysis unit 130, the conversion unit 140, and the push processing unit 150 are located in the storage device 1130, and FIG. 1. 5 may be controlled to perform the operation described with reference to FIG. 5.

The processor 1110, the memory 1120, the storage device 1130, the I / O interface 1140, and the network interface 1150 illustrated in FIG. 6 may be implemented in one computer, a mobile communication terminal, or the like. It may be implemented by being distributed to a computer or a mobile communication terminal.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

Claims (16)

A data management unit for receiving and storing data from a data collection device;
A query receiving unit for receiving a query from a user terminal,
An analysis unit for analyzing the query and extracting a response from the data; and
If the query is a query having a condition, the push processing unit stores the condition, receives a response that meets the condition from the analysis unit and provides it to the user terminal in the form of a push
To the data processing apparatus. In claim 1,
A conversion unit converting the response into a form requested by the user terminal
Data processing device further comprising. 3. The method of claim 2,
And the conversion unit converts the response into a JavaScript object notation (JSON) or an extensible markup language (XML). In claim 1,
The data includes any one of data and sensor data flowing in real time. In claim 1,
The data management unit,
And a data processing device storing the data in a structure including a data source description, data source metadata, and data source properties. In claim 1,
The query language corresponding to the query is a data processing apparatus that conforms to a grammar system of a structured query language (SQL). The method of claim 6,
The query language is
A data processing apparatus in a form that supports at least one of a show data source, a describe data source, and a select statement. A data processing method in which a data processing device provides data from a data collection device to a user terminal
Receiving and storing data from the data collection device,
Receiving a query from the user terminal, and
Analyzing the query and extracting a response from the data
Lt; / RTI >
Extracting a response from the data,
Storing the condition if the query is a query having a condition,
Extracting a response meeting the condition
/ RTI > 9. The method of claim 8,
Providing the response to the user terminal in a push form
Data processing method further comprising. 9. The method of claim 8,
Converting the response into a form requested by the user terminal
Data processing method further comprising. 9. The method of claim 8,
The data includes any one of data and sensor data flowing in real time. 9. The method of claim 8,
Receiving and storing the data,
And storing the data in a structure including a data source description, data source metadata, and data source properties. 9. The method of claim 8,
The query language corresponding to the query is
Follows the grammar scheme of structured query language (SQL),
A data processing method in the form of supporting at least one of a show data source, a describe data source, and a select statement. Receiving and storing data from the data collection device,
Receiving a query from a user terminal,
Storing the condition if the query is a query having a condition,
Extracting a response meeting the condition, and
Receiving a response meeting the condition and providing the response to the user terminal in a push form;
Computer-readable recording medium for recording a program for executing a method on a computer comprising a. The method of claim 14,
Converting the response into a form requested by the user terminal
Computer-readable recording medium for recording a program for executing the method on a computer further comprising. The method of claim 14,
And record the program for executing the method on the computer, wherein the data includes any one of data and sensor data flowing in real time.

Images