KR20170136976A - Method and apparatus for real-time big data processing and distribution based on data specifications - Google Patents

Abstract

A real-time big data processing and distribution method and apparatus based on data specification are provided. A big data providing apparatus can provide the real time processing and distribution of big data based on data specification. The big data providing apparatus manages the data specification and receives a request for processing the data specification from a big data consumer apparatus. In response to the request for processing the data specification, the big data providing apparatus generates processed data by performing processing using the data specification, and transmits the processed data to the big data consumer apparatus. It is possible to collect the big data in real time.

Description

METHOD AND APPARATUS FOR REAL-TIME BIG DATA PROCESSING AND DISTRIBUTION BASED ON DATA SPECIFICATIONS FIELD OF THE INVENTION [0001]

The following embodiments relate to the provision of big data, and more particularly to a method and apparatus for processing real time processing and distribution of big data based on a data specification.

As the use of big data increases, big data service providers collect and use various data such as individuals, institutions, SNS and sensor information. In addition, data users are searching for and purchasing data they want through the data market.

In order to operate the data market for big data distribution, the data vendor should have priority for various kinds of data to meet the requirements of various data users. Further, in data retention, the purpose of analysis, the data transmission method and data format according to the user's system environment, and the like should be considered.

Generally, the data collected or purchased in the process of utilizing the big data by the data user is subjected to a preprocessing process. Through the preprocessing process, the same data can change into various types of data types depending on the internal system or the service environment.

This preprocessing process is mostly simple and iterative. Therefore, the preprocessing process does not require high computing performance.

In addition, the data market is operated in such a manner that the seller of the data discloses his or her own data through the catalog, the data user searches the catalog, and downloads the retrieved data through the catalog. In accordance with this operation, in order to meet the needs of the data consumer, the data seller must pre-sort and process the data for sale and store the classified and processed data in a large-scale data store. This sorting, processing and storage creates high storage operating costs.

In order to more efficiently cope with this situation, the following embodiments disclose a method, apparatus and system for supporting the processing and distribution of data in real time on the basis of data that a data seller has or collects.

In order to construct such data specification-based big data real-time processing and distribution support system, a data seller should be able to develop various kinds of data specifications based on data that he / she initially owns or can collect in real time, It is necessary to satisfy the demand of data users by registering the developed data specification in the system.

Korean Patent Laid-Open Publication No. 2015-0071737 has been presented in connection with a big data transaction system and a transaction method.

In an embodiment, a big data seller who wants to distribute data by operating a data market in a big data environment can write data specimens that can be processed based on his own data or data that can be collected in real time And methods.

One embodiment may provide an apparatus and method for enabling a pre-written data specification to be disclosed in a data market.

One embodiment may provide an apparatus and method for performing processing of data in real time and providing processed data when a big data consumer selects a data specification and wants to purchase data.

One embodiment of the present invention can provide an apparatus and method for allowing a big data seller to generate processing information on various types of data in the form of a data specification based on available data.

One embodiment provides an apparatus and method for providing a data specification to a big data consumer, thereby enabling various types of data to be provided using a minimal amount of storage without maintaining or managing all of the various types of data in the repository .

One embodiment can provide an apparatus and method for allowing a big data consumer to select and utilize various data suitable for his / her system environment.

One embodiment may provide an apparatus and method for reducing the cost of analyzing and utilizing the data required by a big data consumer in selecting and utilizing various data.

An apparatus for real time processing and distributing big data based on a data specification, the system comprising: a data specification managing unit for managing a data specification and receiving a request for processing a data specification from a big data consumer apparatus; And a big data management unit for generating processed data by performing processing using the data specification and transmitting the processed data to the big data consumer apparatus.

The data statement managing unit may include a data statement registering unit that registers the data statement that can be sold to the big data consumer apparatus in the data statement list.

The data specification may include information on source data, a procedure of generating data, and a sales policy of data.

The data statement management unit may include a data use environment setting unit that sets a big data use environment.

The Big Data usage environment may include a type of the processed data provided to the Big Data Consumer Device and a manner in which the Big Data Consumer Device obtains the processed data.

The data statement management unit may include a data statement retrieval unit that provides the retrieval of data statements registered in the apparatus to the big data consumer apparatus.

The big data management unit may further include a real-time data resource management unit for managing information on real-time data resources.

The real-time data resource may not be stored in the device, but may be a resource that the device can download in real time through a data request or a resource that the device can collect.

The information on the real-time data resource may include data access information, collection information, and storage type information.

The real-time data resource management unit may collect real-time data from the external data resource providing apparatus.

The big data management unit may include a data specification publishing unit for generating, in the form of a data specification, data generated for the purpose of sale based on available data resources by a user of the apparatus.

The big data management unit may include a data policy management unit that performs a review on the processed data and corrects the processed data when a modification to the processed data is required.

When the processed data is transmitted to the big data consumer apparatus in the form of a virtualized database, the big data management unit may include a right to write, read, And a data policy management unit for performing control on the data policy management unit.

The big data management unit may include a data processing unit that obtains data stored in the device and real-time data provided from an external device in real time based on the data generation information of the data specification.

The big data management unit may include a data processing unit that generates the processed data by performing processing using the data specification.

And the data processing unit can process the data in a form selected by the big data consumer apparatus according to the setting of the big data use environment.

The big data management unit may include a storage data map management unit for managing where the data stored in the device is stored in the data storage of the device.

The data specification may include a product catalog portion, a user selected portion, and a data generation information portion.

The user selection portion may provide information for setting a range of data to be purchased by the big data consumer device and a method of using the data.

The data generation information part may provide information for generating data based on information selected by the provider of the data and information selected by the big data consumer device.

On another side, receiving a request for processing a data specification from a big data consumer device; Generating processed data by performing processing using the data specification; And transmitting the processed data to the big data consumer device.

The processed data can be processed into a selected form according to the setting of the big data use environment.

In another aspect, there is provided a data processing method comprising the steps of: generating processed data by performing processing of data; Generating a data specification based on a processing history of the processed data; And registering the data specification in a data list, wherein the data includes real-time data collected in real time.

In addition, there is further provided another method, apparatus, system for implementing the invention and a computer readable recording medium for recording a computer program for executing the method.

There is provided a device and method for allowing a big data seller who wants to operate a data market in a big data environment to distribute data to create a data specimen that can be processed based on his own data or data that can be collected in real time do.

An apparatus and method for enabling a previously prepared data specification to be disclosed in a data market is provided.

There is provided an apparatus and method for performing processing of data in real time and providing processed data when a big data consumer selects a data specification and wants to purchase data.

There is provided an apparatus and method for allowing a big data seller to generate processing information on various types of data in the form of a data specification based on available data.

There is provided an apparatus and method for providing a data specification to a big data consumer so that various types of data can be provided using a minimum-sized storage without maintaining or managing all the various types of data in the storage.

An apparatus and method for enabling a big data consumer to select and utilize various data suitable for his / her system environment is provided.

Devices and methods are provided that reduce the cost of analyzing and utilizing the data required by large data consumers in selecting and utilizing the various data.

1 shows a system for providing real-time processing and distribution of big data based on a data specification according to an embodiment.
FIG. 2 shows a structure of a big data providing apparatus according to an embodiment.
3 shows a configuration of a data specification according to an example.
4 shows the information of the workflow of the data specification according to an example.
5 is a flowchart of a data providing method according to an embodiment.
6 is a flowchart of a method of generating a data specification according to an embodiment.
FIG. 7 illustrates an electronic device implementing a big data providing apparatus according to an embodiment.

The following detailed description of exemplary embodiments refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the location or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the embodiments. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the exemplary embodiments is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained.

In the drawings, like reference numerals refer to the same or similar functions throughout the several views. The shape and size of the elements in the figures may be exaggerated for clarity.

The terms used in the examples are intended to illustrate the embodiments and are not intended to limit the invention. In the examples, the singular includes the plural unless otherwise stated in the specification. It is noted that the terms "comprises" and / or "comprising" used in the specification are intended to be inclusive in a manner similar to the components, steps, operations, and / And that additional configurations may be encompassed within the scope of the embodiments of the exemplary embodiments or the technical ideas of the exemplary embodiments. When it is mentioned that a component is "connected" or "connected" to another component, the two components may be directly connected or connected to each other, It is to be understood that other components may be present in the middle of the components.

The terms first and second, etc. may be used to describe various components, but the components should not be limited by the terms above. The above terms are used to distinguish one component from another. For example, without departing from the scope of the right, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

In addition, the elements shown in the embodiments are shown independently to represent different characteristic functions, and do not mean that each element is composed of separate hardware or a single software constituent unit. That is, each component is listed as each component for convenience of explanation. For example, at least two of the components may be combined into a single component. Also, one component can be divided into a plurality of components. The integrated embodiments and the separate embodiments of each of these components are also included in the scope of the right without departing from the essence.

Also, some components are not essential components to perform essential functions, but may be optional components only to improve performance. Embodiments may be implemented only with components that are essential to implementing the essentials of the embodiments, and structures within which the optional components are excluded, such as, for example, components used only for performance enhancement, are also included in the scope of the right.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to facilitate embodiments of the present invention by those skilled in the art. In the following description of the embodiments, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear.

1 shows a system for providing real-time processing and distribution of big data based on a data specification according to an embodiment.

A system 100 for providing real time processing and distribution of big data based on a data specification may include a big data providing apparatus 110, a big data consumer apparatus 120 and an external data resource providing apparatus 130.

The big data providing apparatus 110 can provide real time processing and distribution of the big data based on the data specification.

The big data providing apparatus 110 may represent a system for processing big data in real time based on a data specification, and may represent a system for distributing big data in real time.

The big data consumer device 120 may be used by the big data consumer.

The big data consumer device 120 can retrieve the information of the data specifications registered in the big data providing device 110. [ The big data consumer can select the data specification of the data that the big data consumer wants to purchase through the retrieved data specifications. The big data consumer device 120 can request the data of the selected data specification to the big data providing device 110. [

The big data providing apparatus 110 can perform processing of data based on the data specification selected by the big data consumer apparatus 120. [ The big data providing apparatus 110 can provide the processed data to the big data consumer apparatus 120. [

Here, the data may be data held by the big data providing apparatus 110 and may be data of the external data resource providing apparatus 130. [

The big data providing apparatus 110 may collect real-time data from the external data resource providing apparatus 130. [ The big data providing apparatus 110 can request data to the external data resource providing apparatus 130 in real time and can receive data transmitted from the external data resource providing apparatus 130 in real time upon request.

As described above, the big data providing apparatus 110 may include a data statement management function and a big data management function.

The data statement management functions include: 1) registration of a data specification; 2) retrieval of data from the big data consumer device 120; 3) data request from the big data consumer device 120; and 4) Data provision and the like, and the data processing request according to the data specification can be transmitted to the big data management function.

3) data transmission from the external data resource providing apparatus 130; 4) data transfer from the external data resource providing apparatus 130 to the external data resource providing apparatus 130; And the like.

Detailed functions, processing procedures, configuration of information, and the like of the system 100 for providing real-time processing and distribution of big data based on the data specification according to embodiments will be described in detail below with reference to the accompanying drawings. do.

FIG. 2 shows a structure of a big data providing apparatus according to an embodiment.

The big data providing apparatus 110 can support real-time processing and distribution of the big data based on the data specification.

The big data providing apparatus 110 may include a data specification management unit 210, a big data management unit 220, and a system control unit 230.

The data specification managing unit 210 can manage the data specification.

The data statement management unit 210 can receive a processing request for the data specification from the big data consumer apparatus 120. [

The big data management unit 220 can generate the processed data by performing processing using the data specification.

The big data management unit 220 can transmit the processed data to the big data consumer apparatus.

The data specification management unit 210 may include a data specification registration unit 211, a data usage setting unit 212, and a data specification search unit 213.

The data specification register 211 can register the data specification that can be sold to the big data consumer or the big data consumer device 120 in the data list.

In registering the data statement in the data list, the sales policy of the data of the data specification can be determined by the big data consumer or the big data consumer apparatus 120. [ The data specification registering unit 211 may include the data sales policy in the data specification information.

The sales policy may include the selling price and the like.

The registered data specification may include 1) information about source data, 2) a procedure for generating data, and 3) a sales policy of data.

The data use environment setting unit 212 can set a big data use environment.

The Big Data usage environment may include the type of processed data provided to the Big Data Consumer Device 120 and the way the Big Data Consumer Device 120 obtains the processed data.

The data use configuration unit 212 can receive a request for setting up a big data use environment from the big data consumer apparatus 120. [

The data use configuration unit 212 can set the type of the processed data and the manner in which the big data consumer apparatus 120 acquires the processed data in response to a request for setting of the big data use environment.

For example, the Big Data Seller may enable the Big Data Consumer Device 120 to select one of a plurality of transmission schemes as the transmission method of the processed data through the setting of the manner of acquiring the data.

For example, the data use configuration unit 212 may add a method of receiving the set data to the data specification.

The data specification retrieving unit 213 may provide the big data consumer apparatus 120 with a search for the registered data specifications.

The data specification retrieving unit 213 may receive a request for information of registered data specifications from the big data consumer apparatus 120. [ Upon receipt of the request for the information of the registered data specification, the data description retrieving unit 213 may transmit the information of the registered data specification to the big data consumer apparatus 120.

The big data management unit 220 includes a real data resource management unit 221, a data specification publishing unit 222, a data policy management unit 223, a data processing unit 224, a storage data map management unit 225, a data storage 226, And a data transfer unit 227.

The real-time data resource management unit 221 can manage information on real-time data resources. The real-time data resource is not stored in the data store 226, but may be a resource that can be downloaded in real time via the data request by the big data providing apparatus 110 or a resource that can be collected by the big data providing apparatus 110 .

The information about the real-time data resource may include 1) data access information, 2) collection tool information, and 3) storage type information.

The real-time data resource management unit 221 can collect real-time data from the external data resource provision device 130. [ The real-time data resource management unit 221 can request data to the external data resource providing apparatus 130 in real time and can receive data transmitted from the external data resource providing apparatus 130 in real time upon request.

The data specification publishing unit 122 can generate data generated in the form of a data specification for the purpose of sale based on the available data resources by the user of the big data providing apparatus 110. [ The data specification publishing unit 222 can register the data specification in the data specification registration unit 211. [

The data policy management unit 223 can perform a review on the processed data and modify the processed data when a modification to the processed data is required.

The data policy management unit 223 may write and / or read the processed data when the processed data is transmitted to the big data consumer apparatus 120 in the form of a virtualized database, and / Control of personal information related matters in the data can be performed.

The data processing unit 224 can obtain the data stored in the big data providing apparatus 110 and the real time data provided in real time from the external data resource providing apparatus 130 based on the data generation information of the data specification.

The data processing unit 224 can request data from the storage data map management unit 225 and the real-time data resource management unit 221 based on the data generation information of the data specification. The data may include stored data and real-time data.

For example, the data processing unit 224 can request data from the storage data map manager 225 when the data generation information indicates that data stored in the big data providing apparatus 110 is requested.

For example, the data processing unit 224 may request real-time data from the real-time data resource management unit 221 when the data generation information indicates that a real-time data resource is required.

The data processing section 224 can generate the processed data by performing processing using the data specification. The data processing unit 224 can process data in a form selected by the big data consumer or the big data consumer apparatus 120 according to the setting of the big data use environment.

The data processing section 224 can generate the processed data by processing the prepared data in accordance with the data specification. The prepared data may include data stored in the big data provision apparatus 110 provided by the storage data map management unit 225 and real time data provided by the real time data resource management unit 221. [

The data processing unit 224 may temporarily store the processed data in the data storage 226. [

The storage data map management unit 225 can manage where the data stored in the big data provision apparatus 110 is stored in the data storage 226. [

The data store 226 may physically store the data stored in the big data providing apparatus 110 or the processed data to be transmitted externally.

The data transfer unit 227 can transfer the processed data to the big data consumer device 120. [

The system control unit 230 can manage and control a process for interactions between the data specification management unit 210 and the big data management unit 220. In addition, the system control unit 230 can process the interaction between the big data providing apparatus 110 and the big data consumer apparatus 120. [

The system control unit 230, the data specification registering unit 211, the data usage setting unit 212, the data specification searching unit 213, the data management unit 210, the data management unit 210, the big data management unit 220, The real-time data resource management unit 221, the data specification publishing unit 222, the data policy management unit 223, the data processing unit 224, the storage data map management unit 225, the data storage 226, and the data transfer unit 227 At least some of which may be program modules and may communicate with an external device or system. The program modules may be included in the big data providing apparatus 100 in the form of an operating system, an application program module, and other program modules.

3 shows a configuration of a data specification according to an example.

The data specification may include 1) a product catalog portion, 2) a user selection portion, and 3) a data generation information portion.

The product catalog portion may provide information for the big data consumer or the big data consumer device 120 to identify the product of the data.

The user selection part may provide information for setting the range of data that the big data consumer or big data consumer device 120 desires to purchase and how to use the data.

The data generation information portion may provide information for generating data based on information selected by the provider of the data and information selected by the big data consumer or the big data consumer device 120. [

For example, the product catalog portion of the data specification may include information such as 1) the product name of the data, 2) the data domain of the data, 3) the application field of the data, and 4) the time range of the data.

For example, the user selection portion may provide information on how to purchase the data. The purchase method of the data may include a partial purchase of the data and a transmission method of the data. The data transmission method may be a data download method.

That is to say, the user selection portion can provide information that enables the big data consumer or the big data consumer device 120 to partially purchase the data when it is intended to partially purchase the data. In addition, the user selection portion may provide information that allows the Big Data Consumer or Big Data Consumer Device 120 to select and set one of a plurality of transmission schemes. For example, one of a plurality of transmission schemes may be transmission using cloud storage.

The data generation information may include information on a data source used to generate the data and a work flow for the function. Basically, when a data specification is selected, the values of the configuration variables of the data specification can be defined in advance. For the workflow execution, the setting values of the setting variables can be updated according to the range of data selected by the big data consumer or the data consumer selected by the big data consumer apparatus 120 or the transmission method of the data. The setting values for updating can be transmitted to the data processing unit 224 of the big data management unit 220. [ Data to be sold can be generated in real time as the set values are transmitted.

The information of the workflow is separately described below with reference to Fig.

4 shows the information of the workflow of the data specification according to an example.

The functions of the workflow of the data specification may be plural. As a plurality of functions, FC_3223, FC_0482 and FC_0156 are shown in Fig.

4, the process of determining the values of the setting variables for exemplary functions such as FC_3223, FC_0482, and FC_0156 from the preset values to the big data consumer or the setting value determined by the big data consumer device 120 is shown in Fig. .

The input of the function may include a data specification. Further, the input of the function may further include data. Here, the data specification may be a temporary data specification output from another function. The input of the first of the plurality of functions may include a data specification. The input of the non-first function of the plurality of functions may include a temporary data specification.

The output of the function may be a temporary data specification or data. The output of the final function among the plurality of functions may be data. The output of the non-final function among the plurality of functions may be a temporary data specification.

5 is a flowchart of a data providing method according to an embodiment.

Through the steps 510, 520, 530, 540, 550, 560, 570 and 580, real-time processing and distribution of the big data based on the data specification can be provided.

In step 510, the big data consumer device 120 may retrieve information of the data specifications registered in the big data providing device 110. [

The big data consumer apparatus 120 may transmit a request of information of the registered data specifications to the data description retrieval unit 213. [ The data specification retrieving unit 213 may receive a request for information of registered data specifications from the big data consumer apparatus 120. [

Upon receipt of the request for the information of the registered data specification, the data description retrieving unit 213 may transmit the information of the registered data specification to the big data consumer apparatus 120. The big data consumer apparatus 120 can receive the information of the registered data specifications from the data description retrieving unit 213. [

The Big Data Consumer or Big Data Consumer Device 120 may use the information of the registered data specifications to select the data it desires.

In step 520, the big data consumer device 120 may set the Big Data usage environment.

The big data consumer or big data consumer device 120 may select one of a plurality of types provided by the big data providing device 110 as a form of processed data. In addition, the big data consumer or the big data consumer apparatus 120 can select one of the plurality of transmission methods provided by the big data providing apparatus 110 as the transmission method of the processed data.

The big data consumer apparatus 120 can transmit a request for setting the big data use environment to the data use configuration unit 212. [ The data use configuration unit 212 can receive a request for setting up a big data use environment from the big data consumer apparatus 120. [

The data use configuration unit 212 can set the type of the processed data and the manner in which the big data consumer apparatus 120 acquires the processed data in response to a request for setting of the big data use environment.

At step 530, the big data consumer device 120 may send a request for processing of the data specification to the data statement manager 210. The data statement management unit 210 can receive a processing request for the data specification from the big data consumer apparatus 120. [

The data specification management unit 210 can transmit the data specification to the data processing unit 2240 of the big data management unit 220 through the system control unit 230 and can send the data specification to the data processing unit 2240 of the big data management unit 220, It is possible to send a request for processing to the big data processing unit 224.

When a real-time data resource is required for processing the data specification, the data processing unit 224 may transmit a request for real-time data to the real-time data resource management unit 221. [ The real-time data resource management unit 221 may receive a request for real-time data from the data processing unit 224. [

After transmitting the request of the real-time data to the real-time data resource management unit 221, the data processing unit 224 can wait until the preparation of the real-time data is completed.

In step 540, the real-time data resource management unit 221 may collect real-time data from the external data resource provision apparatus 130. [

In step 550, the data processing unit 224 can recognize that the preparation of data used for processing the data specification is completed. The prepared data may include data stored in the big data provision apparatus 110 provided by the storage data map management unit 225 and real time data provided by the real time data resource management unit 221. [

In step 560, the data processing section 224 can generate the processed data by processing the prepared data in accordance with the data specification.

The data processing unit 224 may temporarily store the processed data in the data storage 226. [

In step 570, the data policy management unit 223 can perform a review of the processed data and modify the processed data when a modification to the processed data is required.

In step 580, the data transfer portion 227 may transfer the processed data to the big data consumer device 120. [

6 is a flowchart of a method of generating a data specification according to an embodiment.

Through the steps 610, 620, 630, 640, 650, 660, and 670, a data specification for real-time processing and distribution of the big data based on the data specification can be generated.

At step 610, the data specification publisher 122 may provide the Big Data Consumer Device 120 with information about the available data resources.

The Big Data Consumer Device 120 can identify the available data resources and, if necessary, send a request for the collection of real time data to the Data Processing portion 224. Data processing unit 224 may receive a request for collection of real-time data from big data consumer device 120. [

The real-time data may be data collected in real time according to the generation of the data specification.

The data processing unit 224 may transmit a request for collection of real-time data to the real-time data resource management unit 221. [ The real-time data resource management unit 221 may receive a request for collection of real-time data from the data processing unit 224. [

The data processing unit 224 may wait until the preparation of the real time data is completed after transmitting the request for the collection of the real time data to the real time data resource management unit 221. [

In operation 620, the real-time data resource management unit 221 may collect real-time data from the external data resource providing apparatus 130. [

In step 630, the data processing unit 224 may recognize that the preparation of data for generation of the data specification is completed. The prepared data may include data stored in the big data provision apparatus 110 provided by the storage data map management unit 225 and real time data provided by the real time data resource management unit 221. [

In step 640, the data processing unit 224 may perform processing of the prepared data. The data processing section 224 can generate the processed data by performing processing of the prepared data. In processing the data, the data processing function provided by the data processing section 224 may be used. The processed data may be data that can be generated in real time through the data specification.

At step 650, when the processing of the data is completed, the data specification publisher 222 can generate the data specification based on the processing history of the processed data.

The processed data can be selectively deleted in the big data providing apparatus 110. In step 660, the data processing section 224 may delete the processed data.

In step 670, the data specification registration unit 211 may register the data specification that has been generated in the data specification list.

In registering the data specification in the data list, the sales policy of the data in the data specification can be determined. The information of the data specification may include a sales policy of the data of the data specification.

Through the procedure described above, the big data seller can generate the data specification and register the generated data specification in the big data providing apparatus 110. [

The Big Data Consumer or Big Data Consumer Device 120 may retrieve the registered data specification and order the data of the retrieved Data Specification. The big data providing apparatus 110 can process the requested data in real time and transmit the processed data to the big data consumer apparatus 120. [

FIG. 7 illustrates an electronic device implementing a big data providing apparatus according to an embodiment.

The big data providing apparatus 110 may be implemented as the electronic apparatus 700 shown in Fig. The electronic device 700 may be a general purpose computer system that operates as the big data provision device 110. [

7, the electronic device 700 may include at least a portion of a processing unit 710, a communication unit 720, a memory 730, a storage 740, and a bus 790. As shown in FIG. Components of the electronic device 700, such as the processing unit 710, the communication unit 720, the memory 730, and the storage 740, may communicate with each other via the bus 790.

The processing unit 710 may be a semiconductor device that executes the processing instructions stored in the memory 730 or the storage 740. For example, the processing unit 710 may be at least one hardware processor.

The processing unit 710 may process the job required for the operation of the electronic device 700. [ The processing unit 710 may execute the code of the operation or step of the processing unit 710 described in the embodiments.

The processing unit 710 may perform generation, storage, and output of information to be described in the following embodiments, and may perform operations of steps performed in the other electronic device 700. [

The communication unit 720 may be connected to the network 799. May receive data or information required for operation of electronic device 700 and may transmit data or information required for operation of electronic device 700. [ The communication unit 720 can transmit data to another device via the network 799 and can receive data from another device. For example, the communication unit 720 may be a network chip or a port.

Memory 730 and storage 740 may be various types of volatile or non-volatile storage media. For example, the memory 730 may include at least one of a ROM 731 and a RAM 732. The storage 740 may include internal storage media such as RAM, flash memory and a hard disk, and may include removable storage media such as memory cards and the like.

The function or operation of the electronic device 700 may be performed as the processing unit 710 executes at least one program module. The memory 730 and / or the storage 740 may store at least one program module. The at least one program module may be configured to be executed by the processing unit 710.

The electronic device 700 may further include a user interface (UI) input device 750 and a UI output device 760. The UI input device 750 may receive the input of the user required for operation of the electronic device 700. The UI output device 760 may output information or data according to the operation of the electronic device 700. [

According to the above-described embodiments, the following scenario can be exemplified.

The Big Data Seller can hold map data, Social Networking Service (SNS) data, store information data, and floating population data. The big data seller can create information on the trade area and preference according to the region as data specification based on the data held by the seller. The data specification can be registered in the big data providing apparatus 110. [

The big data consumer can purchase the clothing maker's preference information for the last one year in the area desired by the big data providing device 110 in a format provided by DB-API.

The seller of the big data can generate the information desired by the big data consumer through the real time processing and distribution function of the big data based on the data specification of the big data providing apparatus 110 based on the data specification, It can be provided in API form.

Through the provision of data in this manner. Big data sellers can minimize the storage capacity for sales of data while meeting the requirements of big data consumers, and big data consumers can use APIs provided for the use of data through their mashups on their devices Thereby saving time and cost for analyzing the data.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

100: System that provides real-time processing and distribution of big data based on data specification
110: Big data provider
120: Big Data Consumer Device
130: External data resource providing device

Claims (20)

An apparatus for processing and distributing big data in real time based on a data specification,
A data statement management unit for managing the data specification and receiving a request for processing of the data specification from the big data consumer device; And
A big data management unit for generating processed data by performing processing using a data specification and transmitting the processed data to the big data consumer apparatus,
/ RTI > The method according to claim 1,
The data statement management unit,
A data specification registering unit for registering the data specification which can be sold to the big data consumer apparatus,
/ RTI > The method according to claim 1,
Wherein the data specification comprises information on source data, a procedure for generating data, and a sales policy for the data. The method according to claim 1,
The data statement management unit,
A data use setting unit for setting a big data use environment
Lt; / RTI >
Wherein the big data usage environment includes a type of the processed data provided to the big data consumer device and a manner in which the big data consumer device obtains the processed data. The method according to claim 1,
The data statement management unit,
A data statement retrieval unit for providing retrieval of data statements registered in the apparatus to the big data consumer apparatus,
/ RTI > The method according to claim 1,
The big data management unit,
A real-time data resource management unit for managing information on real-
Further comprising:
Wherein the real-time data resource is not stored in the device, but is a resource that the device can download in real time via a data request or a resource that the device can collect. The method according to claim 6,
Wherein the information about the real-time data resource includes data access information, collection information, and storage type information. The method according to claim 6,
Wherein the real-time data resource management unit collects real-time data from an external data resource providing device. The method according to claim 1,
The big data management unit,
A data specification publisher for generating data generated for sale purposes in the form of a data specification based on available data resources by a user of the device;
/ RTI > The method according to claim 1,
The big data management unit,
A data policy management unit for performing a review on the processed data and correcting the processed data when a modification to the processed data is required,
/ RTI > The method according to claim 1,
The big data management unit,
When the processed data is transferred to the big data consumer apparatus in the form of a virtualized database, control is performed on the write right, the read right, and the personal information related matters in the data Data policy management unit
/ RTI > The method according to claim 1,
The big data management unit,
A data processing unit for obtaining data stored in the device and real-time data provided from an external device in real time based on data generation information of the data specification,
/ RTI > The method according to claim 1,
Wherein the big data management unit comprises a data processing unit for generating the processed data by performing processing using the data specification,
Lt; / RTI >
And the data processing unit processes data in a form selected by the big data consumer apparatus according to setting of a big data use environment. The method according to claim 1,
The big data management unit,
A storage data map manager for managing where the data stored in the device is stored in the data storage of the device,
/ RTI > The method according to claim 1,
Wherein the data specification comprises a product catalog portion, a user selected portion and a data generation information portion. 16. The method of claim 15,
Wherein the user selection portion provides information for setting a range of data to be purchased by the big data consumer device and a method of using the data. 16. The method of claim 15,
Wherein the data generation information portion provides information for generating data based on information selected by the provider of the data and information selected by the big data consumer device. Receiving a request for processing a data specification from a big data consumer device;
Generating processed data by performing processing using the data specification; And
Transmitting the processed data to the big data consumer device
/ RTI > 19. The method of claim 18,
Wherein the processed data is processed into a selected form according to setting of a big data use environment. Generating processed data by performing processing of the data;
Generating a data specification based on a processing history of the processed data; And
Registering the data statement in a data statement list
Lt; / RTI >
Wherein the data includes real-time data collected in real time.

Images