KR20190043446A - Workflow engine framework - Google Patents

Abstract

The present invention relates to a workflow engine framework for creating a domain-adaptive and further cross-domain adaptive workflow execution platform that is purposeful through an organic configuration of dynamic engine components. According to the present invention, the workflow engine framework comprises: a resource management unit for managing a resource including an engine component and a workflow attribute specification component necessary for execution of a workflow defined by a user; a system configuration unit for generating an engine by configuring necessary engine component containers according to workflow specification by assembling the attribute specification component and dynamically combining the engine component necessary for executing the workflow; and a system control unit for controlling the execution by driving at least one engine generated by the system configuration unit according to a method defined in a workflow attribute specification. In addition, provided in the present invention is a cross-domain adaptive workflow engine framework, in which workflow engine frameworks are assigned to two or more different single-domains respectively, to be connected, through a network, to single-domain workflow engine frameworks, and which includes a cross-domain convergence system determining a single-domain to distribute a necessary engine to each single-domain included in the cross-domain according to a user-defined cross-domain workflow.

Description

Workflow engine framework.

Field of the Invention The present invention relates to workflow and framework techniques and, more particularly, to a workflow and framework technique that can be implemented in a single domain adaptive and further cross domain adaptive To a workflow engine framework capable of creating a workflow execution platform of the workflow engine.

Workflow technology refers to the automation technology of business processes that transfer documents, information, tasks, etc. from one user (application) to another for processing by a series of business procedure rules. In particular, the procedure for creating a data-driven service workflow is as follows. First, determine the data sources and how they are collected. Determine how to process the collected data and determine the analysis method (eg analysis by machine learning, prediction, knowledge inference, etc.). Determine the method to service the analyzed results. The workflow creation procedure is completed by configuring the engine (s) necessary to execute the corresponding workflow and defining the linkage method between the engines.

In recent years, so-called intelligent objects that give artificial intelligence to various objects have been attracting attention. In particular, for intelligent Internet applications (eg, Smart City) that encompass a large number of heterogeneous Internet domains, a system capable of controlling, managing, and controlling heterogeneous object intelligence domains is needed. In addition, problems can arise that require intelligent processing of various business domains (e.g., energy, health, transportation, education, power plants, etc.). Within a single business domain, there can be a variety of target domains, from devices where data is generated and actioned, to edges where data is processed, delivered, instantly analyzed and judged, to the cloud where complex analysis and applications occur. Also, in the task domain or the target domain, there may be a time domain according to the classification of the space and a time domain according to the time classification. In addition, there can be various method domains such as a data processing domain, a learning domain, a prediction domain, and a service domain. Therefore, there is a need for a unified method and system for effectively controlling, managing, and controlling such a complex multi-layer domain (hereinafter referred to as a cross domain).

On the other hand, in order to quickly extract insights contained in Big Data collected through the Internet and extract them from Internet data of objects, various technologies are being developed to support quick and accurate decision making. This requires stream processing technology for real-time analytics as well as platform technology for real-time forecasting / analysis.

In addition, the need to design workflows by utilizing the recently emerging machine learning technologies for IOT big data analysis is growing, and as the IoT domains of various purposes and fields are developed, they are organically integrated so that more insightful analyzes and services The need for a unified platform technology to enable. However, conventional workflow techniques have limitations in combining machine learning with IoT big data having different characteristics depending on device, data, and domain.

In order to overcome the limitations and problems described above, it is possible to create and execute a workflow suitable for the purpose of each domain through the organic configuration of the engine components, and easily apply it to another domain to implement (domain adaptation) Flows can be created and executed, and in particular, a unified system is needed that can be integrally controlled and managed to accommodate cross domains.

Therefore, the present invention proposes a workflow engine framework for creating a domain-adaptive and cross-domain adaptive workflow execution platform that meets the purpose through the organic construction of dynamic engine components.

According to an aspect of the present invention, there is provided an information processing system including a resource management unit for managing resources including a workflow attribute specification component and an engine component necessary for execution of a workflow defined by a user; A system configuration unit for assembling the attribute specification components and dynamically combining the engine components necessary for executing the workflow to construct the engine component containers according to the workflow specification to generate the engine; There is provided a workflow engine framework including a system control section for controlling execution by driving one or more engines generated by the system configuration section in accordance with a manner defined in a workflow attribute specification.

According to another aspect of the present invention, the workflow engine framework is each assigned to two or more different single domains, networked with these single domain workflow engine frameworks, and crossed according to a user-defined cross- A cross-domain adaptive workflow engine framework is provided that includes a cross-domain convergence system that determines a single domain in which to deploy the required engine for each single domain included in the domain.

By organically organizing the engine components, you can create and execute workflows that match the domain and purpose each domain is trying to solve, and you can easily create and run workflows for different domains by applying them to other domains. That is, a domain adaptive workflow engine framework that can dynamically recycle engine components can be easily configured to easily configure an execution platform for solving similar problems in various business domains or destination domains. In addition, by including a cross domain knowledge fusion brain, a cross domain adaptive workflow engine framework can be constructed. Furthermore, in real-time analysis service of large capacity data which combines IoT, big data, and machine learning, it is possible to implement and manage components of a machine learning model and a big data analysis model developed for solving specific problems.

It is not necessary to develop individual solutions for applications in various artificial intelligence services or applications, and it is necessary to develop additional components and recycle components that have been developed and installed so as to satisfy the new workflow. It is possible to realize a framework that can be configured / executed.

1 is a schematic diagram of a workflow engine framework according to the present invention;
FIG. 2 is a detailed configuration diagram of the system configuration section 30. FIG.
3 is a GUI screen example diagram for explaining the configuration of the system definition editor 70
4 is a diagram showing a procedure for configuring an engine from the specification of an engine constituting a workflow
5 is a configuration diagram of an engine as an example of a workflow execution instance configurable by the workflow configuration unit 36 in FIG.
6 is a diagram showing an example of an engine configuration equipped with a unit processor
Figure 7 is a specific embodiment of the data processing engine illustrated in Figure 6
8 is a diagram showing another example of an engine configuration in which a unit processor is mounted;
9 shows a configuration of a workflow execution system in which one or more engines are connected in a pipeline manner.
10 illustrates a configuration of a workflow execution system in which one or more engines run through multiple types of data paths;
Fig. 11 is an explanatory diagram of the IoT and artificial intelligence based illumination / temperature control workflow service scenario
12 is an explanatory diagram of a scenario of a deep-running-based traffic speed monitoring service
FIG. 13 is a diagram showing the internal structure of the workflow engine framework 10 of FIG.
14 is a diagram showing an example of a configuration of a serving engine for providing an intelligent service as an engine for providing a service to a client or a user
FIG. 15 is a block diagram of the internal structure of the workflow engine framework 10 of FIG.
16 is a diagram showing an example of a cross-domain workflow engine framework configuration;
17 is a diagram showing an execution procedure of a cross-domain workflow
18 is a diagram for explaining a scenario for achieving smart city;
19 is a flowchart showing a work flow < RTI ID = 0.0 > One embodiment of the engine framework
20 shows an embodiment of the smart streetlight control recommendation engine configuration of workflow 94 shown in Fig. 19

1 is a configuration diagram showing an embodiment of a workflow engine framework of the present invention. The workflow engine framework 10 of the present invention basically comprises:

A system definition editor 70 for creating a structure and a specification of a work to be executed (workflow)

An attribute specification component associated with the attribute for sorting each engine component on the workflow specification (= specification of the engine, that is, the one or more engines constituting the workflow) to be created by the user through the system definition editor 70 An engine component corresponding to an engine component constituting an engine for executing a workflow, and a resource management unit 20 for managing resources such as an existing workflow specification instance,

A system construction unit (hereinafter, referred to as " engine ") for assembling a workflow attribute specification, dynamically combining engine components necessary for executing a workflow, and generating an engine instance 32 30),

An engine (instance) 32 generated by the system configuration unit 30,

And a system control unit 40 for controlling the execution of the resource management unit 20, the system configuration unit 30, and the engine 32.

In the workflow engine framework 10, the user defines a workflow consisting of one or more engines to create the desired system. At this time, the workflow consists of definitions for one or more engines, and the definition of the engine corresponds to a combination of an engine component container for storing the engine components and an engine component for the engine component container.

An engine component container can be made up of a combination of one or more reader components, a writer component, a runner component, an operator component, and a controller component. These one or more input, component, runner, processor, and controller components are each made up of a combination of a property specification component that defines the properties that determine the component's properties and an executable component that corresponds to the actual implementation of the components. For example, an execution component corresponds to a class such as Java or C ++, and an attribute specification component corresponds to a class that contains a constructor parameter or a constructor parameter that can be included in the class's constructor. By defining the workflow for one or more engines corresponding to the definition of the execution system created in this way, you can dynamically configure the execution engine to create the workflow system required for various purpose business domains.

Fig. 2 shows a specific configuration and additional components of the system of Fig.

An execution instance unit 50 in which the engines 32 composed of a combination of engine components dynamically generated by the system configuration unit 30 are generated and managed in the form of an engine instance (as a result, an engine completed and executed); A component unit 60 as a space in which an attribute specification component 62 and an engine component 64 managed in the resource management unit 20 are physically or virtually stored; Further, a front end 77 may be further included, which receives the workflow specification from the system definition editor 70 and transfers the specification to the system configuration unit 30. [

The front end 77 mediates execution of a process for receiving a request from a client, and plays a role corresponding to various requests such as user management and storage management. For example, the front end 77 may be a general application server, a web application server providing a web-based REST API, or a system including a socket communication-based listener module. In some cases, the front end 77 may be executed on a network different from the system definition editor 70 or the backend constituting the framework 10.

Before describing the workflow engine framework 10 of FIG. 1 and FIG. 2 in full, a workflow for defining a series of work specifications for driving an execution system in the framework 10 is created, The system definition editor 70, which plays a role of requesting to the front end 77 to instruct it to start at the first time, will be described first.

The system definition editor 70 defines a detailed domain for a desired task, defines a structure of a work to be processed for each domain (workflow structure), selects an engine component (s) according to the structure, And to create a specific specification of each engine component. To this end, the attribute specification component and the engine component can be inquired from the framework (in particular, the resource management section 20).

Here, the 'structure of work' is, for example, a process of collecting data, data processing, and learning from a specific object Internet platform, and sequentially processing them may correspond to a workflow execution flow. The 'component specification' specifies what device to collect from where to fetch the data, what process to process and where to export it, what connection method to use, what data to receive, , The memory information, and the storage location, which are specific to each element component.

A user (a data scientist, a model developer, a business developer, etc.) is connected to a system definition editor 70 via an engine component 64 that constitutes a workflow and an attribute specification component that can define parameters that determine the characteristics of the engine component (62) can be defined according to the protocol, and a pair of a component and an attribute specification can be defined and edited.

3 is an exemplary diagram of a GUI screen 71 for explaining the configuration of the system definition editor 70. As shown in FIG. The description of the configuration and operation of the system definition editor 70 will be omitted by explaining the GUI screen 71. [

The GUI screen 71 includes a function menu 72, an engine type selection unit 73, a component selection unit 74, a component attribute inquiry / editing unit 75, and a workflow instance storage / inquiry unit 76.

The function menu 72 is a menu for selecting various functions of the system definition editor 70. The function menu 72 includes functions such as New (Create New Workflow), Open (Load Saved Workflow), Save (Save Workflow) Workflow execution), Result (execution result view), Help (help), and the like.

The engine type selection unit 73 presents various engine types and allows the user to select one of them. The types of engines include, for example, a real-time stream processing engine, a batch analysis engine, an on-demand data processing engine, an evaluation engine, a batch data loading engine, a stream machine learning prediction engine, have.

The component selection unit 74 provides various engine component lists for each component type so that the user can select the component type and the engine component for each type. Table 1 below is an example of a component type and an engine component list presented by the component selection unit 74. [

Component type Engine component

Reader FileReader
HttpServerReader
Kafka Reader
MongodbReader
...
Writer FileWriter
KafkaWriter
MongodbReader
...
Controller SparkSessionController
SparkSessionStreamController
...
Runner SparkRunner
TensoflowRunner
...
Operator MinMaxScaler
Aggregator
...

The component property selecting / editing unit 75 can display the properties of the engine component selected by the component selecting unit 74, and the user can select and edit the property of the selected engine component.

The workflow instance selection unit 76 displays a list in which previously created workflows are stored. Of these, a workflow desired to be recycled may be selected and re-edited, or the execution request may be made to the framework 10 as it is. The unit of recycling can be the whole workflow, or it can be recycled by a single engine included in the workflow and used for editing and execution.

When the workflow specification file is created in the system definition editor 70, it is submitted to the system configuration unit 30 of the framework 10 shown in FIG. At this time, the front end 77 may receive the workflow specification and transmit the workflow specification to the system configuration unit 30.

Returning to FIG. 1 and FIG. 2, the resource management unit 20 of the framework 10 performs a function of managing components necessary for execution of a workflow. Specifically, as shown in Fig. 2, the resource management unit 20,

An attribute specification component 62 including an attribute specification for determining a component property or attribute of the workflow instance and an attribute specification component management unit 22 for managing (updating) the list,

And an engine component management unit 24 for managing (updating) components (engine components) for execution and a list thereof.

In addition, it may further include a workflow specification instance management unit 26 that manages a previously created and stored workflow specification instance. This workflow specification instance management unit 26 stores and manages the workflow specification instance so that it can be utilized at a later time (for example, via the workflow instance selection unit 76 in FIG. 3) at the request of the system definition editor 70 do.

In addition, the system configuration unit 30 in FIG. 1 performs a function of creating components necessary for executing the created workflow. The system configuration unit 30 constructs necessary engine component containers 32 according to a workflow specification (specification) received from the front end 77 to create an engine instance.

2, the system configuration unit 30 includes

A workflow attribute specification assembling unit 34 for creating a series of attribute specification components by binding an execution platform configuration workflow specification received from the system definition editor 70 to an attribute specification component,

- A series of engine components are constructed by extracting the defined engine component information from the assembled attribute specification components 62 and binding the attribute specification component 62 and the engine component 64 to form a workflow execution platform And a workflow constituting unit 36 constituting the workflow.

The workflow attribute specification assembling unit 34 creates a series of attribute specification component instances by binding a workflow specification for constituting an execution platform to the attribute specification component 62. [ Examples of the attribute specification component 62 generated by the workflow attribute specification assembling unit 34 include a Protobuf message object of Google, a Case class of Scala, a Property object of Java, and the like.

The workflow configuration unit 36 binds the engine component 64 constituting the workflow with the attribute specification component 62 defining the parameters determining the characteristics of the engine component 64, Dynamically constructs a workflow execution instance part 50 including a series of engine instances for executing a workflow by binding an instance of components to an engine container. In the workflow execution instance unit 50, one or more engines 32 dynamically generated by the workflow organizing unit 36 are created and executed. These engines 32, which are created to execute one workflow, may be distributed and executed on a computing machine connected to the same computing machine or network as an independent program package, or may be packaged on a virtual machine basis and distributed to different physical computing machines .

The system control unit 40 shown in FIGS. 1 and 2 performs the function of driving the engine instances 32 generated by the system configuration unit 30 in accordance with the processing procedure and terminating the engine instance 32 Is a module that plays a key role in the framework of the present invention. And controls the execution by driving one or more engines 32 created in the workflow execution instance unit 50 according to a method defined in the workflow attribute specification. In other words, when the system control unit 40 requests the workflow execution instance unit 50 to execute the workflow, the workflow is executed. Thereby, it is possible to achieve the objective that the workflow created by the user for any desired domain can achieve.

For example, the system control 40 may control one or more engines 32, which have a number of different types of data as source and destination, to be executed in a pipelined fashion. Or one or more engines with multiple different types of data origin and destination can be controlled to run concurrently. Various types of engine configurations will be described in detail later.

Fig. 4 shows a procedure for configuring the engine 32 from the workflow specification created in the system definition editor 70. Fig.

First, the workflow configuration unit 36 of the system configuration unit 30 receives the workflow specification (340) and acquires a series of attribute specification component instances including an attribute specification for the engine components constituting the engine (342). The attribute specification component (62) used here may be 'Class' which can contain a value in Google's proto buff message, a scalar case class, or other programming language.

The engine component instance is created by designating the generated attribute specification component instance as a generator parameter of the engine component (64) (344).

When an engine instance including the controller, the input device, the launcher, the output device, and the unit processor constituting the engine is generated, the process of creating the engine instance by dynamically binding to the engine component container 31 with the engine component container instance as the producer parameter RTI ID = 0.0 > 346 < / RTI > An engine instance 32 is created for every engine definition on the workflow, and the generated engine instance is executed and managed in the workflow execution instance unit 50. [

The procedure of FIG. 4 dynamically configures the workflow execution instance. Here, one engine can be defined and configured in the form of a virtual machine.

5 to 10 illustrate various engine 32 configuration schemes as an example of a workflow execution instance constituted by the workflow configuration section 36 referred to in Fig.

The basic engine 32 shown in Figure 5 includes a reader 322 for fetching data from one or more data sources, one or more writers for outputting the internally processed data to one or more data destinations, A runner 326 for executing a platform or a session executing a separate execution program or a session management program for processing input data, a series of processes for inputting such data through an input device, And a controller 328 responsible for control.

The controller 328 controls a series of processes using the input unit 322, the output unit 324, the execution unit 326, and the unit processor 323 described below. Requests input to the input unit 322 to read data from a data source (source), issues a processing request to drive the processing framework to process the data read through the input unit to the executor 326 and send it to the output unit, 324) to output the processed data to the data end point.

Input device 322 of engine 32 performs the function of reading data from a data store (not shown) of either type of in-memory buffer or cache, file system, messaging system, database, or network driver. Similarly, the writer 324 performs the function of writing data to an in-memory buffer or cache, a file system, a messaging system, a database, or a data store (not shown) of a type of network driver. The unit processor 323 receives the data, processes it, and exports the data. For example, it may be an implementation of various data processing functions included in the refinement / consolidation / reduction / transformation method mentioned in data mining. The launcher 326 can be any program or external platform / framework needed to process the data, including a Spark for processing large data, a deep learning platform such as Caffe for deep-run analysis, a Tensorflw platform, A connector or controller for interlocking with or executing one of the engines, a session manager, and the like.

5 and 6, when the nodes are configured in the order of the input unit 322, the one or more unit processors 323a to 323c, and the output unit 324, the controller 328 sequentially processes the data by pipelining To a next node, a simultaneous processing method in which each node is simultaneously executed, a simultaneous / sequential processing method in which both nodes are combined, and the like.

6 is a flowchart illustrating a process of processing input data by the controller 328 using one or more successive unit operators 323a-c among the methods of configuring the basic engine 32 of FIG. And sequentially transmits data to the output unit 324 in a pipelined manner. The controller 328 requests the input unit 322 to read data from the data source and issues a processor execution request to the execution unit 326 to drive the framework for data processing to process the data read through the input unit, A pipeline processing execution request is made to each of the unit processors 323a-c, and an output request is made so that the output unit 324 writes the processed data to the data end point. According to the configuration method of FIG. 6, various unit processors can be used in combination to easily perform processing corresponding to the purpose of various domains. The unit processors 323a-c may be implementations of methods for refining, reducing, integrating, and transforming corresponding to general data mining techniques.

Fig. 7 shows a specific embodiment of the data processing engine illustrated in Fig. A unit processor that executes a Spark framework for processing data and reads data through a reader reading data from a file or JDBC or Kafka to remove a specific column, Data processing, including a missing imputation unit, a scaling unit handler, a filter unit handler, a pivot unit handler, a writer that outputs to a file, JDBC, Kafka, etc. The engine can be configured.

8 is an engine of another type in which the controller 328 of FIG. 6 drives the unit processors 323a to 323c, and receives data from the input unit 322 and sequentially processes the data by pipelining Scheme, and at least one unit processor is driven so as to execute simultaneous processing.

As another embodiment of the engine 32 of various configurations described above, the executor 326 may be implemented as a deep learning framework such as Caffe or Tensorflow, a big data processing framework such as Spark or Hadoop MapReduce, an analysis such as R or Python And may be configured to interoperate with or include various processing frameworks or software programs such as frameworks.

In another embodiment, the engine 32 described above can be configured to mount an input device and an output device that use various data paths in the same system as data source and destination. That is, a source (included in the same system) of various logical driver concepts such as an in-memory buffer / cache, a file system, a messaging system, a database, and a network is defined as a data source and a destination, data is input from these, The engine can be configured by configuring the input device and the output device.

As another embodiment, the engine 32 described above can be configured as an engine that uses data paths existing in different systems or networks as data origin and destination, respectively. To this end, network address information, host information, and remote driver information can be included in the settings for the input device and the output device. By using different data source and destination destinations separately for the input and output devices, the engine can be utilized as a stream processing engine or a filter on the data path between the data source and the destination.

9 is a diagram showing a data transfer path 338 in which a destination of data to be output from an output device of one engine is transferred in a pipeline manner to a data source of an input device of another engine when a plurality of workflow execution engines 32a to 32c are configured. ) Is used. Where the data transfer path 338 may be, for example, Kafka. By employing the configuration as shown in Fig. 9, it is possible for the engines 32a to 32c to perform a complicated workflow by cooperating with the engines 32a to 32c that perform workflow processing for a plurality of different purposes.

Also in Fig. 9, one or more engines 32a-c may be located in different physical environments (e.g., networks, clusters, etc.) and may be configured with different types of launchers. For example, if the first engine 32a is associated with an executor that processes data and the second engine 32b is associated with a deep learning framework, It is possible to execute one workflow for solving the problem. In this case, each of the engines can be executed simultaneously, one by one, or individually at a specific point in time.

10 illustrates a method of pipelining different types of data delivery paths (e.g., file system delivery path 340 and network stream delivery path 342) by data origin and destination, respectively, in one or more engines 32a- A workflow execution system is constructed as shown in FIG. In this case, the input unit and the output unit in each engine are composed of a plurality of units. Here, the file system delivery path 340 refers to a batch delivery path, and the network stream delivery path 342 refers to a real time delivery path. A workflow system having a structure similar to that of the lambda architecture can be constructed by the method of Fig. 10, in which real time processing and batch processing can be performed at the same time.

A specific workflow service scenario will be introduced to help understand the structure and operation of the framework of the present invention described above.

11 is an explanatory diagram of an IoT and artificial intelligence based illumination / temperature control workflow service scenario. The workflow engine framework 10 receives the temperature sensing value from the in-building temperature sensor 81 and executes a predetermined workflow through the engine (s) created in the framework 10 to generate the smart bulb 82, So as to control the optimum illuminance and control the smart system cooling / heating unit 83 to perform optimum temperature control.

"80" is a table for predicting a future temperature change in the future, for example, one hour after the predicted model, by inputting data from the temperature sensor 81, estimating the illuminance setting suitable for the predicted temperature, And transmits illumination and temperature control artificial intelligence services for controlling the illumination of the smart bulb 82 and the temperature setting values of the smart system cooler 83. [

FIG. 12 is an explanatory diagram of a deep running-based traffic speed monitoring service scenario. The workflow engine framework 10 of the present invention receives speed information from a traffic speed sensor (simulator) 84 built at a traffic site in a specific area and transmits the speed information via the engine (s) It is possible to execute the predetermined workflow and provide the execution data to the traffic speed predicting intelligent service 85 built in the corresponding area so as to monitor the traffic data through the dashboard 87 or the smartphone screen 88 of the vehicle . The contents monitored through the vehicle 87 or the smartphone 88 are traffic speed information, for example, displayed on a map of a specific area such as '86'.

13 illustrates the internal structure of the workflow engine framework 10 of Fig. The engine generated by the workflow engine framework 10 includes an engine 52 and a serving engine 53 for Ingestion / data / prediction.

The components of the InGSeon stream / data / prediction engine 52 are as follows.

522: Stream input by opening web service port and reading incoming data

528: a controller (controller) for sequentially processing the input device 522, the operators 523a to 525f, and the output device 524 when a predetermined number of streams are read by reading the stream data,

523a: an operator (processor) for extracting a column in which the temperature value of data is located;

523b: an operator that normalizes values to a range of specified values

523c: an operator that converts a column into a column, or a column into a column

523d: an operator for predicting / judging the delivered value using a specified machine learning model

523e: an operator that returns a value to a range of specific values for denormalization

523f: an operator for extracting a specific column including a label value among various values generated as a result of prediction

524: writer writing final value to stream engine (for example, Kafka)

526: An executor providing an environment in which an operator can be executed.

The components of the serving engine 53 are as follows.

532: An input device that reads the values from the Stream Engine as needed

536: A Web server launcher (RestRunner) that opens a specific web port and waits for requests from end users.

538: When a request is received via a specific web port on the Web server running in the Rest Runner 536, the processing result of the Ingestion / Data / Prediction Engine 52 is read through the input unit 342, A controller that controls a series of flows to retrieve and respond to optimal values by referring to the database in combination with the context information values contained in the user request

When the temperature sensing value of the temperature sensor 81 is input to the InGSein stream / data / prediction engine 52 as a REST signal and processed through a plurality of processors designed through the system definition editor 70, And transmitted to the serving engine 53, and the smart bulb 82 is controlled according to the temperature sensing value of the temperature sensor 81 to perform the automatic illuminance adjustment.

14 shows an example of a configuration of a serving engine for providing an intelligent service as an engine for providing a service to a client or a user. The components of the input device 322, the plurality of pipeline processors 323a-c, the output device 324, the executor 326, and the controller 328 are similar to those described in Fig. However, the serving engine of Fig. 14 is configured for serving a service execution in the workflow service of Figs. 11 and 13. [ FIG. 14 shows an external system, Spark, Tensoflow, Jena (time), REST Server, Jena (space) and Jena (domain-specific) linked with each component.

 The controller of the serving engine reads the processed data from the engine constituted by the executor such as Spark or reads the processed data from the engine constituted by the deep learning framework such as tensor flow as the executor and performs ontology based reasoning using the value The intelligent service can be provided by configuring the engine with a processor and an executor that services the result through the REST interface.

Fig. 15 illustrates the internal configuration of the workflow engine framework 10 of Fig. When the time / speed / TRV is transmitted in real time to the engine created in the workflow engine framework 10 from the traffic sensor 84 of 1382 links in real time, it is read out to read the 24 hour series of link per time for the Tensorflow RNN operation and a prediction engine 55 for receiving 24 time series for each link for Tensorflow RNN operation and outputting a predicted value after 15 minutes.

The data engine 54 includes an input 542, a controller 548, an executor 546, and processors 543a-f, similar to FIGS. 5-8. The prediction engine 55 also includes an input unit 552 and a controller 558 similar to those of Figures 5 to 8 except that a tensor flow 555 for performing a circular neural network (RNN) ).

FIG. 16 shows a method for dynamically configuring a cross-domain adaptive workflow engine framework system by cooperatively connecting a plurality of single domain adaptive workflow engine frameworks 10 described above to each other via a network . The cross-domain convergence system 10 'for collectively managing and controlling one or more single domain adaptive workflow engine framework 10 in the case where the object of the workflow configuration of Fig. 1 corresponds to a cross domain , A cross-domain adaptive workflow engine framework can be implemented.

The cross-domain convergence system 10 'basically has a configuration and function similar to the single domain workflow engine framework 10 of FIG. 1 or FIG. 2, and functions to integrate and layer various domains. It is a system that manages resources related to flow execution, assigns a part of cross-domain workflow to each domain as workflow, and executes it on the framework of each domain. Accordingly, the cross-domain fusion system 10 'includes a cross-domain resource management unit 20', a cross-domain system configuration unit 30 ', and a cross-domain system control unit 40'.

Using the system definition editor 70 'of the concept described above, the user can define a workflow, which data acquisition to acquire from which sensor, which model to select, engine to be created, The cross-domain convergence system 10 'determines a domain to which the engine (s) necessary for the corresponding domain is to be distributed, according to the workflow defined in the system definition editor 70' do. In this case, the criterion for determining the engine distribution is, for example, which domain is optimal for use of a particular engine, whether the components for executing the engine exist in the corresponding domain, network load, performance, and distance within the network.

The cross domain resource management unit 20 'periodically receives workflow resources from the resource management unit 20 of a plurality of single domain workflow engine frameworks 10 (see FIG. 1 or 2), updates and manages the workflow resources . For example, the workflow resource may be at least one of a component resource for executing a workflow, network connection information, hardware resource information, specification information of a virtual machine, organizational configuration information, available device information of the organization, and availability status. Also, the cross-domain resource management unit 20 'periodically updates and maintains available components, hardware, network information, and device information in the domain through the resource information received from each domain. The resources managed by the resource management unit 20 'are, for example, a list of engine components for each domain, data for cost estimation, and reliability between domains.

In order to process the cross-domain workflow instance specification received from the system definition editor 70 ', the cross-domain system configuration unit 30' refers to the location of the available resources and the connection method from the cross-domain resource management unit 20 ' One cross domain workflow instance specification is divided into a plurality of single domain workflow specifications and distributed to each single domain workflow engine framework 10 and its execution is controlled.

Each cross-domain system control 40 'is created by one or more single domain workflow engine frameworks 10. That is, control of the execution of one or more engines distributed in the domain, to the system control unit 40 on the single domain workflow engine framework 10, respectively.

Each single domain workflow engine framework 10 registers its own list of resources in the cross-domain convergence system 10 'and periodically requests updates. Here, the list of resources may be a list of component resources for executing a workflow, and may include network connection information, hardware resource information, virtual machine specification information, organization configuration information, available device information of the organization, have.

The cross-domain resource management unit 20 'checks resource information received from each domain, that is, available components and hardware, network information, device information in the domain, etc., and defines a list of optimal engine layout and component configuration within the available range And requests execution of the workflow engine framework 10 of each domain. The workflow engine framework 10 of each domain constructs an engine assigned to each domain dynamically and drives the engine by executing an execution request to perform processing. In this case, the data transfer between the engines may be directly performed through the network driver as mentioned in FIG. 10, or may be carried out using one or more of various data paths such as a messaging system such as a distributed message queue and other distributed file systems.

The system control unit 40 of each single domain workflow engine framework 10 transmits signals such as the progress and completion of the engines to the cross domain system control unit 40 'of the cross domain fusion system 10'.

FIG. 17 shows an execution procedure for receiving the workflow specification in the cross-domain convergence system 10 'of FIG. 16 and effectively distributing and executing it. The workflow at this time may be a single domain or a cross-domain adaptive workflow depending on the problem or application to be solved.

First, the system definition editor 70 'defines a workflow specification including a group of engine component containers for determining a series of workflows and creating engines constituting the series of workflows by a workflow creation procedure (100). When the cross-domain convergence system 10 'receives the workflow specification from the system definition editor 70', the process of determining and distributing the distribution location of the engine component container (s) set in the workflow specification to the domain 200). When the specification of the engine (s) required on the workflow is distributed to the domain (s), all preparations for running a workflow for a single domain or for a cross domain are completed and finally the workflow is executed 300).

The process 200 for determining and distributing the distribution position of each engine component container in the cross-domain system configuration unit 30 'in the cross-domain fusion system 10' will be described in detail.

First, the first source of data required for each engine (the location of the data that the engine should collect such as a specific DB, the sensor location, etc.) and the final destination (for example, the data source for the other engine or the storage location) (210). In this case, the source and destination of the data may be object Internet devices, a structured data store such as a database, a file system, or the like, or may be a web service terminal.

If the source and destination of the data exist in the same domain, the resource management unit inquires 220 whether a series of engine configuration from the initial data source to the final destination is possible using the resources in the corresponding domain. If the engine configuration is possible (e.g., if all of the engine components are available in the resource registry of the domain), the engine specification is distributed 230 to dynamically configure the engine in the domain, .

If it is determined that the configuration of the engine component container is possible by using the resources in the corresponding domain in step 220, if it is determined that the data start point and the destination of the engine do not belong to the same domain, The process proceeds to a step 240 of searching for another configurable domain (searching for a resource manager) and determining an alternative candidate domain group.

When the candidate domain group is determined, cost estimation is performed to determine an alternative domain among them (250). The cost can be calculated by combining one or more of the following: the amount of network transmission costs, such as distance on the network topology, network throughput, expected latency, amount of available computing resources, and reliability across domains.

Next, the optimal domain to be distributed is selected 260 by applying the distribution domain selection policy (s) to the calculated cost. At this time, the domain selection policy may be a process allocation policy that allocates the same amount or the same amount of resources to each domain, an optimal energy policy considering energy efficiency, and a priority policy for prioritizing and allocating domain by domain. The order of steps in cost and policy is non-sequential.

Once all of the required engine specifications for the workflow are distributed to the domain (s) (230), all preparations for executing the workflow for the domain or cross-domain are completed and finally the workflow is executed 300 ). Execution instructions of the system control unit 40 'in the cross-domain fusion system 10' are transmitted to the system control unit 40 of each single domain framework 10 via the control channel and executed.

To further understand the cross-domain adaptive workflow engine framework of FIGS. 16 and 17, a specific related scenario and a diagram for explaining the system configuration (FIGS. 18 to 20) will be further described.

Figure 18 is intended to illustrate scenarios for achieving Smart City. First, Smart City can be achieved through optimal (smart) public energy supply and demand control, public traffic control, and optimal control in other areas. The optimal public energy supply and demand control can be achieved through optimal public energy demand control and supply control according to the public energy supply and demand policy. Here, smart public energy demand control can be achieved through optimal streetlight control and fog control (eg, control of fog removal devices) and other optimal public device controls. The optimal streetlight control can be achieved by referring to an optimal road traffic control policy according to predicted road traffic volume and predicted weather conditions. It is possible to predict the traffic volume by road using the traffic volume per road by referring to the monitoring result.

Figure 19 is a cross-domain workflow < RTI ID = 0.0 > Fig. 4 shows an embodiment of an engine framework. Fig.

First, the traffic volume monitoring includes an engine for collecting sensing data from various traffic sensors, a traffic statistical processing engine for reading the sensing information collected by the collection engine and performing operations for traffic statistics such as totals and averages, And a traffic volume statistical serving engine that responds to the result of the traffic amount statistic processing for the traffic amount monitoring processing workflow 91. [

Next, the road traffic volume prediction is performed by processing the traffic sensor information collection engine of the road traffic volume monitoring workflow 91 and the collected sensing information in the form of input data required for prediction, and reading the processed data to perform machine learning based prediction And a road traffic forecasting workflow 92 composed of a traffic forecasting serving engine that responds to a request for prediction according to a request from the outside can be dynamically configured.

The weather prediction includes a weather information collection engine for collecting and appropriately processing weather information, a weather forecasting engine for predicting weather based on a machine learning model, and a weather forecasting workflow comprising a weather forecasting serving engine for serving weather forecasting The target system can be configured by defining and executing the program 93.

The road traffic control policy is based on a context-based road traffic control policy that extracts the optimal road traffic control policy by using knowledge base or machine learning through inference or recommendation operation based on rules, A system for achieving the object can be configured by defining and executing a road traffic control workflow 90 including a recommendation engine.

The smart streetlight control provides an operation for querying the traffic forecasting serving engine and the weather forecasting serving engine described above to the above-mentioned road traffic control recommendation engine, A smart streetlight control workflow 94 that creates a smart streetlight control recommendation engine that responds to requests from outside, including operations for inferring control, can be configured and executed to configure the system to accomplish the objectives.

The scenarios shown in Fig. 18, which are not explained otherwise, can dynamically configure a system that achieves each objective by constructing each workflow in a manner similar to the above.

Figure 20 illustrates one embodiment of a smart streetlight control recommendation engine configuration in a workflow 94 configuration for smart streetlight control among the scenarios shown in Figure 19; The smart streetlight control recommendation engine includes a Runner component 326 for driving a RESTful Server 325 for responding to external requests and an operator for inquiring weather forecast results when a request is received by the RESTful server 325 An operator 323a for querying the weather forecasting engine 93 'to obtain results to be predicted, an operator 323c for inquiring the traffic amount prediction engine 92' Generates the situation information by receiving the obtained result, inquires the situation-specific road traffic control recommendation engine 90 'by using the generated situation information, obtains the traffic control for each road according to the situation, extracts the streetlight control policy, An operator 323b for transferring the data to the server 325 and a controller 328 for controlling the flow relationship of logic between the runner 326 and the operator.

17, the execution of the necessary engine component container group determination (100) for executing a series of workflows is performed, and a procedure for distribution position determination and distribution to each engine component container A cost estimation 250 and an optimal candidate domain determination 260, which serve as a basis for determining a replacement candidate domain, will be described for each scenario.

In the case of the road traffic control recommendation engine according to the situation in the road traffic control workflow 90, the recommendation result is directly linked to the streetlight control recommendation, so that engine distribution according to the weighting policy of the inter-domain reliability priority is performed.

For the real-time monitoring, the engines are deployed according to the network delivery cost and the weighting policy of the available computing resources in the engine distribution for the sensing data collection engine, the traffic volume statistics processing engine, and the traffic volume statistics serving engine in the road traffic monitoring workflow (91) do.

The Traffic Estimation Engine and the Traffic Estimation Serving Engine of the traffic flow monitoring workflow of the road traffic forecasting workflow (92) must predict the network cost with the domain where the collection engine is deployed because the forecasting should be performed using the collection result of the collection engine First, distribute the engine.

Since weather information is not important for security in the weather forecast workflow 93, the engine is distributed to the available computing resources first.

In the smart streetlight control workflow 94 including the smart streetlight control recommendation engine, since each query target engine is distributed to other domains and contains important information, the engine is distributed according to the inter-domain reliability priority weighting policy.

In the real-time analysis service of mass data that combines and analyzes the object Internet, big data, and machine learning through the above-described configuration and process, a component is implemented in the machine learning model and the big data analysis model developed for solving a specific problem Can be managed. In addition, a single domain or cross-domain adaptive workflow engine framework capable of dynamically reusing components can be implemented to easily configure an execution platform for solving similar problems in various business or destination domains.

Claims (20)

A resource management unit for managing a resource including a workflow attribute specification component and an engine component necessary for execution of a workflow defined by a user,
A system configuration unit for assembling the attribute specification components and dynamically combining the engine components necessary for executing the workflow to construct the necessary engine component containers according to the workflow specification to generate the engine,
And a system control unit for controlling execution by driving one or more engines generated by the system configuration unit in accordance with the manner defined in the workflow attribute specification. The workflow engine framework according to claim 1, further comprising an execution instance portion in which an engine configured by a combination of engine components dynamically generated by the system configuration portion is stored in the form of an engine instance. The apparatus of claim 1, wherein the resource management unit
An attribute specification component including an attribute specification for determining a component property or attribute of the workflow instance, and an attribute specification for managing the attribute specification component,
A workflow engine framework that includes an engine component for execution and an engine component manager that manages the list. 4. The apparatus of claim 3, wherein the resource management unit
A workflow engine framework further comprising a workflow specification instance management unit for managing previously created and stored workflow specification instances. The system of claim 1,
A workflow attribute specification assembly for binding a workflow specification to an attribute specification component to create a series of attribute specification components,
And a workflow configuration unit for configuring a workflow execution platform by configuring a series of engines by extracting the defined engine component information from the assembled attribute specification component and binding the attribute specification component and the engine component, work. The method of claim 5, wherein the workflow constructing unit
An engine for executing a workflow by binding an engine component and an attribute specification component in accordance with the specification of an attribute specification component that defines the engine component that constitutes the workflow and the parameter that determines the characteristics of the engine component is stored in the form of an engine instance A workflow engine framework that further includes a workflow execution instance portion. 2. The system of claim 1, wherein the workflow specification is defined by a system definition editor by a user and submitted to the system configuration in the form of a workflow specification file,
An engine type selection unit for presenting various engine types and allowing the user to select necessary ones among them,
A component selection unit for providing various engine component lists for each component type so that a user can select a component type and an engine component for each type,
And a component attribute selection / editing unit for presenting the attributes of the engine component selected by the component selection unit and allowing the user to select and edit the attribute. 2. The engine of claim 1,
An input device for fetching data from one or more data origin,
One or more unit processors for receiving and processing data from the input device,
One or more output devices for outputting internally processed data to one or more data destinations,
A separate execution program for processing input data or a launcher for executing a platform or managing a session,
And a controller for inputting such data through an input device, for processing on a launcher and for controlling a series of outputs for outputting the data. 9. The apparatus of claim 8,
A sequential processing method in which, when nodes are configured in the order of the input unit, the unit processor, and the output unit, sequential pipelining of data and transfer to the next node are performed; a simultaneous processing method , And a concurrent / sequential processing method that combines the two. 9. The apparatus of claim 8, wherein the controller
Wherein one or more unit processors receive data from the input unit and sequentially process data by pipelining and at least one unit processor is driven to execute simultaneous processing. 9. The method of claim 8,
Wherein the workflow engine framework includes one or more of the workflow execution engines,
In this case, a data transfer path is used between the plurality of engines so that the destination of the data to be output from the output device of one engine is transferred in a pipeline manner to the data origin of the input device of the other engine. 12. The workflow engine framework of claim 11, wherein the one or more engines are located in different physical environments and include different types of launchers. 12. The workflow engine framework of claim 11, wherein the data delivery path comprises one or more different types of data delivery paths, pipelining by data origin and destination. A resource management unit for managing a resource including a workflow attribute specification component and an engine component necessary for execution of a workflow defined by a user; A system configuration unit for assembling the attribute specification components and dynamically combining the engine components necessary for executing the workflow to construct the engine component containers according to the workflow specification to generate the engine; At least one single domain workflow engine framework comprising a system controller for driving one or more engines generated by the system component according to a manner defined in a workflow attribute specification to control execution,
A cross domain convergence system (10) for determining a single domain to be connected to the two or more single domain workflow engine frameworks to distribute an engine required for each single domain included in the cross domain according to a user defined cross domain workflow &Quot;).≪ / RTI > 15. The method of claim 14, wherein the cross-domain fusion system comprises
It periodically reports the workflow resources from the resource manager of the single domain workflow engine framework, updates and manages the workflow resources, periodically collects available components, hardware, network information, and device information in the domain through resource information received from each domain. A cross domain resource management unit for updating and maintaining,
Domain cross-domain workflow specification to divide one cross-domain workflow specification into a plurality of single domain workflow specifications to determine and distribute a deployment location of each engine component container to handle a cross-domain workflow specification, Workflow Engine Framework. 15. The method of claim 14, wherein the cross-domain fusion system is adapted to determine a single domain to distribute the engine needed for each single domain included in the cross domain
1) means for determining whether the data required for each engine exists in the same domain,
2) means for inquiring of the resource management unit whether an engine configuration from an initial data source to a final destination is possible using resources in the domain if the data source and destination of the engine are in the same domain;
3) means for distributing the engine specification to dynamically configure the engine in the domain if possible,
4) If the data source and destination of the engine do not exist in the same domain as a result of the judgment of the means 1) or if it is judged that the engine configuration is impossible in the means 2), another domain capable of the same engine component configuration is searched and replaced Means for determining a possible candidate domain group,
5) means for performing cost estimation to determine an alternative domain among the candidate domain groups if they are determined,
6) A cross-domain adaptive workflow engine framework that includes means for selecting an optimal domain to deploy by applying distribution domain selection policies. 15. The system of claim 14 wherein the workflow specification is defined by a user by a system definition editor and submitted to the cross domain convergence system in the form of a workflow specification file,
An engine type selection unit for presenting various engine types and allowing the user to select necessary ones among them,
A component selection unit for providing various engine component lists for each component type so that a user can select a component type and an engine component for each type,
A cross-domain adaptive workflow engine framework including a component attribute selection / editing unit that presents attributes of engine components selected by the component selection unit and allows the user to select and edit the attributes. 15. The engine of claim 14,
An input device for fetching data from one or more data origin,
One or more unit processors for receiving and processing data from the input device,
One or more output devices for outputting internally processed data to one or more data destinations,
A separate execution program for processing input data or a launcher for executing a platform or managing a session,
A cross domain adaptive workflow engine framework comprising a controller for inputting such data through an input device and for processing on a launcher and for a series of controls for outputting the same. A method for constructing a workflow execution engine by assembling workflow attribute specification components required for execution of a workflow defined by a user and organizing engine component containers by dynamically combining engine components required for execution of a workflow,
Receiving a workflow specification and generating an attribute specification component instance containing an attribute specification for engine components to configure the engine,
Generating an engine component instance by designating the generated attribute specification component instance as a generator parameter of the engine component;
Creating an engine by binding to an engine component container and using an engine component container instance as a constructor parameter. The method of claim 14, wherein the cross-domain convergence system included in the cross-domain adaptive workflow engine framework is a method for determining a single domain to deploy an engine needed for each single domain included in a cross domain,
1) determining whether the source and destination of data necessary for each engine exists in the same domain,
2) If the data source and the destination of the engine exist in the same domain, the step of inquiring and determining whether the engine configuration from the initial data source to the final destination is possible using the resources in the corresponding domain,
3) if engine configuration is possible, distributing the engine specification to dynamically configure the engine in the domain,
4) If it is determined in step 1) that the data origin and the destination of the engine do not exist in the same domain, or if it is determined in step 2) that the engine configuration is impossible, another domain capable of the same engine component configuration is searched Determining an alternate candidate domain group,
5) If the candidate domain group is determined, performing cost estimation to determine an alternative domain among them,
6) applying a distribution domain selection policy to select an optimal domain to distribute.

Images