CN113190328A - System identification-oriented containerized cloud workflow processing system and method - Google Patents
System identification-oriented containerized cloud workflow processing system and method Download PDFInfo
- Publication number
- CN113190328A CN113190328A CN202110561275.2A CN202110561275A CN113190328A CN 113190328 A CN113190328 A CN 113190328A CN 202110561275 A CN202110561275 A CN 202110561275A CN 113190328 A CN113190328 A CN 113190328A
- Authority
- CN
- China
- Prior art keywords
- task
- task sequence
- workflow
- cloud
- output data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 12
- 238000013500 data storage Methods 0.000 claims abstract description 22
- 238000007726 management method Methods 0.000 claims abstract description 14
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 238000012544 monitoring process Methods 0.000 claims description 23
- 230000007246 mechanism Effects 0.000 claims description 9
- 238000003672 processing method Methods 0.000 claims description 7
- 238000004519 manufacturing process Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013479 data entry Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
- G06F8/63—Image based installation; Cloning; Build to order
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Abstract
The invention relates to a containerization cloud workflow processing system and method facing system identification, wherein the system comprises: the edge node is used for collecting input and output data generated by the operation of a controlled object; the mirror image warehouse is used for storing a plurality of system-identified workflow templates; the computing resources include containers created by the kubernets system; the preprocessing module is used for receiving the workflow template and the input and output data, analyzing the input and output data according to the workflow template and generating a task sequence; the task management module is used for receiving the task sequence and releasing the task to the cloud resource pool according to the task sequence; the Kubernetes system is used for receiving the task sequence and creating a container according to the task sequence; the container is used for executing the task sequence and obtaining a system identification result; and the Redis data storage system is used for storing input and output data and a system identification result. The cloud computing method and the cloud computing system have the advantages that the cloud computing is used, and the processing speed of system identification is improved.
Description
Technical Field
The invention relates to the technical field of system identification, in particular to a containerization cloud workflow processing system and method for system identification.
Background
System identification is the process of mathematical modeling of a dynamic system using measured data, and the model established can be used for system analysis, performance monitoring and diagnostics, prediction, optimization, and system design and control. However, the system identification is directly based on a large amount of data, and the calculation speed is limited, so that the system identification is not suitable for processing large-scale identification tasks or identification tasks with real-time constraints. Meanwhile, in order to break the data barrier and mine the depth value of the data, mass data are transmitted to the cloud for storage, sharing, analysis and processing. The current system identification research only considers improving the algorithm structure and improving the algorithm performance, and the processing speed is still to be improved.
Disclosure of Invention
The invention aims to provide a containerization cloud workflow processing system and method for system identification, which improve the processing speed.
In order to achieve the purpose, the invention provides the following scheme:
a system-identification oriented containerized cloud workflow processing system, comprising: the system comprises an edge node, a mirror image warehouse, a cloud resource pool, a preprocessing module, a task management module, a Redis data storage system and a Kubernetes system;
the edge node is used for collecting input and output data generated by the operation of a controlled object;
the mirror image warehouse is used for storing a plurality of workflow templates identified by the system;
the cloud resource pool is used for providing computing resources; the computing resource comprises a container created by the Kubernets system;
the preprocessing module is used for receiving the workflow template and the input and output data, analyzing the input and output data according to the workflow template and generating a task sequence;
the task management module is used for receiving the task sequence and releasing a task to the cloud resource pool according to the task sequence;
the Kubernetes system is used for receiving the task sequence and creating a container according to the task sequence; the container is used for executing the task sequence to obtain a system identification result;
the Redis data storage system is used for storing the input and output data and the system identification result.
Optionally, the system identification-oriented containerized cloud workflow processing system further comprises a monitoring module;
the monitoring module is used for acquiring the consumption of computing resources and the execution state of the task sequence from the Kubernets system in real time.
Optionally, the task management module is further configured to receive, from the monitoring module, the computing resource usage and the execution state of the task sequence in real time; and creating a container according to the computing resource usage, the execution state of the task sequence and the task sequence.
Optionally, the monitoring module includes a resource status tracker and a task status tracker;
the resource state tracker is used for acquiring the consumption of computing resources from the Kubernets system in real time by using a List-watch mechanism;
the task state tracker is used for acquiring the execution state of a task sequence from the Kubernets system in real time by using a List-watch mechanism.
Optionally, the preprocessing module is further configured to decode the task sequence to obtain a plurality of task information, and write each of the task information into a first YAML file;
the first YAML file comprises a task sequence number, a hierarchy of task information in a workflow, a dependency relationship between the task information and a previous task and a subsequent task, and a task mirror image corresponding to the task information.
Optionally, the monitoring module is further configured to write the computing resource usage and the execution state of the task sequence into a second YAML file, and send the second YAML file to the kubernets system.
Optionally, each edge node includes a plurality of controlled objects therein.
Optionally, the system-identification-oriented containerized cloud workflow processing system further comprises a user node;
the user node is used for receiving the system identification result.
The invention also discloses a containerized cloud workflow processing method facing system identification, which is applied to the containerized cloud workflow processing system facing system identification, and comprises the following steps:
manufacturing a plurality of workflow templates identified by the system, and uploading the mirror images of the workflow templates to a mirror image warehouse; the workflow template is a directed acyclic graph with a front-back dependency relationship;
acquiring input and output data generated by the operation of a controlled object, and storing the input and output data into a Redis data storage system through a gateway;
pulling a workflow template corresponding to the input and output data from the mirror image warehouse;
analyzing the input and output data according to the workflow template to generate a task sequence;
creating a container according to the task sequence through a Kubernetes system;
releasing tasks to a cloud resource pool according to the task sequence;
executing the task sequence through the container to obtain a system identification result;
storing the system identification result to the Redis data storage system.
Optionally, the creating, by the kubernets system, a container according to the task sequence specifically includes:
acquiring the consumption of computing resources and the execution state of a task sequence in real time from the Kubernetes system;
and creating a container according to the computing resource usage, the execution state of the task sequence and the task sequence.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
according to the method, the input and output data of the controlled object are analyzed according to the workflow template pulled from the mirror image warehouse, the task sequence is generated, the Kubernets system is utilized, the container is created according to the task sequence, the task sequence is executed through the container, the system identification result is obtained, cloud computing is utilized, and the processing speed of system identification is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a schematic diagram of a containerized cloud workflow processing system for system identification according to the present invention;
FIG. 2 is a schematic structural diagram of an embodiment of a containerized cloud workflow processing system for system identification according to the present invention;
FIG. 3 is a diagram illustrating a resource layer structure according to the present invention;
FIG. 4 is a schematic diagram illustrating specific relationships among modules in a containerized cloud workflow processing system for system identification according to the present invention;
fig. 5 is a schematic flow chart of a containerized cloud workflow processing method for system identification according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a containerization cloud workflow processing system and method for system identification, which improve the processing speed.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a schematic structural diagram of a containerized cloud workflow processing system facing system identification according to the present invention, and as shown in fig. 1, the containerized cloud workflow processing system facing system identification includes: edge node 101, mirror repository 102, cloud resource pool 107, pre-processing module 103, task management module 106, Redis data storage system 104, and Kubernets system 105.
The edge node 101 is configured to collect input and output data generated by the operation of the controlled object. The edge nodes 101 are nodes distributed at different edge positions, equipped with limited resource devices, and connected with controlled objects.
The mirror repository 102 is configured to store a plurality of system-recognized workflow templates. The mirror store 102 is a shared store that stores reusable task mirrors. The workflow templates include templates of system-recognized task images and workflow topology dependencies.
The cloud resource pool 107 is used for providing computing resources; the computing resources include containers created by the kubernets system 105. Cloud resource pool 107 includes physical machines, virtual machines, and containers.
The preprocessing module 103 is configured to receive the workflow template and the input/output data, analyze the input/output data according to the workflow template, and generate a task sequence.
The task management module 106 is configured to receive the task sequence, and release a task to the cloud resource pool 107 according to the task sequence.
The Kubernetes system 105 is used for receiving the task sequence and creating a container according to the task sequence; the container is used for executing the task sequence and obtaining a system identification result.
The Redis data storage system 104 is used for storing the input and output data and the system identification result.
The containerized cloud workflow processing system facing the system identification further comprises a monitoring module;
the monitoring module is used for acquiring the computing resource usage and the execution state of the task sequence from the Kubernets system 105 in real time.
The task management module 106 is further configured to receive, from the monitoring module, the amount of computing resources and the execution state of the task sequence in real time; and creating a container according to the computing resource usage, the execution state of the task sequence and the task sequence.
The monitoring module comprises a resource state tracker and a task state tracker;
the resource state tracker is used for acquiring the calculation resource usage from the Kubernets system 105 in real time by using a List-watch mechanism;
the task state tracker is used for acquiring the execution state of a task sequence from the Kubernets system 105 in real time by using a List-watch mechanism.
The preprocessing module 103 is further configured to decode the task sequence to obtain a plurality of task information, and write each of the task information into a first YAML file;
the first YAML file comprises a task sequence number, a hierarchy of task information in a workflow, a dependency relationship between the task information and a previous task and a subsequent task, and a task mirror image corresponding to the task information.
The monitoring module is further configured to write the amount of computing resources and the execution state of the task sequence into a second YAML file, and send the second YAML file to the kubernets system 105.
Each edge node 101 includes a plurality of controlled objects therein.
The containerized cloud workflow processing system facing the system identification further comprises a user node;
the user node is used for receiving the system identification result. The user node is a node where a user performing various operations remotely locates.
As shown in fig. 2, the containerized cloud workflow processing system facing system identification of the present invention includes three layers: a cloud platform layer, a terminal layer (edge nodes and user nodes), and a resource layer.
Cloud platform layer: after receiving the input/output data of the controlled object from the edge node 101, the data collection module transmits the input/output data to the workflow data entry module. Meanwhile, the system identification workflow template is pulled from the mirror repository 102 and is imported into the containerized cloud workflow processing system through the workflow template entry (fig. 2 star mark module). Containers are created by the containerized cloud workflow processing system and allocated onto distributed computing nodes in cloud resource pool 107. Finally, the identification model (system identification result) is sent to the user node through the network. The input and output data specifically include input data and output data.
In the resource layer, a K system (Kubernetes system) is controlled to create a container by interaction of a containerization cloud workflow processing system and the K system in the resource layer.
A terminal layer: including edge nodes 101 and user nodes. Each node is a relatively independent system, distributed over different geographical locations. Each edge node 101 includes a plurality of controlled objects, but the computing power of the edge devices is limited due to cost, space, and energy consumption. The output data of the controlled object is sensed by the sensor, recorded together with the input information actually applied to the controlled object, and transmitted to the cloud platform layer through the network for processing. At the user node, the received recognition model can be applied to various functions such as system monitoring, data-driven control, fault detection and the like.
Resource layer: the cloud service management system is responsible for managing bottom computing resources, collecting and providing computing resources for cloud services, and the specific structure of the resource layer is shown in fig. 3. The bottommost resource is a physical server distributed in each data center; based on KVM (kernel-based virtual machine) technology, the physical server is virtualized into more virtual machines, which will be as the running nodes (nodes) of the container (Pod); further, the workflow container is identified using the kubernets system 105 creation, management and orchestration system. Finally, the underlying computing resources are packaged and provided to the containerized cloud system identification service. In addition, the image repository 102, the cloud platform security module and the monitoring module are connected through API interfaces.
The containerization cloud workflow processing system is a core part of a cloud platform layer, and various data are led into the containerization cloud workflow processing system through an API (application programming interface) inlet, a workflow template inlet and a workflow data inlet. The system consists of a preprocessing module 103 (preprocessing part), a task management module 106 (task management part) and a monitoring module (monitoring part).
The containerized cloud workflow processing system generates and submits a resource request file to the kubernets system 105, which creates and schedules containers in the cloud resource pool 107 by the kubernets system 105. The three components of the containerized workflow processing system and the design in the Kubernetes system 105 are described separately below.
A pretreatment part: the part receives and processes the workflow template and the input and output data from the outer layer, and converts the workflow template and the input and output data into the form required by the subsequent part. And the workflow analyzer of the preprocessing part is responsible for analyzing the workflow template, generating a task sequence and decoding task information. These task information will be written by the task manager to the YAML file (first YAML file), which includes:
1. task sequence number and hierarchy in the workflow;
2. the dependency relationship between the task predecessor and the task successor;
3. the task needs to pull the image from the image repository 102.
Meanwhile, input and output data of the controlled object are stored in the Redis data storage system 104, and the Redis data storage system 104 can update and share data among different hosts online. When a workflow executes, data is imported to the container through the Redis data storage system 104, indexed by the task number.
The task manager part: this part is responsible for controlling the release of tasks into cloud resource pool 107. The task release controller is the core of the task manager part and executes the release strategy of the workflow task.
And a resource state tracker and a task state manager of the monitoring part provide the resource usage and the task execution state in the current execution environment for the task release controller, so that a subsequent task release decision is supported. Finally, the output policy is converted into the YAML file format of the kubernets system 105 standard by the result transcoder, and the generated YAML file (second YAML file) is submitted to the kubernets system 105.
A monitoring part: the part comprises a resource state tracker and a task state tracker, and the resource usage and the task execution state are acquired in real time from the Kubernets system 105 by using a List-watch mechanism, and specifically comprise the following steps:
the resource state tracker: and acquiring the resource usage of each node in the cloud resource pool 107, and providing the resource usage to the task release controller.
Task state tracker: the execution state of the containers in the node is monitored, including run, completion, and execution failures. When the task state changes, the task release controller is notified.
Design in kubernets system 105: the resource manager (Master) is responsible for collecting computing resources and creating new containers in each node. The original data is imported into the container through the Redis data storage system 104, and the final result is also stored in the system and finally output to the outside.
The invention discloses a containerized cloud workflow processing method facing system identification, which is applied to the containerized cloud workflow processing system facing system identification, and as shown in fig. 5, the method comprises the following steps:
step 201: manufacturing a plurality of workflow templates identified by the system, and uploading the mirror images of the workflow templates to a mirror image warehouse; the workflow template is a directed acyclic graph with a front-back dependency relationship.
Step 202: and acquiring input and output data generated by the operation of the controlled object, and storing the input and output data into a Redis data storage system through a gateway.
Step 203: and pulling the workflow template corresponding to the input and output data from the mirror image warehouse.
Step 204: and analyzing the input and output data according to the workflow template to generate a task sequence.
Step 205: creating a container from the task sequence via a Kubernetes system.
Step 206: and releasing the task to the cloud resource pool according to the task sequence.
Step 207: and executing the task sequence through the container to obtain a system identification result.
Step 208: storing the system identification result to the Redis data storage system.
Wherein, step 205 specifically includes:
and acquiring the consumption of computing resources and the execution state of the task sequence from the Kubernets system in real time.
And creating a container according to the computing resource usage, the execution state of the task sequence and the task sequence.
The following describes a containerized cloud workflow processing method oriented to system identification according to a specific embodiment, which specifically includes the following steps:
s1, a user makes a system identification workflow template, converts system identification into a directed acyclic graph with a front-back dependency relationship, namely a workflow form, and uploads the directed acyclic graph to a mirror image warehouse.
And S2, operating the controlled object of the edge node and generating original input and output data.
And S3, uploading the original input and output data to a data collection module at the cloud end through a gateway, and importing the original input and output data to a Redis data storage system in the containerized cloud workflow processing system through a workflow data inlet.
And S4, pulling the system identification workflow template from the mirror image warehouse by the workflow template inlet, and leading the workflow template into the containerization cloud workflow processing system.
And S5, in the containerized cloud workflow processing system, receiving the workflow template by the workflow analyzer, generating a task sequence, decoding task information, and transmitting the information to the task release manager.
And S6, receiving the original input and output data of the controlled object by the Redis data storage system, and sharing the original input and output data to a container for executing the identification task of the system through a network. The container acquires the task sequence number from the YAML file analyzed by the workflow template, and then extracts the required data from the Redis data storage system according to the task sequence number.
And S7, the resource state tracker acquires the resource usage of each node in the cloud resource pool by using a List-watch mechanism and provides the resource usage for the task release controller.
S8, the task state tracker monitors the execution state of the container in the node by using a List-watch mechanism, wherein the execution state comprises operation, completion and execution failure. When the task state changes, the task release controller is notified.
And S9, the task release manager receives a task sequence waiting for release, resource consumption of each node in the cloud resource pool and a current task execution state, generates a task release command, generates a YAML file through a result transcoder and transmits the YAML file to the Kubernetes system.
S10, the resource distributor (Master) receives the YAML file generated in the step S9, collects computing resources, creates a container according to the pulled mirror image (the resource distributor puts forward a request for pulling the mirror image to the mirror image warehouse, the mirror image warehouse transmits the mirror image to the target computing node selected by the resource distributor through the network, and creates a container in the target computing node according to the pulled mirror image), and distributes the newly created container to each target computing node. The inlet container reads original data from the Redis data storage system to start running, and the outlet container stores the identification result into the Redis data storage system and finally outputs the identification result to the outside.
And S11, sending the identification result to a remote user node through a network by the data transmission module. And the user node performs monitoring, control, fault detection and other operations according to the system identification result.
The containerization cloud workflow processing method facing system identification can be based on cloud workflow processing and container technology, and greatly improves the processing efficiency of system identification tasks.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (10)
1. A system-identification-oriented containerized cloud workflow processing system, comprising: the system comprises an edge node, a mirror image warehouse, a cloud resource pool, a preprocessing module, a task management module, a Redis data storage system and a Kubernetes system;
the edge node is used for collecting input and output data generated by the operation of a controlled object;
the mirror image warehouse is used for storing a plurality of workflow templates identified by the system;
the cloud resource pool is used for providing computing resources; the computing resource comprises a container created by the Kubernets system;
the preprocessing module is used for receiving the workflow template and the input and output data, analyzing the input and output data according to the workflow template and generating a task sequence;
the task management module is used for receiving the task sequence and releasing a task to the cloud resource pool according to the task sequence;
the Kubernetes system is used for receiving the task sequence and creating a container according to the task sequence; the container is used for executing the task sequence to obtain a system identification result;
the Redis data storage system is used for storing the input and output data and the system identification result.
2. The system-recognition oriented containerized cloud workflow processing system of claim 1, further comprising a monitoring module;
the monitoring module is used for acquiring the consumption of computing resources and the execution state of the task sequence from the Kubernets system in real time.
3. The system identification-oriented containerized cloud workflow processing system of claim 2, wherein the task management module is further configured to receive computing resource usage and an execution status of a sequence of tasks from the monitoring module in real-time; and creating a container according to the computing resource usage, the execution state of the task sequence and the task sequence.
4. The system-recognition oriented containerized cloud workflow processing system of claim 2, wherein the monitoring module comprises a resource state tracker and a task state tracker;
the resource state tracker is used for acquiring the consumption of computing resources from the Kubernets system in real time by using a List-watch mechanism;
the task state tracker is used for acquiring the execution state of a task sequence from the Kubernets system in real time by using a List-watch mechanism.
5. The system-recognition-oriented containerized cloud workflow processing system of claim 1, wherein the preprocessing module is further configured to decode the task sequence, obtain a plurality of task information, and write each of the task information into a first YAML file;
the first YAML file comprises a task sequence number, a hierarchy of task information in a workflow, a dependency relationship between the task information and a previous task and a subsequent task, and a task mirror image corresponding to the task information.
6. The system-recognition oriented containerized cloud workflow processing system of claim 1, wherein the monitoring module is further configured to write computing resource usage and execution status of task sequences to a second YAML file and send the second YAML file to the kubernets system.
7. The system-recognition oriented containerized cloud workflow processing system of claim 1 wherein each edge node includes a plurality of controlled objects therein.
8. The system-recognition oriented containerized cloud workflow processing system of claim 1, further comprising a user node;
the user node is used for receiving the system identification result.
9. A system identification-oriented containerized cloud workflow processing method, applied to the system identification-oriented containerized cloud workflow processing system according to any one of claims 1 to 8, the method comprising:
manufacturing a plurality of workflow templates identified by the system, and uploading the mirror images of the workflow templates to a mirror image warehouse; the workflow template is a directed acyclic graph with a front-back dependency relationship;
acquiring input and output data generated by the operation of a controlled object, and storing the input and output data into a Redis data storage system through a gateway;
pulling a workflow template corresponding to the input and output data from the mirror image warehouse;
analyzing the input and output data according to the workflow template to generate a task sequence;
creating a container according to the task sequence through a Kubernetes system;
releasing tasks to a cloud resource pool according to the task sequence;
executing the task sequence through the container to obtain a system identification result;
storing the system identification result to the Redis data storage system.
10. The system-identification-oriented containerized cloud workflow processing method according to claim 1, wherein the creating a container according to the task sequence by a kubernets system specifically comprises:
acquiring the consumption of computing resources and the execution state of a task sequence in real time from the Kubernetes system;
and creating a container according to the computing resource usage, the execution state of the task sequence and the task sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110561275.2A CN113190328A (en) | 2021-05-22 | 2021-05-22 | System identification-oriented containerized cloud workflow processing system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110561275.2A CN113190328A (en) | 2021-05-22 | 2021-05-22 | System identification-oriented containerized cloud workflow processing system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113190328A true CN113190328A (en) | 2021-07-30 |
Family
ID=76984660
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110561275.2A Pending CN113190328A (en) | 2021-05-22 | 2021-05-22 | System identification-oriented containerized cloud workflow processing system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113190328A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114895701A (en) * | 2022-04-18 | 2022-08-12 | 深圳织算科技有限公司 | Unmanned aerial vehicle inspection method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106888254A (en) * | 2017-01-20 | 2017-06-23 | 华南理工大学 | A kind of exchange method between container cloud framework based on Kubernetes and its each module |
| US20200218798A1 (en) * | 2019-01-03 | 2020-07-09 | NeuVector, Inc. | Automatic deployment of application security policy using application manifest and dynamic process analysis in a containerization environment |
| CN111897622A (en) * | 2020-06-10 | 2020-11-06 | 中国科学院计算机网络信息中心 | High-throughput computing method and system based on container technology |
| CN112783653A (en) * | 2021-01-25 | 2021-05-11 | 中国工商银行股份有限公司 | Resource scheduling method and device based on containerization |
-
2021
- 2021-05-22 CN CN202110561275.2A patent/CN113190328A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106888254A (en) * | 2017-01-20 | 2017-06-23 | 华南理工大学 | A kind of exchange method between container cloud framework based on Kubernetes and its each module |
| US20200218798A1 (en) * | 2019-01-03 | 2020-07-09 | NeuVector, Inc. | Automatic deployment of application security policy using application manifest and dynamic process analysis in a containerization environment |
| CN111897622A (en) * | 2020-06-10 | 2020-11-06 | 中国科学院计算机网络信息中心 | High-throughput computing method and system based on container technology |
| CN112783653A (en) * | 2021-01-25 | 2021-05-11 | 中国工商银行股份有限公司 | Resource scheduling method and device based on containerization |
Non-Patent Citations (1)
| Title |
|---|
| 夏元清 等: "绿色能源互补智能电厂云控制系统研究", 《自动化学报》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114895701A (en) * | 2022-04-18 | 2022-08-12 | 深圳织算科技有限公司 | Unmanned aerial vehicle inspection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110888722B (en) | Task processing method and device, electronic equipment and computer readable storage medium | |
| Dai et al. | Cloud service reliability: Modeling and analysis | |
| CN110083455B (en) | Graph calculation processing method, graph calculation processing device, graph calculation processing medium and electronic equipment | |
| Talagala et al. | {ECO}: Harmonizing Edge and Cloud with {ML/DL} Orchestration | |
| Hu et al. | Cloudroid: A cloud framework for transparent and QoS-aware robotic computation outsourcing | |
| Helu et al. | Scalable data pipeline architecture to support the industrial internet of things | |
| CN108306804A (en) | A kind of Ethercat main station controllers and its communication means and system | |
| Antonini et al. | Fog computing architectures: A reference for practitioners | |
| CN110941421A (en) | Development machine learning device and using method thereof | |
| CN102566443A (en) | Simulation verification system and method for integrated avionics system model based on artifact design description language (ADDL) | |
| US20210326184A1 (en) | Dynamically allocated cloud worker management system and method therefor | |
| CN112130812A (en) | Analysis model construction method and system based on data stream mixed arrangement | |
| Roman et al. | Big data pipelines on the computing continuum: tapping the dark data | |
| CN113190328A (en) | System identification-oriented containerized cloud workflow processing system and method | |
| CN112148926A (en) | Graph data stream processing method, processing device and storage medium | |
| US11658878B2 (en) | Systems and methods for automated network state and network inventory tracking | |
| CN110766163B (en) | System for implementing machine learning process | |
| Faro et al. | Middleware for Quantum: An orchestration of hybrid quantum-classical systems | |
| CN112817706B (en) | Distributed task scheduling system and method | |
| Golis et al. | Creating microservices and using infrastructure as code within the CI/CD for dynamic container creation | |
| CN102819508B (en) | Method for responding to signals from security monitoring subsystem | |
| Theiss et al. | A Java software agent framework for hard real-time manufacturing control | |
| Moser et al. | Making expert knowledge explicit to facilitate tool support for integrating complex information systems in the atm domain | |
| Liu et al. | A cloud control robotics platform based on intelligent deployment of micro-services | |
| WO2023164822A1 (en) | Dynamic workflow implementation method and system, medium, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210730 |
|
| RJ01 | Rejection of invention patent application after publication |