WO2024139011A1 - Information processing method - Google Patents

Information processing method Download PDF

Info

Publication number
WO2024139011A1
WO2024139011A1 PCT/CN2023/094144 CN2023094144W WO2024139011A1 WO 2024139011 A1 WO2024139011 A1 WO 2024139011A1 CN 2023094144 W CN2023094144 W CN 2023094144W WO 2024139011 A1 WO2024139011 A1 WO 2024139011A1
Authority
WO
WIPO (PCT)
Prior art keywords
node
workflow
processed
processing
information
Prior art date
Application number
PCT/CN2023/094144
Other languages
French (fr)
Chinese (zh)
Inventor
朱劲松
罗强
Original Assignee
上海爱数信息技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海爱数信息技术股份有限公司 filed Critical 上海爱数信息技术股份有限公司
Publication of WO2024139011A1 publication Critical patent/WO2024139011A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation

Definitions

  • the present application provides an information processing method, which can improve the versatility of the information processing method while ensuring data consistency.
  • an embodiment of the present application provides an information processing method, which is applied to an information processing system, wherein the information processing system includes a workflow management module, an orchestration engine module, and a processing module, and the method includes:
  • the workflow management module After receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, where the target workflow is the workflow corresponding to the processing request;
  • the first information to be processed is processed by the processing module to obtain a first processing result.
  • an embodiment of the present application provides an electronic device, the electronic device comprising one or more A processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and when executed by the one or more processors, the above method is performed.
  • FIG2 is a structural diagram of a workflow definition model provided according to Embodiment 1 of the present application.
  • FIG3 is a structural diagram of a node model provided according to Embodiment 1 of the present application.
  • FIG4 is a flow chart of an information processing method provided according to Embodiment 2 of the present application.
  • FIG5 is a structural diagram of an information processing method provided according to Embodiment 2 of the present application.
  • FIG6 is a structural diagram of a workflow provided according to Embodiment 2 of the present application.
  • FIG. 7 is a schematic diagram of the hardware structure of an electronic device provided according to Embodiment 3 of the present application.
  • Figure 1 is a flowchart of an information processing method provided according to Example 1 of the present application. This embodiment can be applicable to the situation of processing the system.
  • the method can be executed by an information processing device.
  • the information processing device can be implemented in the form of hardware and/or software.
  • the information processing device can be configured in a processing system.
  • AnyShare includes three subsystems: Object Storage Service (OSS) gateway, MongoDB (an unstructured database) and MySQL (a relational database), and each subsystem is a distributed cluster system. Then, to perform a complete and correct backup of AnyShare, it is necessary to back up the data in the three subsystems separately and ensure the data consistency among the three subsystems. Otherwise, due to the inconsistency of the restored data among the three subsystems, some data will be unavailable or lost.
  • OSS Object Storage Service
  • MongoDB an unstructured database
  • MySQL a relational database
  • the first type is to back up the subsystems under each application system separately.
  • each backup task is independent, data consistency cannot be guaranteed.
  • the second type is to adapt each application system separately based on a deep understanding of the implementation principles of each application system, that is, to develop backup logic suitable for each different application system.
  • the backup method may be: start the OSS gateway backup first, but proceed to the next step without waiting for the OSS gateway backup to complete; back up MongoDB, and proceed to the next step after the backup is complete; back up MySQL, and proceed to the next step after the backup is complete; notify the OSS gateway; wait for the OSS gateway backup to complete, and when the OSS gateway backup is completed, the AnyShare application system backup is completed.
  • the third category requires that the application system be deployed on a virtualization platform.
  • the backup method is You can also: add the virtual machines where the three AnyShare subsystems are located to a group; execute the script, suspend the input and output (IO) operations of the virtual machines, and create snapshots for each virtual machine; read data from the snapshots for backup. Therefore, this method requires that the application system must be deployed on the virtual machine, and the backup is the entire virtual host, not just the application system.
  • the embodiment of the present application provides an information processing method, which can meet the processing of different types and complex application systems and has strong versatility.
  • the method includes:
  • the workflow management module After receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, where the target workflow is the workflow corresponding to the processing request.
  • the workflow management module can be considered as a module that manages workflows in the information processing system, such as being responsible for managing sent workflows, verifying the legitimacy of workflows, and processing workflow commands from clients.
  • Workflow commands may include: create, edit, delete, query, start, stop, etc.
  • a processing request may refer to a request by a client to process a workflow, and the processing of a workflow may include, for example, backup, recovery and/or data consistency management, etc.
  • the target workflow may be understood as the workflow in the workflow management module corresponding to the processing request.
  • the protocol management unit in the workflow management module can receive the workflow, perform a validity check on the workflow, and store the workflow that has successfully been verified.
  • the verification method may include, for example, determining whether the workflow satisfies preset verification conditions.
  • the verification conditions may include a workflow definition model and/or processing and recovery business rule constraints, etc.
  • the workflow definition model and processing and recovery business rule constraints may be considered as rules pre-set by relevant personnel for verifying the legitimacy of the workflow.
  • FIG. 2 is a structural diagram of a workflow definition model provided according to Example 1 of the present application.
  • the workflow definition model can be considered as a directed acyclic graph from the entrance (i.e., the start node) to the exit (i.e., the end node).
  • Each node is regarded as a node in the graph.
  • Each node is connected by a connecting line.
  • the node pointed by the arrow is called the downstream node, and the node connected by the tail of the arrow is called the upstream node.
  • the data of the upstream node is allowed to be passed to the downstream node, for example, the data of resource node 2 is passed to node 3.
  • the workflow management module further includes a workflow management unit, and after receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, including:
  • the workflow management unit After receiving the processing request from the client, the workflow management unit obtains the stored target workflow
  • the processing request and the target workflow are sent to the orchestration engine module through the workflow management unit.
  • the node to be processed may refer to a node to be processed in the target workflow.
  • the number of nodes to be processed may be one or more, and the number of nodes to be processed may be determined based on the actual content of the workflow.
  • the specified type may be a pre-specified type. In this embodiment, the specified type may be a user-defined node, such as a service node.
  • the first information to be processed may be the information to be processed corresponding to the node to be processed of the specified type, and the information to be processed may be understood as information corresponding to the node, such as a node identifier.
  • the process of determining the to-be-processed nodes and sending the first to-be-processed information to the processing module includes, for example, that the orchestration engine module can judge all nodes contained in the target workflow after receiving the processing request and the target workflow to obtain all the to-be-processed nodes, and send the first to-be-processed information corresponding to the to-be-processed nodes of the specified type in the to-be-processed nodes to the processing module; the orchestration engine module can also judge the nodes in the target workflow from left to right in sequence, and after the current node is determined to be the to-be-processed node, judge the type of the to-be-processed node again, and after the judgment, send the first to-be-processed information corresponding to the to-be-processed node of the specified type to the processing module, and then continue to make similar judgments on the next node until all nodes in the target workflow are judged.
  • the means for determining the node to be processed in the target workflow may be determined according to the type of the node, or may be determined according to the node identifier of the node, etc.
  • the timing of sending the first information to be processed to the processing module may be, for example, directly sending the first information to be processed to the processing module after determining the node to be processed, or when the orchestration engine module receives the task request of the processing module, sending the first information to be processed corresponding to the task request to the processing module, and the task request may be used for the processing module to request whether there is an executable processing task.
  • the number of first information to be processed sent to the processing module may be, for example, one or more first information to be processed may be sent to the processing module at the same time.
  • the business worker when a business worker is started, the business worker connects to the orchestration engine module to query whether there are executable tasks. If there are, the orchestration engine module returns the details of the task to be executed (i.e., the first pending information) to the business worker. After receiving the task details, the business worker starts to execute the task. After the task is executed, the task execution result (i.e., the first processing result) can be reported to the orchestration engine module, and the next executable task can be requested at the same time.
  • the orchestration engine module returns the details of the task to be executed (i.e., the first pending information) to the business worker. After receiving the task details, the business worker starts to execute the task. After the task is executed, the task execution result (i.e., the first processing result) can be reported to the orchestration engine module, and the next executable task can be requested at the same time.
  • Table 2 shows the gateway type nodes, which may include parallel gateway nodes, parallel gateway synchronization nodes, conditional gateway nodes, multi-way selection gateway nodes, multi-way merging gateway nodes, and event gateway nodes.
  • Table 3 shows the business nodes, which may include script nodes, Service nodes, and various application nodes.
  • the Service node may be a custom node, that is, any node defined by the user.
  • Table 4 shows notification nodes, which may include email notification nodes, SMS notification nodes, and instant message notification nodes.
  • Figure 3 is a structural diagram of a node model provided according to Example 1 of the present application. As shown in Figure 3, for a node in a workflow, its node information may include name, type, provider, upstream and downstream nodes, input/output parameters, etc., and the node information is used for subsequent processing.
  • the modeler may be responsible for providing users with workflow building services, based on which users may build workflows for different processes according to different business scenarios.
  • the process of building a workflow may be, for example, building a basic workflow consisting of multiple nodes by dragging and connecting in a modeler, and configuring the nodes in the basic workflow to form a usable automated workflow.
  • the process of building a workflow can also be, for example, as follows: the user manually writes a JSON/YAMl file defining the workflow according to the workflow definition model; after writing, put it in the directory specified by the modeler, and you can see the defined workflow in the modeler. You can also manually write a file as a custom node and put it into the modeler for calling and use.
  • the candidate node includes a resource node
  • the information processing system further includes a resource management service module
  • the method further includes:
  • the resource information of the resource node is stored through the resource management service module.
  • the resource management service module can be considered as a module for managing resources.
  • resource information can be stored in the resource management service module.
  • the resource information includes information of each resource platform.
  • the resource platforms include virtualization platforms, cloud platforms, object storage, cloud native platforms and/or physical hosts, etc.
  • the resource information may include: resource type, resource access address, resource authorized users and/or resource type, etc.
  • the candidate nodes may include resource nodes.
  • the resource nodes may be selected for creation, and then the resource information bound to the resource nodes may be used to process the nodes.
  • the orchestration engine module of the second embodiment also includes an execution unit, and the method also includes: determining, by the scheduling unit, a target execution unit corresponding to the current node to be processed when it is determined that the current node to be processed is not a node to be processed of a specified type; and the scheduling unit sends a processing instruction containing the second information to be processed to the target execution unit.
  • S220 Determine, by the scheduling unit, a current node to be processed in the target workflow according to the node information of the target workflow.
  • the current node to be processed is the node to be processed in the current processing cycle. It can be considered that in this embodiment, the nodes to be processed can be processed in sequence, that is, after the current processing node is processed, the next node to be processed is processed.
  • the current processing cycle is the current processing cycle, and the current node to be processed is the node that needs to be processed in the current processing cycle.
  • the scheduling unit can be considered as a unit that performs scheduling in the orchestration engine module, which is used to determine the current node to be processed and send the first processing information to the processing module.
  • the node information can be understood as the information of the node in the target workflow, such as the upstream and downstream relationship of the node, the workflow execution progress, etc.
  • the current node is a node to be processed, it means that the current node needs to be processed, then the current node can be determined as the current node to be processed in the target workflow to execute subsequent steps; if the current node is not a node to be processed, it means that the current node does not need to be processed, then the next node can be continued to be judged, and the operation of determining the current node in the target workflow according to the node information of the target workflow is returned, that is, the next node is taken as the current node, and it is continued to be determined whether the current node is a node to be processed, and the above operation is repeated.
  • the type of the current node to be processed may be judged again, for example, whether the current node to be processed is a node to be processed of a specified type, and different steps may be executed according to the result of the judgment.
  • steps S240 - S250 may be executed, and the process returns to execute step S220 until the end condition is met.
  • steps S260-S270 can be executed, and the process returns to step S220 until the end condition is met.
  • S250 Process the first information to be processed by the processing module to obtain a first processing result.
  • the method further includes:
  • the scheduling unit receives the first processing result returned by the processing module, and returns to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit until an end condition is met.
  • the processing module can return the first processing result to the scheduling unit; then, after the scheduling unit receives the first processing result returned by the processing module, it can continue to return to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow through the scheduling unit until the end condition is met.
  • the end condition can be, for example, that the current node is the end node.
  • S260 Determine, by the scheduling unit, a target execution unit corresponding to the current node to be processed.
  • the current node to be backed up is a node of a type other than the specified type
  • the second information to be processed can be considered as the information to be processed corresponding to the current node to be backed up.
  • the target execution unit can be considered as the execution unit corresponding to the current node to be processed in the orchestration engine module, which is used to process the second information to be processed.
  • the target execution unit can be determined, and then a processing instruction containing the second information to be processed is sent to the target execution unit to instruct the target execution unit to perform processing.
  • the scheduling unit sends a processing instruction including the second information to be processed to the target execution unit.
  • the method further comprises:
  • the second processing result returned by the target execution unit is received by the scheduling unit, and the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit is returned to be executed until an end condition is met.
  • the target execution unit after receiving the second information to be processed, can process it to obtain a second processing result, and then return the second processing result to the scheduling unit; after receiving the second processing result returned by the target execution unit, the scheduling unit can return to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow until the end condition is met.
  • a processing method provided in the second embodiment of the present application can provide different processing logics for different nodes to be processed by determining the current node to be processed according to the node information of the target workflow and judging whether the current node to be processed is a node to be processed of a specified type, thereby ensuring data consistency while ensuring data consistency.
  • processing improve the efficiency of processing.
  • FIG. 5 is a structural diagram of a processing method provided according to Embodiment 2 of the present application. As shown in FIG. 5 , the processing method may include the following steps:
  • Resource registration First, various resources can be registered in the resource management service module. After successful registration, the resource management service module will monitor the successfully registered resources to update the resource status in real time (i.e., the resource information of the resource node is stored by the resource management service module).
  • the workflow creation module can use a modeler, drag and drop with a mouse, or create a workflow definition JSON/YAML file to create a workflow (ie, create a workflow based on candidate nodes through the workflow creation module).
  • Execute the Service node task When the business worker is ready to execute the task, it connects to the orchestration engine module to inquire whether there is a task that can be executed; the orchestration engine module will take out the matching node task (i.e., the first information to be processed) from the queue to be executed according to the requested task type in the business worker, and send it to the business worker (i.e., the processing module); after the business worker completes the task, it will report to the orchestration engine module that the task has been completed and the execution result (i.e., the first processing result is returned to the scheduling unit through the processing module).
  • the matching node task i.e., the first information to be processed
  • OSS gateway is a user-specific application system component, it is necessary to develop OSS processing recovery business nodes (i.e. candidate nodes) and business workers (i.e. processing modules).
  • OSS processing recovery business nodes i.e. candidate nodes
  • business workers i.e. processing modules
  • a general Service node can be used in the modeler and renamed as an OSS node to represent the OSS gateway application. Therefore, there is no need to develop additional specific processing recovery business nodes.
  • the source of data is defined as the source data end and the destination of data is defined as the destination end.
  • the source data end of the recovery is the processed data generated by various application systems through processing technology; the destination end is the destination of the processed data of the application system.
  • the registration information includes: resource type, resource access address, resource authorized user, resource type, etc.
  • the resource management module monitors that a resource has been successfully registered, it can extract the resource access address and resource authorized user from the resource registration information, and collect resource information on the corresponding resource platform based on this information.
  • the change information of the resource platform is monitored.
  • new resource information is collected in real time into the information processing system (ie, the resource information of the resource node is stored through the resource management service module).
  • the collected resources are tagged with application system labels to indicate that the resources can provide a running environment for those application systems.
  • a single disaster recovery system cannot cover the processing and recovery of all application systems, but it can usually cover the processing and recovery of the components that make up these application systems.
  • the components of an application system database, middleware, configuration files, etc., can be processed and recovered for application systems that are not supported by such disaster recovery systems through combined orchestration.
  • the modeler will expand an editable canvas; drag candidate nodes from the node list on the right to the canvas; connect multiple nodes with connecting lines; except for the start node and the end node, you can set the parameters of each node.
  • Figure 6 is a structural diagram of a workflow provided according to Example 2 of the present application.
  • the workflow may include a start node, a policy node, an OSS gateway node, a MongoDB node, a MySQL node, a message node and an end node.
  • the system When setting the parameters of the OSS node, the MongoDB node and the MySQL node, the system will automatically adapt according to the application type and resource tag to select the processing destination.
  • the scheduler in the orchestration engine module After receiving the workflow command, the scheduler in the orchestration engine module will generate a workflow instance according to the definition of the workflow; the scheduler finds the start node of the workflow, and searches forward along the connection line from the start node to find the first node to be executed (that is, the scheduling unit determines the current node to be processed in the target workflow according to the node information of the target workflow, and the current node to be processed is the node to be processed in the current processing cycle; the scheduling unit determines whether the current node to be processed is a node to be processed of a specified type; the scheduling unit sends the first information to be processed to the processing module when the current node to be processed is a node to be processed of a specified type).
  • the policy node it is the policy node.
  • the orchestration engine will return the OSS task to the OSS Worker (that is, the scheduling unit determines the current node to be processed in the target workflow according to the node information of the target workflow, and the current node to be processed is the node to be processed in the current processing cycle; the scheduling unit determines whether the current node to be processed is a node to be processed of a specified type; when the scheduling unit determines that the current node to be processed is a node to be processed of a specified type, the first to-be-processed information is sent to the processing module).
  • the OSS Worker After receiving the specified message, the OSS Worker will continue to execute the remaining unexecuted business logic until completion, and report the execution result to the orchestration engine (i.e., process the second to-be-processed information through the target execution unit, obtain the second processing result, and return the second processing result to the scheduling unit).
  • the orchestration engine i.e., process the second to-be-processed information through the target execution unit, obtain the second processing result, and return the second processing result to the scheduling unit.
  • the processing method provided in this application has the ability to recover from workflow interruptions.
  • the orchestration engine will persist the process data generated by the workflow during execution. Once any exception occurs and the workflow process is interrupted, when the exception is repaired, the orchestration engine will load the process data of the workflow execution from the persistent storage and recreate the memory mapping of the moment when the workflow execution process was interrupted in the memory, so as to continue to advance the execution of the workflow.
  • This application abstracts the common parts of different application system processing flows to form independent services for use by different businesses, so that application system processing developers only need to focus on the development of the application system's own processing and recovery capabilities, reducing development difficulty and improving processing stability.
  • the processor 710, the memory 720, the input device 730 and the output device 740 in the electronic device may
  • the connection is via a bus or other methods.
  • FIG. 7 takes the connection via a bus as an example.
  • the input device 730 can receive input digital or character information
  • the output device 740 can include a display device such as a display screen.
  • This embodiment also provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the above method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Stored Programmes (AREA)

Abstract

Disclosed in the present application is an information processing method, which is applied to an information processing system. The information processing system comprises a workflow management module, an orchestration engine module and a processing module. The method comprises: after receiving a processing request from a client, by means of a workflow management module, sending the processing request and a target workflow to an orchestration engine module; by means of the orchestration engine module, determining a node to be processed in the target workflow, and sending to a processing module first information to be processed, wherein said first information is information to be processed that corresponds to a node to be processed of a specified type; and processing said first information by means of the processing module, so as to obtain a first processing result. By means of the mutual cooperation among a workflow management module, an orchestration engine module and a processing module in a processing system, data consistency can be ensured and the universality of a processing method can be improved.

Description

信息处理方法Information processing methods
本公开要求在2022年12月28日提交中国专利局、申请号为202211699196.9的中国专利申请的优先权,以上申请的全部内容通过引用结合在本申请中。This disclosure claims the priority of the Chinese patent application filed with the China Patent Office on December 28, 2022, with application number 202211699196.9. The entire contents of the above application are incorporated by reference into this application.
技术领域Technical Field
本申请涉及数据安全技术领域,例如涉及一种信息处理方法。The present application relates to the field of data security technology, for example, to an information processing method.
背景技术Background technique
随着信息化的不断发展,各行各业均在进行数字化转型。无论是上层的应用系统还是支撑应用系统的底层基础设施、操作系统、中间件等在各行各业用户中种类较为繁多。故如何处理(如备份)越来越复杂且种类繁多的应用系统为当前亟需解决的问题。With the continuous development of informatization, all walks of life are undergoing digital transformation. Whether it is the upper-level application system or the underlying infrastructure, operating system, middleware, etc. that support the application system, there are many types of users in all walks of life. Therefore, how to handle (such as backup) the increasingly complex and diverse application systems is an urgent problem that needs to be solved.
相关技术的一些技术方案对应用系统中的子系统分别进行处理,并不能保证数据的一致性;一些技术方案对不同的应用系统分别开发对应的处理逻辑,处理成本较高,且通用性较差。Some technical solutions of the related art process the subsystems in the application system separately and cannot guarantee the consistency of data; some technical solutions develop corresponding processing logic for different application systems, which has high processing costs and poor versatility.
发明内容Summary of the invention
本申请提供了一种信息处理方法,以在保证数据一致性的同时,能够提高信息处理方法的通用性。The present application provides an information processing method, which can improve the versatility of the information processing method while ensuring data consistency.
第一方面,本申请实施例提供了一种信息处理方法,应用于信息处理系统,所述信息处理系统包括工作流管理模块、编排引擎模块和处理模块,所述方法包括:In a first aspect, an embodiment of the present application provides an information processing method, which is applied to an information processing system, wherein the information processing system includes a workflow management module, an orchestration engine module, and a processing module, and the method includes:
通过所述工作流管理模块在接收到客户端的处理请求后,将所述处理请求和目标工作流发送至所述编排引擎模块,所述目标工作流为与所述处理请求对应的工作流;After receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, where the target workflow is the workflow corresponding to the processing request;
通过所述编排引擎模块确定所述目标工作流中的待处理节点,并将第一待处理信息发送至所述处理模块,所述第一待处理信息为指定类型的待处理节点所对应的待处理信息;Determine the to-be-processed node in the target workflow by the orchestration engine module, and send the first to-be-processed information to the processing module, where the first to-be-processed information is the to-be-processed information corresponding to the to-be-processed node of a specified type;
通过所述处理模块对所述第一待处理信息进行处理,得到第一处理结果。The first information to be processed is processed by the processing module to obtain a first processing result.
第二方面,本申请实施例提供一种电子设备,该电子设备包括一个或多个 处理器、存储器以及一个或多个程序,所述一个或多个程序存储在所述存储器中,当被所述一个或多个处理器执行时,执行上述方法。In a second aspect, an embodiment of the present application provides an electronic device, the electronic device comprising one or more A processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and when executed by the one or more processors, the above method is performed.
第三方面,本申请实施例提供一种计算机可读存储介质,所述计算机可读存储介质存储有计算机可执行指令,所述计算机可执行指令用于执行上述方法。In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to execute the above method.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
为了说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作介绍,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be introduced below. The drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
图1是根据本申请实施例一提供的一种信息处理方法的流程图;FIG1 is a flow chart of an information processing method provided according to Embodiment 1 of the present application;
图2是根据本申请实施例一提供的一种工作流定义模型的结构图;FIG2 is a structural diagram of a workflow definition model provided according to Embodiment 1 of the present application;
图3是根据本申请实施例一提供的一种节点模型的结构图;FIG3 is a structural diagram of a node model provided according to Embodiment 1 of the present application;
图4是根据本申请实施例二提供的一种信息处理方法的流程图;FIG4 is a flow chart of an information processing method provided according to Embodiment 2 of the present application;
图5是根据本申请实施例二提供的一种信息处理方法的结构图;FIG5 is a structural diagram of an information processing method provided according to Embodiment 2 of the present application;
图6是根据本申请实施例二提供的一种工作流的结构图;FIG6 is a structural diagram of a workflow provided according to Embodiment 2 of the present application;
图7是根据本申请实施例三提供的一种电子设备的硬件结构示意图。FIG. 7 is a schematic diagram of the hardware structure of an electronic device provided according to Embodiment 3 of the present application.
具体实施方式Detailed ways
为了使本技术领域的人员更好地理解本申请方案,下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行描述,所描述的实施例仅仅是本申请一部分的实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都应当属于本申请保护的范围。In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. The described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.
需要说明的是,本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换,以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外,术语“包括”和“具有”以及他们的任何变形,意图在于覆盖不排他的包含,例如,包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于列出的那些步骤或单元,而是可包括没有列出的或对于这些过程、方法、产品或设备固有的其它步骤或 单元。It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those listed steps or units, but may include other steps or units that are not listed or inherent to these processes, methods, products or apparatuses. unit.
实施例一Embodiment 1
图1是根据本申请实施例一提供的一种信息处理方法的流程图,本实施例可适用于对系统进行处理的情况,该方法可以由信息处理装置来执行,该信息处理装置可以采用硬件和/或软件的形式实现,该信息处理装置可配置于处理系统中。Figure 1 is a flowchart of an information processing method provided according to Example 1 of the present application. This embodiment can be applicable to the situation of processing the system. The method can be executed by an information processing device. The information processing device can be implemented in the form of hardware and/or software. The information processing device can be configured in a processing system.
可以认为的是,在计算机应用的早期,应用系统通常是单机系统,且数据保存在单机的数据库系统中。在这种情形下,只需要对该单机数据库进行处理,如备份,即可完成对此应用系统的备份。It can be considered that in the early days of computer applications, application systems were usually stand-alone systems, and data was stored in a stand-alone database system. In this case, only the stand-alone database needs to be processed, such as backed up, to complete the backup of the application system.
随着信息化的不断发展,各行各业都在进行数字化转型。稳态IT+敏态IT建设趋势越发明显,无论是上层的应用系统还是支撑应用系统的底层基础设施、操作系统、中间件等在各行各业用户中是种类繁多的。故如何处理越来越复杂且种类繁多的应用系统,是当前处理技术亟需解决的问题。With the continuous development of informatization, all walks of life are undergoing digital transformation. The trend of steady-state IT + agile IT construction is becoming more and more obvious. Whether it is the upper-level application system or the underlying infrastructure, operating system, middleware, etc. that support the application system, there are many types of users in all walks of life. Therefore, how to deal with increasingly complex and diverse application systems is a problem that current processing technology urgently needs to solve.
下述以备份为例进行描述:假设一产品AnyShare包括三个子系统:对象存储服务(Object Storage Service,OSS)网关、MongoDB(一种无结构数据库)和MySQL(一种关系型数据库),而每个子系统又是一个分布式集群系统。那么对AnyShare进行一次完整、正确的备份,即要分别备份三个子系统中的数据,又要保证三个子系统间的数据一致性,缺一不可;否则由于恢复后的数据在三个子系统间不一致,会出现部分数据不可用、丢失等问题。The following description uses backup as an example: Assume that a product AnyShare includes three subsystems: Object Storage Service (OSS) gateway, MongoDB (an unstructured database) and MySQL (a relational database), and each subsystem is a distributed cluster system. Then, to perform a complete and correct backup of AnyShare, it is necessary to back up the data in the three subsystems separately and ensure the data consistency among the three subsystems. Otherwise, due to the inconsistency of the restored data among the three subsystems, some data will be unavailable or lost.
相关技术中存在几类备份方法:第一类是对每个应用系统下面的子系统,分别进行备份,然而由于每个备份任务是独立的,故数据一致性得不到任何保障。There are several types of backup methods in the related art: the first type is to back up the subsystems under each application system separately. However, since each backup task is independent, data consistency cannot be guaranteed.
第二类是在深入理解每个应用系统实现原理的基础上,针对每个应用系统分别进行适配,即对每个不同的应用系统,分别开发其适用于该应用系统的备份逻辑。The second type is to adapt each application system separately based on a deep understanding of the implementation principles of each application system, that is, to develop backup logic suitable for each different application system.
示例性的,备份方法例如可以为:先启动OSS网关备份,但不必等待OSS网关备份完成就可以转入下一步骤;备份MongoDB,备份完成后再进行下一步骤;备份MySQL,备份完成后再进行下一步骤;通知OSS网关;等待OSS网关备份完成,当OSS网关备份完成后,AnyShare应用系统备份完成。Exemplarily, the backup method may be: start the OSS gateway backup first, but proceed to the next step without waiting for the OSS gateway backup to complete; back up MongoDB, and proceed to the next step after the backup is complete; back up MySQL, and proceed to the next step after the backup is complete; notify the OSS gateway; wait for the OSS gateway backup to complete, and when the OSS gateway backup is completed, the AnyShare application system backup is completed.
可以看出,此备份方法仅特定于AnyShare这个产品,并不适用于其它应用系统的备份,因此,此种方法需要针对每个应用系统开发专有的备份逻辑,费时费力,通用性较差。It can be seen that this backup method is specific to the AnyShare product and is not applicable to the backup of other application systems. Therefore, this method requires the development of proprietary backup logic for each application system, which is time-consuming and labor-intensive and has poor versatility.
第三类是要求应用系统部署在虚拟化平台上面,示例性的,备份方法例如 还可以为:把AnyShare三个子系统所在虚拟机添加到一个组里;执行脚本,暂停虚拟机的输入输出(Input Output,IO)操作,并对每个虚拟机创建快照;从快照读取数据进行备份。故此方法要求应用系统必须部署在虚拟机上面,并且备份的是整个虚拟主机,并不是单纯的应用系统。The third category requires that the application system be deployed on a virtualization platform. For example, the backup method is You can also: add the virtual machines where the three AnyShare subsystems are located to a group; execute the script, suspend the input and output (IO) operations of the virtual machines, and create snapshots for each virtual machine; read data from the snapshots for backup. Therefore, this method requires that the application system must be deployed on the virtual machine, and the backup is the entire virtual host, not just the application system.
综上所述,相关技术的信息处理方法并不能很好的解决复杂应用系统的处理问题,通用性较差。In summary, the information processing methods of related technologies cannot solve the processing problems of complex application systems very well and have poor versatility.
基于此,本申请实施例提供了一种信息处理方法,能够满足不同种类、复杂应用系统的处理,具备较强的通用性。如图1所示,该方法包括:Based on this, the embodiment of the present application provides an information processing method, which can meet the processing of different types and complex application systems and has strong versatility. As shown in Figure 1, the method includes:
S110、通过所述工作流管理模块在接收到客户端的处理请求后,将所述处理请求和目标工作流发送至所述编排引擎模块,所述目标工作流为与所述处理请求对应的工作流。S110. After receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, where the target workflow is the workflow corresponding to the processing request.
工作流管理模块可以认为是信息处理系统中对工作流进行管理的模块,如负责管理发送的工作流、验证工作流的合法性以及处理客户端的工作流命令等,工作流命令可以包括:创建、编辑、删除、查询、启动、停止等。The workflow management module can be considered as a module that manages workflows in the information processing system, such as being responsible for managing sent workflows, verifying the legitimacy of workflows, and processing workflow commands from clients. Workflow commands may include: create, edit, delete, query, start, stop, etc.
处理请求可以是指客户端对工作流进行处理的请求,对工作流进行处理例如可以包括备份、恢复和/或数据一致性管理等;目标工作流则可以理解为工作流管理模块中与处理请求对应的工作流。A processing request may refer to a request by a client to process a workflow, and the processing of a workflow may include, for example, backup, recovery and/or data consistency management, etc. The target workflow may be understood as the workflow in the workflow management module corresponding to the processing request.
示例性的,工作流管理模块在接收到客户端的处理请求后,可以将处理请求和目标工作流发送至编排引擎模块,以进行后续步骤的处理。可以认为的是,工作流管理模块可以存储有一个或多个工作流,以等待客户端的处理请求,当接收到客户端的处理请求后,则可将处理请求以及对应的目标工作流发送至编排引擎模块,来进行后续的处理。其中,工作流的来源如可以由信息处理系统中其他模块创建并获取得到,也可以直接由其他系统输入得到。Exemplarily, after receiving the processing request from the client, the workflow management module can send the processing request and the target workflow to the orchestration engine module for subsequent processing. It can be considered that the workflow management module can store one or more workflows to wait for the processing request from the client. After receiving the processing request from the client, the processing request and the corresponding target workflow can be sent to the orchestration engine module for subsequent processing. Among them, the source of the workflow can be created and obtained by other modules in the information processing system, or it can be directly input from other systems.
在一个实施例中,所述工作流管理模块包括规约管理单元,在所述通过所述工作流管理模块将所述处理请求和目标工作流发送至所述编排引擎模块之前,该方法还包括:In one embodiment, the workflow management module includes a protocol management unit, and before sending the processing request and the target workflow to the orchestration engine module through the workflow management module, the method further includes:
通过所述规约管理单元接收工作流,对所述工作流进行合法性校验,并存储校验成功的工作流。The workflow is received through the protocol management unit, the workflow is verified for legality, and the workflow that succeeds in verification is stored.
规约管理单元可以认为是工作流管理模块中对工作流进行校验的单元,用于对工作流进行合法性校验。The protocol management unit can be considered as a unit in the workflow management module that verifies the workflow and is used to verify the validity of the workflow.
在一个实施方式中,在工作流管理模块将处理请求和目标工作流发送至编排引擎模块之前,工作流管理模块中的规约管理单元可以接收工作流,对工作流进行合法性校验,并将校验成功的工作流进行存储。对工作流进行合法性校 验的方式例如可以包括判断工作流是否满足预设的校验条件,如校验条件可以包括工作流定义模型和/或处理恢复业务规则约束等,工作流定义模型和处理恢复业务规则约束可以认为是相关人员预先设置的规则,用于对工作流进行合法性校验。In one embodiment, before the workflow management module sends the processing request and the target workflow to the orchestration engine module, the protocol management unit in the workflow management module can receive the workflow, perform a validity check on the workflow, and store the workflow that has successfully been verified. The verification method may include, for example, determining whether the workflow satisfies preset verification conditions. For example, the verification conditions may include a workflow definition model and/or processing and recovery business rule constraints, etc. The workflow definition model and processing and recovery business rule constraints may be considered as rules pre-set by relevant personnel for verifying the legitimacy of the workflow.
图2是根据本申请实施例一提供的一种工作流定义模型的结构图,如图2所示,工作流定义模型可以认为一个从入口(即开始节点)到出口(即结束节点)的有向无环图,每个节点都被视为图中的一个节点,每个节点之间通过连接线连接起来,箭头所指向的节点被称为下游节点,箭尾连接的节点被称为上游节点,上游节点的数据允许传递给下游节点,例如资源节点2的数据被传递给节点3。Figure 2 is a structural diagram of a workflow definition model provided according to Example 1 of the present application. As shown in Figure 2, the workflow definition model can be considered as a directed acyclic graph from the entrance (i.e., the start node) to the exit (i.e., the end node). Each node is regarded as a node in the graph. Each node is connected by a connecting line. The node pointed by the arrow is called the downstream node, and the node connected by the tail of the arrow is called the upstream node. The data of the upstream node is allowed to be passed to the downstream node, for example, the data of resource node 2 is passed to node 3.
在一个实施例中,所述工作流管理模块还包括工作流管理单元,所述通过所述工作流管理模块在接收到客户端的处理请求后,将所述处理请求和目标工作流发送至所述编排引擎模块,包括:In one embodiment, the workflow management module further includes a workflow management unit, and after receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, including:
通过所述工作流管理单元在接收到客户端的处理请求后,获取存储的所述目标工作流;After receiving the processing request from the client, the workflow management unit obtains the stored target workflow;
通过所述工作流管理单元将所述处理请求和所述目标工作流发送至所述编排引擎模块。The processing request and the target workflow are sent to the orchestration engine module through the workflow management unit.
工作流管理单元可以认为是工作流管理模块中对工作流进行管理的单元,如负责管理发送工作流以及处理客户端的处理请求等。The workflow management unit can be considered as a unit in the workflow management module that manages the workflow, such as being responsible for managing the sending of workflows and processing client processing requests.
在一个实施方式中,工作流管理单元在接收到客户端的处理请求后,首先可以获取存储的工作流中与处理请求对应的目标工作流,如根据处理请求中的标识信息对目标工作流进行确定并获取;然后在获取到目标工作流后,将处理请求和目标工作流发送至编排引擎模块,以进行后续的处理。In one embodiment, after receiving a processing request from a client, the workflow management unit may first obtain a target workflow corresponding to the processing request in the stored workflows, such as determining and obtaining the target workflow based on identification information in the processing request; and then after obtaining the target workflow, send the processing request and the target workflow to the orchestration engine module for subsequent processing.
S120、通过所述编排引擎模块确定所述目标工作流中的待处理节点,并将第一待处理信息发送至所述处理模块,所述第一待处理信息为指定类型的待处理节点所对应的待处理信息。S120: Determine a to-be-processed node in the target workflow through the orchestration engine module, and send first to-be-processed information to the processing module, where the first to-be-processed information is to-be-processed information corresponding to a to-be-processed node of a specified type.
编排引擎模块可以认为是信息处理系统中对工作流进行编排的模块。在本实施例中编排引擎模块可以负责执行目标工作流,例如可以为:接收工作流管理模块的启动、停止命令(即处理请求)以及工作流定义(即目标工作流);解析工作流数据,产生新的工作流实例;从工作流的开始节点起,根据节点间的上下游关系、节点流转规则、节点执行规则、工作流执行进度等信息,循环的将可执行的节点放到调度队列中,并等待执行单元来取。此外,工作流在执行过程中产生的全部过程数据,都会被编排引擎持久化存储,在遭遇异常情况 后,比如服务器断电等,也不会丢失数据。在该异常修复之后,编排引擎模块会从持久化存储中重新加载被中断的工作流,还原内存映射,继续执行工作流。The orchestration engine module can be considered as a module that orchestrates workflows in an information processing system. In this embodiment, the orchestration engine module can be responsible for executing the target workflow, for example, it can be: receiving the start and stop commands (i.e., processing requests) and workflow definitions (i.e., target workflows) from the workflow management module; parsing workflow data and generating new workflow instances; starting from the start node of the workflow, according to the upstream and downstream relationships between nodes, node transfer rules, node execution rules, workflow execution progress and other information, cyclically placing executable nodes in the scheduling queue and waiting for the execution unit to fetch them. In addition, all process data generated during the execution of the workflow will be persistently stored by the orchestration engine, and in the event of an abnormal situation. After that, even if the server loses power, data will not be lost. After the exception is fixed, the orchestration engine module will reload the interrupted workflow from the persistent storage, restore the memory mapping, and continue to execute the workflow.
待处理节点可以是指目标工作流中将要进行处理的节点,待处理节点的个数可以为一个或多个,待处理节点的个数可以根据工作流的实际内容进行确定;指定类型可以为预先指定的类型,在本实施例中,指定类型可以为用户定义的节点,如service节点;第一待处理信息可以为指定类型的待处理节点所对应的待处理信息,待处理信息可以理解为节点对应的信息,如节点标识等。The node to be processed may refer to a node to be processed in the target workflow. The number of nodes to be processed may be one or more, and the number of nodes to be processed may be determined based on the actual content of the workflow. The specified type may be a pre-specified type. In this embodiment, the specified type may be a user-defined node, such as a service node. The first information to be processed may be the information to be processed corresponding to the node to be processed of the specified type, and the information to be processed may be understood as information corresponding to the node, such as a node identifier.
在本实施例中,编排引擎模块在接收到处理请求和目标工作流后,可以对目标工作流中的待处理节点进行确定,然后将待处理节点中指定类型的待处理节点对应的第一待处理信息发送至处理模块,以进行后续节点的处理。其中,确定待处理节点以及发送第一待处理信息至处理模块的过程例如包括编排引擎模块可以在接收到处理请求和目标工作流后,对目标工作流中所包含的所有节点进行判断,来得到所有的待处理节点,并将待处理节点中指定类型的待处理节点应的第一待处理信息发送至处理模块;编排引擎模块还可以依次从左至右对目标工作流中的节点进行判断,在当前节点确定为待处理节点后,再次对待处理节点的类型进行判断,并在判断后将指定类型的待处理节点对应的第一待处理信息发送至处理模块,然后继续对下一节点进行类似的判断,直至将目标工作流中所有节点判断完为止。In this embodiment, after receiving the processing request and the target workflow, the orchestration engine module can determine the to-be-processed nodes in the target workflow, and then send the first to-be-processed information corresponding to the to-be-processed nodes of the specified type in the to-be-processed nodes to the processing module for subsequent node processing. Wherein, the process of determining the to-be-processed nodes and sending the first to-be-processed information to the processing module includes, for example, that the orchestration engine module can judge all nodes contained in the target workflow after receiving the processing request and the target workflow to obtain all the to-be-processed nodes, and send the first to-be-processed information corresponding to the to-be-processed nodes of the specified type in the to-be-processed nodes to the processing module; the orchestration engine module can also judge the nodes in the target workflow from left to right in sequence, and after the current node is determined to be the to-be-processed node, judge the type of the to-be-processed node again, and after the judgment, send the first to-be-processed information corresponding to the to-be-processed node of the specified type to the processing module, and then continue to make similar judgments on the next node until all nodes in the target workflow are judged.
在一个实施方式中,确定目标工作流中待处理节点的手段如可以根据节点的类型进行确定,也可以根据节点的节点标识进行确定等。In one embodiment, the means for determining the node to be processed in the target workflow may be determined according to the type of the node, or may be determined according to the node identifier of the node, etc.
在一个实施方式中,发送第一待处理信息至处理模块的时机如可以在确定待处理节点之后,直接将第一待处理信息发送至处理模块,也可以在编排引擎模块接收到处理模块的任务请求时,将与任务请求对应的第一待处理信息发送至处理模块,任务请求可以用于处理模块请求是否有可执行的处理任务。发送第一待处理信息至处理模块的个数如可以同时将一个或多个第一待处理信息发送至处理模块。In one embodiment, the timing of sending the first information to be processed to the processing module may be, for example, directly sending the first information to be processed to the processing module after determining the node to be processed, or when the orchestration engine module receives the task request of the processing module, sending the first information to be processed corresponding to the task request to the processing module, and the task request may be used for the processing module to request whether there is an executable processing task. The number of first information to be processed sent to the processing module may be, for example, one or more first information to be processed may be sent to the processing module at the same time.
S130、通过所述处理模块对所述第一待处理信息进行处理,得到第一处理结果。S130: Process the first information to be processed by the processing module to obtain a first processing result.
处理模块可以认为是对第一待处理信息进行处理的模块;第一处理结果即为对第一待处理信息进行处理的结果。The processing module can be considered as a module that processes the first information to be processed; the first processing result is the result of processing the first information to be processed.
示例性的,当处理模块接收到编排引擎模块发送的第一待处理信息后,可以对第一待处理信息进行处理来得到第一处理结果。在本实施例中,处理模块可以包括一个或多个业务Worker,每个业务Worker分别执行对应类型的处理任 务。在处理模块接收到编排引擎模块发送的第一待处理信息后,可以根据第一待处理信息中的标识信息来对业务Worker进行确定,然后交由确定的业务Worker对第一待处理信息进行处理,以此得到第一处理结果。Exemplarily, when the processing module receives the first information to be processed sent by the orchestration engine module, it can process the first information to be processed to obtain a first processing result. In this embodiment, the processing module may include one or more business workers, each of which performs a corresponding type of processing task. After the processing module receives the first information to be processed sent by the orchestration engine module, the service Worker can be determined according to the identification information in the first information to be processed, and then the first information to be processed is processed by the determined service Worker to obtain a first processing result.
示例性的,在一个实施方式中,在业务Worker启动的时候,业务Worker连接到编排引擎模块,来查询是否有可执行的任务,若有,则编排引擎模块向业务Worker返回待执行任务的详情(即第一待处理信息),业务Worker在接收到任务详情后,开始执行任务。在任务执行完成后,可以向编排引擎模块报告任务执行结果(即第一处理结果),同时请求下一个可执行的任务。Exemplarily, in one embodiment, when a business worker is started, the business worker connects to the orchestration engine module to query whether there are executable tasks. If there are, the orchestration engine module returns the details of the task to be executed (i.e., the first pending information) to the business worker. After receiving the task details, the business worker starts to execute the task. After the task is executed, the task execution result (i.e., the first processing result) can be reported to the orchestration engine module, and the next executable task can be requested at the same time.
在一个实施方式中,业务Worker可以根据业务Worker所在服务器的工作负载,灵活的向编排引擎模块请求不定数量的可执行任务。例如,当业务Worker所在服务器的工作负载较轻时,一次可以请求多个可执行任务,反之,则可以请求一个可执行任务。在此基础上,极大地提高工作流的执行效率。In one embodiment, the business worker can flexibly request an indefinite number of executable tasks from the orchestration engine module according to the workload of the server where the business worker is located. For example, when the workload of the server where the business worker is located is light, multiple executable tasks can be requested at a time, otherwise, one executable task can be requested. On this basis, the execution efficiency of the workflow is greatly improved.
在一个实施方式中,为了避免在没有可执行任务的情况下,业务Worker频繁的向编排引擎模块发起轮询,编排引擎模块在此种情况下可以不立即反馈业务Worker没有可执行的任务,而是维持此连接不断开,直到有可执行任务到来时,才向业务Worker返回可执行任务。In one implementation, in order to avoid the business worker frequently initiating polls to the orchestration engine module when there are no executable tasks, the orchestration engine module may not immediately feedback to the business worker that there are no executable tasks, but instead maintain the connection until an executable task arrives, and then return the executable task to the business worker.
本申请实施例一提供的一种信息处理方法,应用于信息处理系统,所述信息处理系统包括工作流管理模块、编排引擎模块和处理模块,所述方法包括:通过所述工作流管理模块在接收到客户端的处理请求后,将所述处理请求和目标工作流发送至所述编排引擎模块,所述目标工作流为与所述处理请求对应的工作流;通过所述编排引擎模块确定所述目标工作流中的待处理节点,并将第一待处理信息发送至所述处理模块,所述第一待处理信息为指定类型的待处理节点所对应的待处理信息;通过所述处理模块对第一待处理信息进行处理,得到第一处理结果。利用该方法,通过信息处理系统中工作流管理模块、编排引擎模块和处理模块之间的相互配合,能够在保证数据一致性的同时,提高信息处理方法的通用性。An information processing method provided in the first embodiment of the present application is applied to an information processing system, wherein the information processing system includes a workflow management module, an orchestration engine module and a processing module, wherein the method includes: after receiving a processing request from a client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, wherein the target workflow is the workflow corresponding to the processing request; the orchestration engine module determines the to-be-processed node in the target workflow, and sends the first to-be-processed information to the processing module, wherein the first to-be-processed information is the to-be-processed information corresponding to the to-be-processed node of a specified type; and the processing module processes the first to-be-processed information to obtain a first processing result. By using this method, through the mutual cooperation between the workflow management module, the orchestration engine module and the processing module in the information processing system, the versatility of the information processing method can be improved while ensuring data consistency.
在一个实施例中,所述信息处理系统还包括工作流创建模块,所述方法还包括:In one embodiment, the information processing system further includes a workflow creation module, and the method further includes:
通过所述工作流创建模块基于候选节点创建工作流,并将所述工作流发送至所述工作流管理模块。A workflow is created based on the candidate nodes by the workflow creation module, and the workflow is sent to the workflow management module.
工作流创建模块可以用于创建工作流;候选节点可以认为是供工作流创建模块创建工作流的节点。The workflow creation module can be used to create a workflow; the candidate node can be considered as a node for the workflow creation module to create a workflow.
在一个实施例中,所述候选节点包括预设节点和/或自定义节点。 In one embodiment, the candidate nodes include preset nodes and/or custom nodes.
预设节点可以是指预先配置的通用节点,为工作流的基础构成单元,预设节点可以包括不同类型的节点,如通知类节点或网关类节点等;自定义节点可以认为是用户自定义的节点。不同类型的节点可以实现不同的功能,通过不同类型节点的组合即可满足不同的业务需求。A preset node can refer to a pre-configured general node, which is the basic building block of a workflow. The preset node can include different types of nodes, such as notification nodes or gateway nodes, etc. A custom node can be considered a user-defined node. Different types of nodes can implement different functions, and different business needs can be met by combining different types of nodes.
在本步骤中工作流创建模块可以基于候选节点对工作流进行创建,然后将创建的工作流发送至工作流管理模块进行后续的合法性验证等,其中创建工作流的方法不限,只要能得到工作流即可。In this step, the workflow creation module can create a workflow based on the candidate nodes, and then send the created workflow to the workflow management module for subsequent legality verification, etc., wherein the method of creating the workflow is not limited, as long as the workflow can be obtained.
示例性的,表1-4为本实施例提供的几类候选节点,如表1为事件/消息类节点,可以包括开始节点、结束节点、消息节点和定时器节点等。
Illustratively, Tables 1-4 are several types of candidate nodes provided in this embodiment. For example, Table 1 is an event/message type node, which may include a start node, an end node, a message node, a timer node, and the like.
表2为网关类节点,可以包括并行网关节点、并行网关同步节点、条件网关节点、多路选择网关节点、多路合并网关节点和事件网关节点等。
Table 2 shows the gateway type nodes, which may include parallel gateway nodes, parallel gateway synchronization nodes, conditional gateway nodes, multi-way selection gateway nodes, multi-way merging gateway nodes, and event gateway nodes.
表3为业务类节点,可以包括脚本节点、Service节点和多种应用类节点等, 其中,Service节点可以为自定义节点,即用户自定义的任意节点。
Table 3 shows the business nodes, which may include script nodes, Service nodes, and various application nodes. The Service node may be a custom node, that is, any node defined by the user.
表4为通知类节点,可以包括邮件通知节点、短信通知节点和即时消息通知节点等。
Table 4 shows notification nodes, which may include email notification nodes, SMS notification nodes, and instant message notification nodes.
编排引擎模块除了内置了大量的通用节点,还提供了通用的Service任务节点,可用于适配任意的应用系统,其业务参数支持使用JSON(JavaScript Object Notation)或者YAML格式,从而能够兼容任意应用的业务参数。In addition to a large number of built-in general nodes, the orchestration engine module also provides a general Service task node that can be used to adapt to any application system. Its business parameters support the use of JSON (JavaScript Object Notation) or YAML format, making it compatible with the business parameters of any application.
图3是根据本申请实施例一提供的一种节点模型的结构图,如图3所示,对于工作流中的一节点而言,其节点信息可以包括名称、类型、提供者、上下游节点、输入/输出参数等,并依据这些节点信息用于后续的处理。Figure 3 is a structural diagram of a node model provided according to Example 1 of the present application. As shown in Figure 3, for a node in a workflow, its node information may include name, type, provider, upstream and downstream nodes, input/output parameters, etc., and the node information is used for subsequent processing.
在一个实施方式中,建模器可以用于负责向用户提供构建工作流,基于此用户可以根据不同的业务场景构建不同流程的工作流。In one embodiment, the modeler may be responsible for providing users with workflow building services, based on which users may build workflows for different processes according to different business scenarios.
构建工作流的过程例如可以为:在建模器中通过拖拽和连线方式构建由多个节点构成的基础工作流,并对该基础工作流中的节点进行配置,以形成可用的自动化工作流。The process of building a workflow may be, for example, building a basic workflow consisting of multiple nodes by dragging and connecting in a modeler, and configuring the nodes in the basic workflow to form a usable automated workflow.
构建工作流的过程例如还可以为:用户按照工作流的定义模型,手动编写工作流定义的JSON/YAMl文件;编写完成后,放到建模器指定的目录中,即可在建模器中看到已经定义好的工作流。还可以手动编写文件作为自定义节点,放入建模器中调用使用。The process of building a workflow can also be, for example, as follows: the user manually writes a JSON/YAMl file defining the workflow according to the workflow definition model; after writing, put it in the directory specified by the modeler, and you can see the defined workflow in the modeler. You can also manually write a file as a custom node and put it into the modeler for calling and use.
在一个实施例中,所述候选节点包括资源节点,所述信息处理系统还包括资源管理服务模块,所述方法还包括:In one embodiment, the candidate node includes a resource node, the information processing system further includes a resource management service module, and the method further includes:
通过所述资源管理服务模块存储所述资源节点的资源信息。 The resource information of the resource node is stored through the resource management service module.
资源管理服务模块可以认为是对资源进行管理的模块,例如资源管理服务模块中可以存储有资源信息,资源信息如包括每个资源平台的信息,资源平台包括如虚拟化平台、云平台、对象存储、云原生平台和/或物理主机等,资源信息可以包括:资源类型、资源访问地址、资源授权用户和/或资源类型等。The resource management service module can be considered as a module for managing resources. For example, resource information can be stored in the resource management service module. The resource information includes information of each resource platform. The resource platforms include virtualization platforms, cloud platforms, object storage, cloud native platforms and/or physical hosts, etc. The resource information may include: resource type, resource access address, resource authorized users and/or resource type, etc.
在一个实施方式中,候选节点中可以包括资源节点,在创建工作流时可以选取资源节点进行创建,之后即可使用资源节点绑定的资源信息进行该节点的处理。In one embodiment, the candidate nodes may include resource nodes. When creating a workflow, the resource nodes may be selected for creation, and then the resource information bound to the resource nodes may be used to process the nodes.
实施例二Embodiment 2
图4是根据本申请实施例二提供的一种处理方法的流程图,本实施例二在上述各实施例的基础上进行优化。在本实施例中,通过所述编排引擎模块确定所述目标工作流中的待处理节点,并将第一待处理信息发送至所述处理模块可以包括:通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点,所述当前待处理节点为在当前处理周期进行处理的待处理节点;通过所述调度单元判断所述当前待处理节点是否为指定类型的待处理节点;通过所述调度单元在判断所述当前待处理节点为指定类型的待处理节点时,将第一待处理信息发送至所述处理模块。Figure 4 is a flow chart of a processing method provided according to Embodiment 2 of the present application, and Embodiment 2 is optimized on the basis of the above embodiments. In this embodiment, determining the to-be-processed node in the target workflow by the orchestration engine module, and sending the first to-be-processed information to the processing module may include: determining the current to-be-processed node in the target workflow according to the node information of the target workflow by the scheduling unit, the current to-be-processed node being the to-be-processed node processed in the current processing cycle; judging by the scheduling unit whether the current to-be-processed node is a to-be-processed node of a specified type; and sending the first to-be-processed information to the processing module when the scheduling unit judges that the current to-be-processed node is a to-be-processed node of a specified type.
同时,本实施例二的所述编排引擎模块还包括执行单元,所述方法还包括:通过所述调度单元在判断所述当前待处理节点不为指定类型的待处理节点时,确定与当前待处理节点对应的目标执行单元;所述调度单元将包含第二待处理信息的处理指令发送至所述目标执行单元。At the same time, the orchestration engine module of the second embodiment also includes an execution unit, and the method also includes: determining, by the scheduling unit, a target execution unit corresponding to the current node to be processed when it is determined that the current node to be processed is not a node to be processed of a specified type; and the scheduling unit sends a processing instruction containing the second information to be processed to the target execution unit.
本实施例尚未详尽的内容请参考实施例一。For details not yet provided in this embodiment, please refer to Embodiment 1.
如图4所示,该方法包括:As shown in FIG4 , the method includes:
S210、通过所述工作流管理模块在接收到客户端的处理请求后,将所述处理请求和目标工作流发送至所述编排引擎模块。S210: After receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module.
S220、通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点。S220: Determine, by the scheduling unit, a current node to be processed in the target workflow according to the node information of the target workflow.
其中,所述当前待处理节点为在当前处理周期进行处理的待处理节点,可以认为的是,在本实施例中,可以依次对待处理节点进行处理,即处理完成当前处理节点后,再对下一个待处理节点进行处理,当前处理周期即为当前的处理周期,当前待处理节点则为在当前处理周期需要处理的节点。调度单元可以认为是编排引擎模块中进行调度的单元,用于确定当前待处理节点并将第一待处理信息发送至处理模块。节点信息可以理解为目标工作流中节点的信息,如节点的上下游关系、工作流执行进度等。 Among them, the current node to be processed is the node to be processed in the current processing cycle. It can be considered that in this embodiment, the nodes to be processed can be processed in sequence, that is, after the current processing node is processed, the next node to be processed is processed. The current processing cycle is the current processing cycle, and the current node to be processed is the node that needs to be processed in the current processing cycle. The scheduling unit can be considered as a unit that performs scheduling in the orchestration engine module, which is used to determine the current node to be processed and send the first processing information to the processing module. The node information can be understood as the information of the node in the target workflow, such as the upstream and downstream relationship of the node, the workflow execution progress, etc.
本步骤中调度单元可以根据目标工作流的节点信息来对目标工作流中的当前待处理节点进行确定,确定的过程可以根据节点的实际情况进行确定。In this step, the scheduling unit may determine the current to-be-processed node in the target workflow according to the node information of the target workflow, and the determination process may be determined according to the actual situation of the node.
在一个实施例中,所述通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点,包括:In one embodiment, determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit includes:
根据所述目标工作流的节点信息确定所述目标工作流中的当前节点;Determine the current node in the target workflow according to the node information of the target workflow;
判断所述当前节点是否为待处理节点,若是,则将所述当前节点确定为所述目标工作流中的当前待处理节点;若否,则返回执行所述根据所述目标工作流的节点信息确定所述目标工作流中的当前节点的操作。Determine whether the current node is a node to be processed. If so, determine the current node as the current node to be processed in the target workflow; if not, return to execute the operation of determining the current node in the target workflow according to the node information of the target workflow.
示例性的,在根据目标工作流的节点信息确定目标工作流中的当前待处理节点的过程中,可以先根据目标工作流的节点信息对目标工作流中的当前节点进行确定,然后判断当前节点是否为待处理节点,如果当前节点是待处理节点,说明当前节点需要处理,那么可以将当前节点确定为目标工作流中的当前待处理节点,以进行后续步骤的执行;如果当前节点不是待处理节点,说明当前节点不需要处理,则可以继续对下一个节点进行判断,返回执行根据目标工作流的节点信息确定目标工作流中的当前节点的操作,即将下一个节点作为当前节点,继续判断当前节点是否为待处理节点,并重复上述操作。Exemplarily, in the process of determining the current node to be processed in the target workflow according to the node information of the target workflow, the current node in the target workflow can be first determined according to the node information of the target workflow, and then it is determined whether the current node is a node to be processed. If the current node is a node to be processed, it means that the current node needs to be processed, then the current node can be determined as the current node to be processed in the target workflow to execute subsequent steps; if the current node is not a node to be processed, it means that the current node does not need to be processed, then the next node can be continued to be judged, and the operation of determining the current node in the target workflow according to the node information of the target workflow is returned, that is, the next node is taken as the current node, and it is continued to be determined whether the current node is a node to be processed, and the above operation is repeated.
S230、通过所述调度单元判断所述当前待处理节点是否为指定类型的待处理节点,若是,则执行S240;若否,则执行S250-S260。S230. Determine, by the scheduling unit, whether the current node to be processed is a node to be processed of a specified type. If so, execute S240; if not, execute S250-S260.
在确定了当前待处理节点后,可以再次对当前待处理节点的类型进行判断,例如判断当前待处理节点是否为指定类型的待处理节点,并根据判断的结果执行不同的步骤。After the current node to be processed is determined, the type of the current node to be processed may be judged again, for example, whether the current node to be processed is a node to be processed of a specified type, and different steps may be executed according to the result of the judgment.
示例性的,如果当前待处理节点为指定类型的待处理节点,说明当前待处理节点为指定的节点,则可以执行步骤S240-S250,并返回执行步骤S220,直到满足结束条件。Exemplarily, if the current node to be processed is a node to be processed of a specified type, indicating that the current node to be processed is a specified node, then steps S240 - S250 may be executed, and the process returns to execute step S220 until the end condition is met.
如果当前待处理节点不为指定类型的待处理节点,说明当前待处理节点为除指定类型之外的其他类型的节点,则可以执行步骤S260-S270,并返回执行步骤S220,直到满足结束条件。If the current node to be processed is not a node to be processed of the specified type, it means that the current node to be processed is a node of other types except the specified type, then steps S260-S270 can be executed, and the process returns to step S220 until the end condition is met.
S240、通过所述调度单元将第一待处理信息发送至所述处理模块。S240. Send the first information to be processed to the processing module through the scheduling unit.
S250、通过所述处理模块对第一待处理信息进行处理,得到第一处理结果。S250: Process the first information to be processed by the processing module to obtain a first processing result.
在一个实施例中,在所述通过所述处理模块对第一待处理信息进行处理,得到第一处理结果之后,还包括:In one embodiment, after the first information to be processed is processed by the processing module to obtain a first processing result, the method further includes:
通过所述处理模块将所述第一处理结果返回至所述调度单元; Returning the first processing result to the scheduling unit through the processing module;
通过所述调度单元接收所述处理模块返回的第一处理结果,并返回执行所述通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点的操作,直到满足结束条件。The scheduling unit receives the first processing result returned by the processing module, and returns to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit until an end condition is met.
可以认为的是,在处理模块对第一待处理信息进行处理,得到第一处理结果之后,处理模块可以将第一处理结果返回至调度单元;随后调度单元接收处理模块返回的第一处理结果后,可以继续返回执行通过调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点的操作,直到满足结束条件。结束条件例如可以是当前节点为结束节点等。It can be considered that after the processing module processes the first information to be processed and obtains the first processing result, the processing module can return the first processing result to the scheduling unit; then, after the scheduling unit receives the first processing result returned by the processing module, it can continue to return to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow through the scheduling unit until the end condition is met. The end condition can be, for example, that the current node is the end node.
S260、通过所述调度单元确定与所述当前待处理节点对应的目标执行单元。S260. Determine, by the scheduling unit, a target execution unit corresponding to the current node to be processed.
在本实施例中,当前待备份节点为除指定类型之外的其他类型的节点,第二待处理信息则可以认为是当前待备份节点所对应的待处理信息。目标执行单元可以认为是编排引擎模块中与当前待处理节点对应的执行单元,用于对第二待处理信息进行处理。In this embodiment, the current node to be backed up is a node of a type other than the specified type, and the second information to be processed can be considered as the information to be processed corresponding to the current node to be backed up. The target execution unit can be considered as the execution unit corresponding to the current node to be processed in the orchestration engine module, which is used to process the second information to be processed.
本实施例中,在调度单元判断当前待处理节点不为指定类型的待处理节点后,可以对目标执行单元进行确定,随后将包含第二待处理信息的处理指令发送至目标执行单元,以指示目标执行单元进行处理。In this embodiment, after the scheduling unit determines that the current node to be processed is not a node to be processed of the specified type, the target execution unit can be determined, and then a processing instruction containing the second information to be processed is sent to the target execution unit to instruct the target execution unit to perform processing.
S270、所述调度单元将包含第二待处理信息的处理指令发送至所述目标执行单元。S270: The scheduling unit sends a processing instruction including the second information to be processed to the target execution unit.
在一个实施例中,所述方法还包括:In one embodiment, the method further comprises:
通过所述目标执行单元对所述第二待处理信息进行处理,得到第二处理结果,并将所述第二处理结果返回至所述调度单元;Processing the second information to be processed by the target execution unit to obtain a second processing result, and returning the second processing result to the scheduling unit;
通过所述调度单元接收所述目标执行单元返回的第二处理结果,并返回执行所述通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点的操作,直到满足结束条件。The second processing result returned by the target execution unit is received by the scheduling unit, and the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit is returned to be executed until an end condition is met.
第二处理结果即为目标执行单元对第二待处理信息进行处理的结果。The second processing result is the result of the target execution unit processing the second information to be processed.
在一个实施方式中,目标执行单元接收到第二待处理信息后,可以进行处理,得到第二处理结果,然后将第二处理结果返回至调度单元;调度单元在接收到目标执行单元返回的第二处理结果后,可以返回执行根据目标工作流的节点信息确定目标工作流中的当前待处理节点的操作,直到满足结束条件。In one embodiment, after receiving the second information to be processed, the target execution unit can process it to obtain a second processing result, and then return the second processing result to the scheduling unit; after receiving the second processing result returned by the target execution unit, the scheduling unit can return to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow until the end condition is met.
本申请实施例二提供的一种处理方法,通过根据目标工作流的节点信息确定当前待处理节点,并判断当前待处理节点是否为指定类型的待处理节点,能够为不同的待处理节点提供不同的处理逻辑,从而能够在保证数据一致性的同 时,提高处理的效率。A processing method provided in the second embodiment of the present application can provide different processing logics for different nodes to be processed by determining the current node to be processed according to the node information of the target workflow and judging whether the current node to be processed is a node to be processed of a specified type, thereby ensuring data consistency while ensuring data consistency. When processing, improve the efficiency of processing.
图5是根据本申请实施例二提供的一种处理方法的结构图,如图5所示,处理方法可以包括下述几个步骤:FIG. 5 is a structural diagram of a processing method provided according to Embodiment 2 of the present application. As shown in FIG. 5 , the processing method may include the following steps:
(1)资源注册。首先可以将各种各样的资源注册到资源管理服务模块中,注册成功之后,资源管理服务模块会对注册成功的资源进行监测,以实时的更新资源状态(即通过所述资源管理服务模块存储所述资源节点的资源信息)。(1) Resource registration. First, various resources can be registered in the resource management service module. After successful registration, the resource management service module will monitor the successfully registered resources to update the resource status in real time (i.e., the resource information of the resource node is stored by the resource management service module).
(2)创建工作流。工作流创建模块可以使用建模器,通过鼠标拖拽、或者创建工作流定义JSON/YAML文件来创建工作流(即通过所述工作流创建模块基于候选节点创建工作流)。(2) Creating a workflow. The workflow creation module can use a modeler, drag and drop with a mouse, or create a workflow definition JSON/YAML file to create a workflow (ie, create a workflow based on candidate nodes through the workflow creation module).
(3)发布工作流。当发布工作流时,工作流管理模块中的规约管理器(即规约管理单元)会对工作流的合法性进行校验,只有通过合法性校验的工作流,才会发布成功并持久化到存储中(即通过规约管理单元接收工作流,对工作流进行合法性校验,并存储校验成功的工作流);未通过合法性校验的工作流发布不成功,修改后可以继续发布。(3) Publishing workflows. When publishing a workflow, the protocol manager (i.e., protocol management unit) in the workflow management module will verify the validity of the workflow. Only workflows that pass the validity verification will be successfully published and persisted in the storage (i.e., the protocol management unit receives the workflow, verifies the validity of the workflow, and stores the successfully verified workflow). Workflows that fail to pass the validity verification will not be published successfully, but can be published again after modification.
(4)调度工作流。工作流发布成功之后,可以启动运行工作流。启动时,编排引擎模块中的解析器会从存储中加载工作流并进行解析,产生内存映射;调度器(即调度单元)会根据该内存映射产生工作流实例,根据节点的上下游关系、节点流转规则、节点执行规则、工作流执行进度等信息,选出待执行的节点(即根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点,所述当前待处理节点为在当前处理周期进行处理的待处理节点),并将待执行的节点加入到待执行队列中,然后针对不同待执行节点的类型执行对应的操作(即判断所述当前待处理节点是否为指定类型的待备份节点;通过所述调度单元在所述当前待处理节点为指定类型的待备份节点时,将第一待处理信息发送至所述处理模块);否则,产生对应节点的执行器(即执行单元),将节点交由执行器执行(即通过所述调度单元在所述当前待处理节点不为指定类型的待备份节点时,确定与当前待处理节点对应的目标执行单元;所述调度单元将包含第二待处理信息的处理指令发送至所述目标执行单元)。(4) Scheduling workflows. After the workflow is successfully published, you can start running the workflow. At startup, the parser in the orchestration engine module will load the workflow from the storage and parse it to generate a memory map; the scheduler (i.e., the scheduling unit) will generate a workflow instance based on the memory map, and select the nodes to be executed based on the upstream and downstream relationships of the nodes, node flow rules, node execution rules, workflow execution progress and other information (i.e., determine the current node to be processed in the target workflow based on the node information of the target workflow, and the current node to be processed is the node to be processed in the current processing cycle), and add the nodes to be executed to the queue to be executed, and then for different classes of nodes to be executed, The type performs the corresponding operation (i.e., determines whether the current node to be processed is a node to be backed up of the specified type; when the current node to be processed is a node to be backed up of the specified type, the first information to be processed is sent to the processing module through the scheduling unit); otherwise, generates an executor (i.e., an execution unit) of the corresponding node, and hands the node to the executor for execution (i.e., when the current node to be processed is not a node to be backed up of the specified type, the scheduling unit determines the target execution unit corresponding to the current node to be processed; the scheduling unit sends the processing instruction containing the second information to be processed to the target execution unit).
(5)执行Service节点任务。当业务Worker准备好执行任务时,连接到编排引擎模块,询问是否有可以执行任务;编排引擎模块会根据业务Worker中的请求任务类型,从待执行的队列中取出匹配的节点任务(即第一待处理信息),发送给业务Worker(即处理模块);业务Worker在执行完成任务后,会向编排引擎模块报告任务已完成和执行结果(即通过所述处理模块将所述第一处理结果返回至所述调度单元)。 (5) Execute the Service node task. When the business worker is ready to execute the task, it connects to the orchestration engine module to inquire whether there is a task that can be executed; the orchestration engine module will take out the matching node task (i.e., the first information to be processed) from the queue to be executed according to the requested task type in the business worker, and send it to the business worker (i.e., the processing module); after the business worker completes the task, it will report to the orchestration engine module that the task has been completed and the execution result (i.e., the first processing result is returned to the scheduling unit through the processing module).
(6)进行下一轮调度。当编排引擎模块接收到业务Worker报告的任务已完成消息后,会根据当前节点任务的执行结果、上游节点、下游节点的执行状态,选择待执行的下游节点,进入新一轮的执行过程(即通过所述调度单元接收所述处理模块返回的第一处理结果,并返回执行所述通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点的操作,直到满足结束条件)。(6) Perform the next round of scheduling. When the orchestration engine module receives the task completion message reported by the business Worker, it selects the downstream node to be executed based on the execution result of the current node task, the execution status of the upstream node, and the downstream node, and enters a new round of execution process (i.e., the scheduling unit receives the first processing result returned by the processing module, and returns to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit, until the end condition is met).
下面对本申请实施例提供的处理方法进行示例性的描述,假设AnyShare应用系统由OSS网关、MongoDB和MySQL三个子系统构成。The following is an exemplary description of the processing method provided in the embodiment of the present application, assuming that the AnyShare application system is composed of three subsystems: OSS gateway, MongoDB and MySQL.
1.开发特定应用的处理恢复节点。1. Develop application-specific processing recovery nodes.
由于OSS网关是用户特有的应用系统组件,故需要开发OSS处理恢复业务节点(即候选节点)和业务Worker(即处理模块)。在本实施例中,建模器中可以使用通用的Service节点,并重命名为OSS节点来代表OSS网关应用,因此,不需要再额外开发特定的处理恢复业务节点。Since the OSS gateway is a user-specific application system component, it is necessary to develop OSS processing recovery business nodes (i.e. candidate nodes) and business workers (i.e. processing modules). In this embodiment, a general Service node can be used in the modeler and renamed as an OSS node to represent the OSS gateway application. Therefore, there is no need to develop additional specific processing recovery business nodes.
开发OSS网关应用的处理恢复业务Worker只需要实现二个接口:循环的调用编排引擎(即编排引擎模块)的接口,请求当前可执行的OSS任务;当OSS处理或者恢复完成之后,向编排引擎报告已完成,并上报执行结果(即第一处理结果)。To develop the OSS gateway application's processing and recovery service Worker, you only need to implement two interfaces: cyclically call the interface of the orchestration engine (i.e., the orchestration engine module) to request the currently executable OSS tasks; when the OSS processing or recovery is completed, report the completion to the orchestration engine and report the execution result (i.e., the first processing result).
2.资源统一管理2. Unified resource management
通常在灾备业务领域,有定义数据的来源为源数据端和数据指向的目的地为目的端。恢复的源数据端为对各类应用系统通过处理技术所处理产生的处理数据;目的端为对应用系统的处理数据演练到的目的地。Usually in the field of disaster recovery, the source of data is defined as the source data end and the destination of data is defined as the destination end. The source data end of the recovery is the processed data generated by various application systems through processing technology; the destination end is the destination of the processed data of the application system.
不同类型的应用系统,进行演练所需的演练资源的形态各不相同,有物理客户端、虚拟化平台、云平台、对象存储等。因此需要满足应用系统兼容普适性要求,需要解决处理恢复资源的统一管理的问题。Different types of application systems require different forms of drill resources for drills, including physical clients, virtualization platforms, cloud platforms, object storage, etc. Therefore, it is necessary to meet the compatibility and universality requirements of application systems and solve the problem of unified management of recovery resources.
首先可以将虚拟化平台、云平台、对象存储、云原生平台、物理主机注册到资源管理模块,注册信息包括:资源类型、资源访问地址、资源授权用户、资源类型等。First, you can register the virtualization platform, cloud platform, object storage, cloud native platform, and physical host to the resource management module. The registration information includes: resource type, resource access address, resource authorized user, resource type, etc.
然后资源管理模块监控到有资源注册成功后,可以从资源注册信息中提取出资源访问地址、资源授权用户,根据这些信息到对应的资源平台上采集资源信息。Then, after the resource management module monitors that a resource has been successfully registered, it can extract the resource access address and resource authorized user from the resource registration information, and collect resource information on the corresponding resource platform based on this information.
继而在首次采集资源信息完成后,监听资源平台的变更信息,当资源平台有变更时,实时采集新的资源信息到信息处理系统中(即通过所述资源管理服务模块存储所述资源节点的资源信息)。 After the initial collection of resource information is completed, the change information of the resource platform is monitored. When the resource platform is changed, new resource information is collected in real time into the information processing system (ie, the resource information of the resource node is stored through the resource management service module).
最后给已经采集到资源打上应用系统标签,用来表示该资源可以为那些应用系统提供运行环境。Finally, the collected resources are tagged with application system labels to indicate that the resources can provide a running environment for those application systems.
3.工作流的构建3. Workflow Construction
各行各业的应用系统种类繁多,单一的灾备系统不可能涵盖所有应用系统的处理恢复,但是通常可以覆盖组成这些应用系统的组件的处理恢复,比如一应用系统的组件:数据库、中间件、配置文件等,可以对此类灾备系统不支持的应用系统通过组合编排的方式进行处理恢复。There are many types of application systems in various industries. A single disaster recovery system cannot cover the processing and recovery of all application systems, but it can usually cover the processing and recovery of the components that make up these application systems. For example, the components of an application system: database, middleware, configuration files, etc., can be processed and recovered for application systems that are not supported by such disaster recovery systems through combined orchestration.
首先,可以打开建模器,并点击“新建”,建模器将展开一个可编辑的画布;从右边的节点列表中依次拖到候选节点到画布中;多个节点之间使用连接线连接起来;除开始节点和结束节点之外,可以设置每个节点的参数。在全部完成之后,可以点击“发布”,建模器会将工作流的定义发送到工作流管理模块(即通过所述工作流创建模块基于候选节点创建工作流,并将所述工作流发送至所述工作流管理模块)。First, you can open the modeler and click "New", the modeler will expand an editable canvas; drag candidate nodes from the node list on the right to the canvas; connect multiple nodes with connecting lines; except for the start node and the end node, you can set the parameters of each node. After everything is completed, you can click "Publish", and the modeler will send the definition of the workflow to the workflow management module (that is, create a workflow based on the candidate nodes through the workflow creation module, and send the workflow to the workflow management module).
图6是根据本申请实施例二提供的一种工作流的结构图,如图6所示,工作流可以包括开始节点、策略节点、OSS网关节点、MongoDB节点、MySQL节点、消息节点和结束节点,在设置OSS节点、MongoDB节点和MySQL节点的参数时,系统会根据应用类型、资源标签来进行自动的适配,来选择处理的目的地。Figure 6 is a structural diagram of a workflow provided according to Example 2 of the present application. As shown in Figure 6, the workflow may include a start node, a policy node, an OSS gateway node, a MongoDB node, a MySQL node, a message node and an end node. When setting the parameters of the OSS node, the MongoDB node and the MySQL node, the system will automatically adapt according to the application type and resource tag to select the processing destination.
然后,工作流管理模块会对工作流进行合法性验证,验证通过后就会持久性(即通过所述规约管理单元接收工作流,对所述工作流进行合法性校验,并存储校验成功的工作流)。当工作流管理模块接收到“启动”命令之后,会从数据库中提取工作流定义,然后将该命令连同工作流定义,一并发送到编排引擎模块(即通过所述工作流管理模块在接收到客户端的处理请求后,将所述处理请求和目标工作流发送至所述编排引擎模块)。Then, the workflow management module will verify the legitimacy of the workflow, and it will be persistent after the verification (i.e., the workflow is received through the protocol management unit, the workflow is verified for legitimacy, and the workflow that succeeds the verification is stored). After the workflow management module receives the "start" command, it will extract the workflow definition from the database, and then send the command together with the workflow definition to the orchestration engine module (i.e., after receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module).
编排引擎模块中的调度器在接收到工作流命令之后,会根据工作流的定义产生工作流实例;调度器找到工作流的开始节点,并从开始节点,沿着连接线往前查找,找到第一个待执行的节点(即通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点,所述当前待处理节点为在当前处理周期进行处理的待处理节点;通过所述调度单元判断所述当前待处理节点是否为指定类型的待处理节点;通过所述调度单元在所述当前待处理节点为指定类型的待处理节点时,将第一待处理信息发送至所述处理模块),在这里,是策略节点。由于策略节点并没有实例的业务逻辑,调度器会越过策略节点,继续往前查找,找到待执行的节点,在这里,是OSS节点。调度器产生OSS节点实例,并加入到待执行队列中,等待OSS Worker来取。由于OSS节点任务未执行完成,调度器暂停对工作流的调度。在等待一段时间之后,OSS Worker 询问编排引擎是否有可执行的活动,此时,编排引擎会将OSS任务返回给OSS Worker(即通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点,所述当前待处理节点为在当前处理周期进行处理的待处理节点;通过所述调度单元判断所述当前待处理节点是否为指定类型的待处理节点;通过所述调度单元在判断所述当前待处理节点为指定类型的待处理节点时,将第一待处理信息发送至所述处理模块)。After receiving the workflow command, the scheduler in the orchestration engine module will generate a workflow instance according to the definition of the workflow; the scheduler finds the start node of the workflow, and searches forward along the connection line from the start node to find the first node to be executed (that is, the scheduling unit determines the current node to be processed in the target workflow according to the node information of the target workflow, and the current node to be processed is the node to be processed in the current processing cycle; the scheduling unit determines whether the current node to be processed is a node to be processed of a specified type; the scheduling unit sends the first information to be processed to the processing module when the current node to be processed is a node to be processed of a specified type). Here, it is the policy node. Since the policy node does not have the business logic of the instance, the scheduler will skip the policy node and continue to search forward to find the node to be executed, which is the OSS node here. The scheduler generates an OSS node instance and adds it to the queue to be executed, waiting for the OSS Worker to take it. Since the OSS node task has not been completed, the scheduler suspends the scheduling of the workflow. After waiting for a period of time, the OSS Worker Ask the orchestration engine whether there is an executable activity. At this time, the orchestration engine will return the OSS task to the OSS Worker (that is, the scheduling unit determines the current node to be processed in the target workflow according to the node information of the target workflow, and the current node to be processed is the node to be processed in the current processing cycle; the scheduling unit determines whether the current node to be processed is a node to be processed of a specified type; when the scheduling unit determines that the current node to be processed is a node to be processed of a specified type, the first to-be-processed information is sent to the processing module).
当OSS Worker执行完成任务之后,会向编排引擎报告任务已执行完成,并上报执行结果(即通过所述处理模块将所述第一处理结果返回至所述调度单元)。When the OSS Worker completes the task, it will report to the orchestration engine that the task has been completed and report the execution result (that is, the first processing result is returned to the scheduling unit through the processing module).
编排引擎调度器在接收到OSS Worker上报的结果之后,会更新OSS节点任务的执行状态为完成,并同步更新执行结果。若结果为成功,则调度器会继续去寻找OSS节点的下游节点(即通过所述调度单元接收所述处理模块返回的第一处理结果,并返回执行所述通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点的操作),即MongoDB和MySQL二个节点,因为这二个节点都是OSS节点的直接下游节点。调度器产生MongoDB和MySQL这二个节点的节点任务,并把这二个任务加入到待执行队列中。After receiving the results reported by the OSS Worker, the orchestration engine scheduler will update the execution status of the OSS node task to completion, and synchronously update the execution result. If the result is successful, the scheduler will continue to look for the downstream node of the OSS node (that is, receive the first processing result returned by the processing module through the scheduling unit, and return to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow through the scheduling unit), that is, the MongoDB and MySQL nodes, because these two nodes are the direct downstream nodes of the OSS node. The scheduler generates node tasks for the two nodes, MongoDB and MySQL, and adds these two tasks to the queue to be executed.
同样的,当MongoDB Worker和MySQL Worker来询问编排引擎是否有可执行的活动时,编排引擎会将这二个任务分别返回给MongoDB Worker和MySQL Worker。Similarly, when MongoDB Worker and MySQL Worker ask the orchestration engine whether there are any executable activities, the orchestration engine will return the two tasks to MongoDB Worker and MySQL Worker respectively.
类似的,当MongoDB Worker和MySQL Worker执行完成各自的任务之后,分别向编排引擎上报任务执行结果。Similarly, after MongoDB Worker and MySQL Worker complete their respective tasks, they report the task execution results to the orchestration engine respectively.
编排引擎调度器在接收到MongoDB Worker和MySQL Worker上报的结果之后,会更新MongoDB节点任务和MySQL节点任务的执行状态为完成,并同步更新执行结果。若结果为成功,则调度器会继续去寻找下游节点,即消息节点。由于消息节点的执行并不需要业务Worker的参与,因此,调度器产生消息节点任务后,直接交由编排引擎执行。在本实施例中,编排引擎会发送指定的消息给OSS Worker。After receiving the results reported by MongoDB Worker and MySQL Worker, the orchestration engine scheduler will update the execution status of the MongoDB node task and the MySQL node task to completion, and synchronously update the execution results. If the result is successful, the scheduler will continue to look for downstream nodes, namely message nodes. Since the execution of message nodes does not require the participation of business workers, after the scheduler generates message node tasks, they are directly handed over to the orchestration engine for execution. In this embodiment, the orchestration engine will send the specified message to the OSS Worker.
OSS Worker在接收到指定的消息后会继续执行剩余未执行完的业务逻辑直到完成,向编排引擎上报执行结果(即通过所述目标执行单元对所述第二待处理信息进行处理,得到第二处理结果,并将所述第二处理结果返回至所述调度单元)。After receiving the specified message, the OSS Worker will continue to execute the remaining unexecuted business logic until completion, and report the execution result to the orchestration engine (i.e., process the second to-be-processed information through the target execution unit, obtain the second processing result, and return the second processing result to the scheduling unit).
编排引擎调度器继续去寻找消息节点的下游节点,在这个实施例中,即为结束节点。此时,调度器结束该工作流的运行,以此完成整个工作流的处理(即通过所述调度单元接收所述目标执行单元返回的第二处理结果,并返回执行所 述通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点的操作,直到满足结束条件)。The orchestration engine scheduler continues to search for the downstream node of the message node, which is the end node in this embodiment. At this point, the scheduler ends the operation of the workflow, thereby completing the processing of the entire workflow (i.e., the scheduling unit receives the second processing result returned by the target execution unit and returns the execution result). The operation of the current node to be processed in the target workflow is determined by the scheduling unit according to the node information of the target workflow until an end condition is met).
可以看出,与相关技术相比,本申请实施例包括以下有益效果:It can be seen that compared with the related art, the embodiments of the present application include the following beneficial effects:
1.通过将各种各样的复杂应用系统进行分解,构造多个业务Worker节点,同时提供对这些业务Worker节点的编排能力,从而能够形成一种新的能力。利用这个新能力,来满足不同用户、不同应用系统、不同业务场景的处理恢复需求。1. By decomposing various complex application systems, constructing multiple business Worker nodes, and providing the ability to orchestrate these business Worker nodes, a new capability can be formed. This new capability can be used to meet the processing and recovery needs of different users, different application systems, and different business scenarios.
2.本申请提供的处理方法具备工作流中断可恢复能力。编排引擎会持久化工作流在执行过程中产生的过程数据,一旦发生任何异常导致工作流过程被中断,当该异常被修复后,编排引擎会从持久化存储中加载工作流执行的过程数据,在内存中重新创建工作流执行过程被中断的那个时刻的内存映射,从而能够继续推进工作流的执行。2. The processing method provided in this application has the ability to recover from workflow interruptions. The orchestration engine will persist the process data generated by the workflow during execution. Once any exception occurs and the workflow process is interrupted, when the exception is repaired, the orchestration engine will load the process data of the workflow execution from the persistent storage and recreate the memory mapping of the moment when the workflow execution process was interrupted in the memory, so as to continue to advance the execution of the workflow.
3.本申请提供的处理方法是由业务Worker自主决定一次取多少个任务来执行,从而以一种简单、直接的方式实现了工作负载的控制。当业务Worker服务器工作负载高时,可以减少任务执行的数量;当工作负载高时,可以增加任务执行的数量。3. The processing method provided by this application is that the business Worker independently decides how many tasks to execute at a time, thereby achieving workload control in a simple and direct way. When the workload of the business Worker server is high, the number of tasks to be executed can be reduced; when the workload is high, the number of tasks to be executed can be increased.
4.本申请提供的处理技术不与任何应用系统绑定,因此具备较强的兼容各种应用系统的能力。4. The processing technology provided in this application is not bound to any application system, so it has strong compatibility with various application systems.
5.本申请提供的处理技术也不与任何基础设施绑定,比如虚拟化平台,因此具备较强的兼容各种运行环境的能力。5. The processing technology provided in this application is not bound to any infrastructure, such as a virtualization platform, and therefore has a strong ability to be compatible with various operating environments.
6.本申请内置了常用的应用系统的处理恢复能力,能够做到开箱即用。6. This application has built-in processing and recovery capabilities for commonly used application systems and can be used out of the box.
7.本申请将不同应用系统处理流程中的公共部分抽象出来,形成独立的服务供不同业务使用,使得应用系统处理开发人员只需要专注于应用系统本身的处理恢复能力的开发,降低了开发难度,提高了处理的稳定性。7. This application abstracts the common parts of different application system processing flows to form independent services for use by different businesses, so that application system processing developers only need to focus on the development of the application system's own processing and recovery capabilities, reducing development difficulty and improving processing stability.
8.本申请提供的方法具备广泛的适用性。本申请并没有限定自定义的业务Worker的实现方式和实现内容,如编程语言,因此,本方法不仅可以用于处理恢复领域,也可以用于满足其它业务领域,比如灾难恢复演练、数据脱敏等。8. The method provided in this application has wide applicability. This application does not limit the implementation method and content of the customized business Worker, such as programming language. Therefore, this method can be used not only in the field of processing recovery, but also in other business fields, such as disaster recovery drills, data desensitization, etc.
实施例三Embodiment 3
图7是根据本实施例的一种电子设备的硬件结构示意图,如图7所示,该电子设备包括:一个或多个处理器710和存储器720。图7中以一个处理器710为例。Fig. 7 is a schematic diagram of the hardware structure of an electronic device according to this embodiment. As shown in Fig. 7, the electronic device includes: one or more processors 710 and a memory 720. Fig. 7 takes one processor 710 as an example.
所述电子设备还可以包括:输入装置730和输出装置740。The electronic device may further include: an input device 730 and an output device 740 .
所述电子设备中的处理器710、存储器720、输入装置730和输出装置740可 以通过总线或者其他方式连接,图7中以通过总线连接为例。The processor 710, the memory 720, the input device 730 and the output device 740 in the electronic device may The connection is via a bus or other methods. FIG. 7 takes the connection via a bus as an example.
输入装置730可以接收输入的数字或字符信息,输出装置740可以包括显示屏等显示设备。The input device 730 can receive input digital or character information, and the output device 740 can include a display device such as a display screen.
存储器720作为一种计算机可读存储介质,可用于存储软件程序、计算机可执行程序以及模块。处理器710通过运行存储在存储器720中的软件程序、指令以及模块,从而执行多种功能应用以及数据处理,以实现上述实施例中的任意一种方法。The memory 720 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules. The processor 710 executes various functional applications and data processing by running the software programs, instructions and modules stored in the memory 720 to implement any of the methods in the above embodiments.
存储器720可以包括存储程序区和存储数据区,其中,存储程序区可存储操作系统、至少一个功能所需要的应用程序;存储数据区可存储根据电子设备的使用所创建的数据等。此外,存储器可以包括随机存取存储器(Random Access Memory,RAM)等易失性存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件、闪存器件或者其他非暂态固态存储器件。The memory 720 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory may include a volatile memory such as a random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device.
存储器720可以是非暂态计算机存储介质或暂态计算机存储介质。该非暂态计算机存储介质,例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实施例中,存储器720可选包括相对于处理器710远程设置的存储器,这些远程存储器可以通过网络连接至电子设备。上述网络的实例可以包括互联网、企业内部网、局域网、移动通信网及其组合。The memory 720 may be a non-transitory computer storage medium or a transient computer storage medium. The non-transitory computer storage medium may be, for example, at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 720 may optionally include a memory remotely disposed relative to the processor 710, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network may include the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
输入装置730可用于接收输入的数字或字符信息,以及产生与电子设备的用户设置以及功能控制有关的键信号输入。输出装置740可包括显示屏等显示设备。The input device 730 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 740 may include a display device such as a display screen.
本实施例的电子设备还可以包括通信装置750,通过通信网络传输和/或接收信息。The electronic device of this embodiment may further include a communication device 750 for transmitting and/or receiving information via a communication network.
上述实施例方法中的全部或部分流程可以通过计算机程序来执行相关的硬件来完成的,该程序可存储于一个非暂态计算机可读存储介质中,该程序在执行时,可包括如上述方法的实施例的流程,其中,该非暂态计算机可读存储介质可以为磁碟、光盘或随机存取存储器(Read Only Memory,ROM)等。All or part of the processes in the above-mentioned embodiment method can be completed by executing related hardware through a computer program. The program can be stored in a non-transitory computer-readable storage medium. When the program is executed, it can include the processes of the embodiment of the above-mentioned method, wherein the non-transitory computer-readable storage medium can be a magnetic disk, an optical disk or a random access memory (Read Only Memory, ROM), etc.
实施例四Embodiment 4
本实施例还提供一种计算机可读存储介质,存储有计算机可执行指令,所述计算机可执行指令用于执行上述方法。 This embodiment also provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the above method.

Claims (13)

  1. 一种信息处理方法,应用于信息处理系统,所述信息处理系统包括工作流管理模块、编排引擎模块和处理模块,所述方法包括:An information processing method is applied to an information processing system, wherein the information processing system includes a workflow management module, an orchestration engine module, and a processing module, and the method includes:
    通过所述工作流管理模块在接收到客户端的处理请求后,将所述处理请求和目标工作流发送至所述编排引擎模块,所述目标工作流为与所述处理请求对应的工作流;After receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, where the target workflow is the workflow corresponding to the processing request;
    通过所述编排引擎模块确定所述目标工作流中的待处理节点,并将第一待处理信息发送至所述处理模块,所述第一待处理信息为指定类型的待处理节点所对应的待处理信息;Determine the to-be-processed node in the target workflow by the orchestration engine module, and send the first to-be-processed information to the processing module, where the first to-be-processed information is the to-be-processed information corresponding to the to-be-processed node of a specified type;
    通过所述处理模块对所述第一待处理信息进行处理,得到第一处理结果。The first information to be processed is processed by the processing module to obtain a first processing result.
  2. 根据权利要求1所述的方法,其中,所述工作流管理模块包括规约管理单元,在所述通过所述工作流管理模块将所述处理请求和目标工作流发送至所述编排引擎模块之前,所述方法还包括:The method according to claim 1, wherein the workflow management module includes a specification management unit, and before the processing request and the target workflow are sent to the orchestration engine module through the workflow management module, the method further includes:
    通过所述规约管理单元接收工作流,对所述工作流进行合法性校验,并存储校验成功的工作流。The workflow is received through the protocol management unit, the workflow is verified for legality, and the workflow that has successfully been verified is stored.
  3. 根据权利要求2所述的方法,其中,所述工作流管理模块还包括工作流管理单元,所述通过所述工作流管理模块在接收到客户端的处理请求后,将所述处理请求和目标工作流发送至所述编排引擎模块,包括:The method according to claim 2, wherein the workflow management module further comprises a workflow management unit, and the workflow management module, after receiving the processing request from the client, sends the processing request and the target workflow to the orchestration engine module, comprising:
    通过所述工作流管理单元在接收到客户端的处理请求后,获取存储的所述目标工作流;After receiving the processing request from the client, the workflow management unit obtains the stored target workflow;
    通过所述工作流管理单元将所述处理请求和所述目标工作流发送至所述编排引擎模块。The processing request and the target workflow are sent to the orchestration engine module through the workflow management unit.
  4. 根据权利要求1所述的方法,其中,所述编排引擎模块包括调度单元,所述通过所述编排引擎模块确定所述目标工作流中的待处理节点,并将第一待处理信息发送至所述处理模块,包括:The method according to claim 1, wherein the orchestration engine module includes a scheduling unit, and determining the to-be-processed node in the target workflow through the orchestration engine module and sending the first to-be-processed information to the processing module comprises:
    通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点,所述当前待处理节点为在当前处理周期进行处理的待处理节点;Determining, by the scheduling unit, a current node to be processed in the target workflow according to the node information of the target workflow, wherein the current node to be processed is a node to be processed in a current processing cycle;
    通过所述调度单元判断所述当前待处理节点是否为指定类型的待处理节点;Determining, by the scheduling unit, whether the current node to be processed is a node to be processed of a specified type;
    通过所述调度单元在判断所述当前待处理节点为指定类型的待处理节点时,将所述第一待处理信息发送至所述处理模块。When the scheduling unit determines that the current to-be-processed node is a to-be-processed node of a specified type, the first to-be-processed information is sent to the processing module.
  5. 根据权利要求4所述的方法,在所述通过所述处理模块对所述第一待处 理信息进行处理,得到第一处理结果之后,所述方法还包括:The method according to claim 4, wherein the first to-be-processed After the processing information is processed to obtain a first processing result, the method further includes:
    通过所述处理模块将所述第一处理结果返回至所述调度单元;Returning the first processing result to the scheduling unit through the processing module;
    通过所述调度单元接收所述处理模块返回的第一处理结果,并返回执行所述通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点的操作,直到满足结束条件。The scheduling unit receives the first processing result returned by the processing module, and returns to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit until an end condition is met.
  6. 根据权利要求4所述的方法,其中,所述编排引擎模块还包括执行单元,所述方法还包括:The method according to claim 4, wherein the orchestration engine module further comprises an execution unit, and the method further comprises:
    通过所述调度单元在判断所述当前待处理节点不为指定类型的待处理节点时,确定与所述当前待处理节点对应的目标执行单元;When the scheduling unit determines that the current node to be processed is not a node to be processed of a specified type, determining a target execution unit corresponding to the current node to be processed;
    所述调度单元将包含第二待处理信息的处理指令发送至所述目标执行单元。The scheduling unit sends a processing instruction including second information to be processed to the target execution unit.
  7. 根据权利要求6所述的方法,所述方法还包括:The method according to claim 6, further comprising:
    通过所述目标执行单元对所述第二待处理信息进行处理,得到第二处理结果,并将所述第二处理结果返回至所述调度单元;Processing the second information to be processed by the target execution unit to obtain a second processing result, and returning the second processing result to the scheduling unit;
    通过所述调度单元接收所述目标执行单元返回的第二处理结果,并返回执行所述通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点的操作,直到满足结束条件。The second processing result returned by the target execution unit is received by the scheduling unit, and the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit is returned to be executed until an end condition is met.
  8. 根据权利要求4所述的方法,其中,所述通过所述调度单元根据所述目标工作流的节点信息确定所述目标工作流中的当前待处理节点,包括:The method according to claim 4, wherein determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit comprises:
    根据所述目标工作流的节点信息确定所述目标工作流中的当前节点;Determine the current node in the target workflow according to the node information of the target workflow;
    判断所述当前节点是否为待处理节点,响应于所述当前节点为待处理节点,将所述当前节点确定为所述目标工作流中的当前待处理节点;响应于所述当前节点不为待处理节点,返回执行所述根据所述目标工作流的节点信息确定所述目标工作流中的当前节点的操作。Determine whether the current node is a node to be processed; in response to the current node being a node to be processed, determine the current node as the current node to be processed in the target workflow; in response to the current node not being a node to be processed, return to execute the operation of determining the current node in the target workflow according to the node information of the target workflow.
  9. 根据权利要求1所述的方法,其中,所述信息处理系统还包括工作流创建模块,所述方法还包括:The method according to claim 1, wherein the information processing system further comprises a workflow creation module, and the method further comprises:
    通过所述工作流创建模块基于候选节点创建工作流,并将所述工作流发送至所述工作流管理模块。A workflow is created based on the candidate nodes by the workflow creation module, and the workflow is sent to the workflow management module.
  10. 根据权利要求9所述的方法,其中,所述候选节点包括预设节点和自定义节点中的至少之一。The method according to claim 9, wherein the candidate node includes at least one of a preset node and a custom node.
  11. 根据权利要求9所述的方法,其中,所述候选节点包括资源节点,所述 信息处理系统还包括资源管理服务模块,所述方法还包括:The method according to claim 9, wherein the candidate node comprises a resource node, The information processing system also includes a resource management service module, and the method further includes:
    通过所述资源管理服务模块存储所述资源节点的资源信息。The resource information of the resource node is stored through the resource management service module.
  12. 一种电子设备,包括:An electronic device, comprising:
    一个或多个处理器;one or more processors;
    存储器,设置为存储一个或多个程序;a memory configured to store one or more programs;
    当所述一个或多个程序被所述一个或多个处理器执行,使得所述一个或多个处理器实现如权利要求1-11中任一项所述的方法。When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.
  13. 一种计算机可读存储介质,所述计算机可读存储介质存储有计算机可执行指令,所述计算机可执行指令被处理器执行时实现如权利要求1-11中任一项所述的方法。 A computer-readable storage medium stores computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 11.
PCT/CN2023/094144 2022-12-28 2023-05-15 Information processing method WO2024139011A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211699196.9A CN115827331A (en) 2022-12-28 2022-12-28 Processing method
CN202211699196.9 2022-12-28

Publications (1)

Publication Number Publication Date
WO2024139011A1 true WO2024139011A1 (en) 2024-07-04

Family

ID=85519025

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/094144 WO2024139011A1 (en) 2022-12-28 2023-05-15 Information processing method

Country Status (2)

Country Link
CN (1) CN115827331A (en)
WO (1) WO2024139011A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115827331A (en) * 2022-12-28 2023-03-21 上海爱数信息技术股份有限公司 Processing method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106897810A (en) * 2015-12-17 2017-06-27 北京奇虎科技有限公司 Method for processing business and system, workflow engine and system, operation system
US9898471B1 (en) * 2015-08-14 2018-02-20 VCE IP Holding LLC Computer implemented system and method, and computer program product, for generic source control for orchestration workflows
CN114371961A (en) * 2021-12-31 2022-04-19 上海爱数信息技术股份有限公司 Universal application system automatic disaster recovery drilling method and storage medium
CN115150474A (en) * 2022-06-15 2022-10-04 阿里巴巴(中国)有限公司 Information processing method, device, equipment and storage medium
CN115827331A (en) * 2022-12-28 2023-03-21 上海爱数信息技术股份有限公司 Processing method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9898471B1 (en) * 2015-08-14 2018-02-20 VCE IP Holding LLC Computer implemented system and method, and computer program product, for generic source control for orchestration workflows
CN106897810A (en) * 2015-12-17 2017-06-27 北京奇虎科技有限公司 Method for processing business and system, workflow engine and system, operation system
CN114371961A (en) * 2021-12-31 2022-04-19 上海爱数信息技术股份有限公司 Universal application system automatic disaster recovery drilling method and storage medium
CN115150474A (en) * 2022-06-15 2022-10-04 阿里巴巴(中国)有限公司 Information processing method, device, equipment and storage medium
CN115827331A (en) * 2022-12-28 2023-03-21 上海爱数信息技术股份有限公司 Processing method

Also Published As

Publication number Publication date
CN115827331A (en) 2023-03-21

Similar Documents

Publication Publication Date Title
EP3842948B1 (en) Method and apparatus for testing edge computing, device, and readable storage medium
US9015315B2 (en) Identification and monitoring of distributed business transactions
US7779298B2 (en) Distributed job manager recovery
US8370802B2 (en) Specifying an order for changing an operational state of software application components
CN113169952A (en) Container cloud management system based on block chain technology
CN111258565B (en) Method, system, server and storage medium for generating applet
US8538793B2 (en) System and method for managing real-time batch workflows
CN113569987A (en) Model training method and device
WO2024139011A1 (en) Information processing method
CN113157411B (en) Celery-based reliable configurable task system and device
CN106600226B (en) Method and device for optimizing process management system
CN111190732A (en) Timed task processing system and method, storage medium and electronic device
CN110908793A (en) Long-time task execution method, device, equipment and readable storage medium
CN110727575A (en) Information processing method, system, device and storage medium
CN111666141A (en) Task scheduling method, device and equipment and computer storage medium
CN110890987A (en) Method, device, equipment and system for automatically creating cluster
CN114880159A (en) Data processing method, device, equipment and storage medium
CN115242877A (en) Spark collaborative calculation and operation method and device for multiple K8s clusters
CN113220480B (en) Distributed data task cross-cloud scheduling system and method
CN109725916B (en) Topology updating system and method for stream processing
CN110874319A (en) Automated testing method, automated testing platform, automated testing equipment and computer-readable storage medium
CN110895485A (en) Task scheduling system
CN112148420B (en) Abnormal task processing method based on container technology, server and cloud platform
CN116319758A (en) Data migration method, device, electronic equipment and readable storage medium
CN112367205B (en) Processing method and scheduling system for HTTP scheduling request

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23908959

Country of ref document: EP

Kind code of ref document: A1