WO2024139011A1

WO2024139011A1 - Information processing method

Info

Publication number: WO2024139011A1
Application number: PCT/CN2023/094144
Authority: WO
Inventors: 朱劲松; 罗强
Original assignee: 上海爱数信息技术股份有限公司
Priority date: 2022-12-28
Filing date: 2023-05-15
Publication date: 2024-07-04
Also published as: CN115827331A

Abstract

Disclosed in the present application is an information processing method, which is applied to an information processing system. The information processing system comprises a workflow management module, an orchestration engine module and a processing module. The method comprises: after receiving a processing request from a client, by means of a workflow management module, sending the processing request and a target workflow to an orchestration engine module; by means of the orchestration engine module, determining a node to be processed in the target workflow, and sending to a processing module first information to be processed, wherein said first information is information to be processed that corresponds to a node to be processed of a specified type; and processing said first information by means of the processing module, so as to obtain a first processing result. By means of the mutual cooperation among a workflow management module, an orchestration engine module and a processing module in a processing system, data consistency can be ensured and the universality of a processing method can be improved.

Description

Information processing methods

This disclosure claims the priority of the Chinese patent application filed with the China Patent Office on December 28, 2022, with application number 202211699196.9. The entire contents of the above application are incorporated by reference into this application.

Technical Field

The present application relates to the field of data security technology, for example, to an information processing method.

Background technique

With the continuous development of informatization, all walks of life are undergoing digital transformation. Whether it is the upper-level application system or the underlying infrastructure, operating system, middleware, etc. that support the application system, there are many types of users in all walks of life. Therefore, how to handle (such as backup) the increasingly complex and diverse application systems is an urgent problem that needs to be solved.

Some technical solutions of the related art process the subsystems in the application system separately and cannot guarantee the consistency of data; some technical solutions develop corresponding processing logic for different application systems, which has high processing costs and poor versatility.

Summary of the invention

The present application provides an information processing method, which can improve the versatility of the information processing method while ensuring data consistency.

In a first aspect, an embodiment of the present application provides an information processing method, which is applied to an information processing system, wherein the information processing system includes a workflow management module, an orchestration engine module, and a processing module, and the method includes:

After receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, where the target workflow is the workflow corresponding to the processing request;

Determine the to-be-processed node in the target workflow by the orchestration engine module, and send the first to-be-processed information to the processing module, where the first to-be-processed information is the to-be-processed information corresponding to the to-be-processed node of a specified type;

The first information to be processed is processed by the processing module to obtain a first processing result.

In a second aspect, an embodiment of the present application provides an electronic device, the electronic device comprising one or more A processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and when executed by the one or more processors, the above method is performed.

In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to execute the above method.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be introduced below. The drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a flow chart of an information processing method provided according to Embodiment 1 of the present application;

FIG2 is a structural diagram of a workflow definition model provided according to Embodiment 1 of the present application;

FIG3 is a structural diagram of a node model provided according to Embodiment 1 of the present application;

FIG4 is a flow chart of an information processing method provided according to Embodiment 2 of the present application;

FIG5 is a structural diagram of an information processing method provided according to Embodiment 2 of the present application;

FIG6 is a structural diagram of a workflow provided according to Embodiment 2 of the present application;

FIG. 7 is a schematic diagram of the hardware structure of an electronic device provided according to Embodiment 3 of the present application.

Detailed ways

In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. The described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those listed steps or units, but may include other steps or units that are not listed or inherent to these processes, methods, products or apparatuses. unit.

Embodiment 1

Figure 1 is a flowchart of an information processing method provided according to Example 1 of the present application. This embodiment can be applicable to the situation of processing the system. The method can be executed by an information processing device. The information processing device can be implemented in the form of hardware and/or software. The information processing device can be configured in a processing system.

It can be considered that in the early days of computer applications, application systems were usually stand-alone systems, and data was stored in a stand-alone database system. In this case, only the stand-alone database needs to be processed, such as backed up, to complete the backup of the application system.

With the continuous development of informatization, all walks of life are undergoing digital transformation. The trend of steady-state IT + agile IT construction is becoming more and more obvious. Whether it is the upper-level application system or the underlying infrastructure, operating system, middleware, etc. that support the application system, there are many types of users in all walks of life. Therefore, how to deal with increasingly complex and diverse application systems is a problem that current processing technology urgently needs to solve.

The following description uses backup as an example: Assume that a product AnyShare includes three subsystems: Object Storage Service (OSS) gateway, MongoDB (an unstructured database) and MySQL (a relational database), and each subsystem is a distributed cluster system. Then, to perform a complete and correct backup of AnyShare, it is necessary to back up the data in the three subsystems separately and ensure the data consistency among the three subsystems. Otherwise, due to the inconsistency of the restored data among the three subsystems, some data will be unavailable or lost.

There are several types of backup methods in the related art: the first type is to back up the subsystems under each application system separately. However, since each backup task is independent, data consistency cannot be guaranteed.

The second type is to adapt each application system separately based on a deep understanding of the implementation principles of each application system, that is, to develop backup logic suitable for each different application system.

Exemplarily, the backup method may be: start the OSS gateway backup first, but proceed to the next step without waiting for the OSS gateway backup to complete; back up MongoDB, and proceed to the next step after the backup is complete; back up MySQL, and proceed to the next step after the backup is complete; notify the OSS gateway; wait for the OSS gateway backup to complete, and when the OSS gateway backup is completed, the AnyShare application system backup is completed.

It can be seen that this backup method is specific to the AnyShare product and is not applicable to the backup of other application systems. Therefore, this method requires the development of proprietary backup logic for each application system, which is time-consuming and labor-intensive and has poor versatility.

The third category requires that the application system be deployed on a virtualization platform. For example, the backup method is You can also: add the virtual machines where the three AnyShare subsystems are located to a group; execute the script, suspend the input and output (IO) operations of the virtual machines, and create snapshots for each virtual machine; read data from the snapshots for backup. Therefore, this method requires that the application system must be deployed on the virtual machine, and the backup is the entire virtual host, not just the application system.

In summary, the information processing methods of related technologies cannot solve the processing problems of complex application systems very well and have poor versatility.

Based on this, the embodiment of the present application provides an information processing method, which can meet the processing of different types and complex application systems and has strong versatility. As shown in Figure 1, the method includes:

S110. After receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, where the target workflow is the workflow corresponding to the processing request.

The workflow management module can be considered as a module that manages workflows in the information processing system, such as being responsible for managing sent workflows, verifying the legitimacy of workflows, and processing workflow commands from clients. Workflow commands may include: create, edit, delete, query, start, stop, etc.

A processing request may refer to a request by a client to process a workflow, and the processing of a workflow may include, for example, backup, recovery and/or data consistency management, etc. The target workflow may be understood as the workflow in the workflow management module corresponding to the processing request.

Exemplarily, after receiving the processing request from the client, the workflow management module can send the processing request and the target workflow to the orchestration engine module for subsequent processing. It can be considered that the workflow management module can store one or more workflows to wait for the processing request from the client. After receiving the processing request from the client, the processing request and the corresponding target workflow can be sent to the orchestration engine module for subsequent processing. Among them, the source of the workflow can be created and obtained by other modules in the information processing system, or it can be directly input from other systems.

In one embodiment, the workflow management module includes a protocol management unit, and before sending the processing request and the target workflow to the orchestration engine module through the workflow management module, the method further includes:

The workflow is received through the protocol management unit, the workflow is verified for legality, and the workflow that succeeds in verification is stored.

The protocol management unit can be considered as a unit in the workflow management module that verifies the workflow and is used to verify the validity of the workflow.

In one embodiment, before the workflow management module sends the processing request and the target workflow to the orchestration engine module, the protocol management unit in the workflow management module can receive the workflow, perform a validity check on the workflow, and store the workflow that has successfully been verified. The verification method may include, for example, determining whether the workflow satisfies preset verification conditions. For example, the verification conditions may include a workflow definition model and/or processing and recovery business rule constraints, etc. The workflow definition model and processing and recovery business rule constraints may be considered as rules pre-set by relevant personnel for verifying the legitimacy of the workflow.

Figure 2 is a structural diagram of a workflow definition model provided according to Example 1 of the present application. As shown in Figure 2, the workflow definition model can be considered as a directed acyclic graph from the entrance (i.e., the start node) to the exit (i.e., the end node). Each node is regarded as a node in the graph. Each node is connected by a connecting line. The node pointed by the arrow is called the downstream node, and the node connected by the tail of the arrow is called the upstream node. The data of the upstream node is allowed to be passed to the downstream node, for example, the data of resource node 2 is passed to node 3.

In one embodiment, the workflow management module further includes a workflow management unit, and after receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, including:

After receiving the processing request from the client, the workflow management unit obtains the stored target workflow;

The processing request and the target workflow are sent to the orchestration engine module through the workflow management unit.

The workflow management unit can be considered as a unit in the workflow management module that manages the workflow, such as being responsible for managing the sending of workflows and processing client processing requests.

In one embodiment, after receiving a processing request from a client, the workflow management unit may first obtain a target workflow corresponding to the processing request in the stored workflows, such as determining and obtaining the target workflow based on identification information in the processing request; and then after obtaining the target workflow, send the processing request and the target workflow to the orchestration engine module for subsequent processing.

S120: Determine a to-be-processed node in the target workflow through the orchestration engine module, and send first to-be-processed information to the processing module, where the first to-be-processed information is to-be-processed information corresponding to a to-be-processed node of a specified type.

The orchestration engine module can be considered as a module that orchestrates workflows in an information processing system. In this embodiment, the orchestration engine module can be responsible for executing the target workflow, for example, it can be: receiving the start and stop commands (i.e., processing requests) and workflow definitions (i.e., target workflows) from the workflow management module; parsing workflow data and generating new workflow instances; starting from the start node of the workflow, according to the upstream and downstream relationships between nodes, node transfer rules, node execution rules, workflow execution progress and other information, cyclically placing executable nodes in the scheduling queue and waiting for the execution unit to fetch them. In addition, all process data generated during the execution of the workflow will be persistently stored by the orchestration engine, and in the event of an abnormal situation. After that, even if the server loses power, data will not be lost. After the exception is fixed, the orchestration engine module will reload the interrupted workflow from the persistent storage, restore the memory mapping, and continue to execute the workflow.

The node to be processed may refer to a node to be processed in the target workflow. The number of nodes to be processed may be one or more, and the number of nodes to be processed may be determined based on the actual content of the workflow. The specified type may be a pre-specified type. In this embodiment, the specified type may be a user-defined node, such as a service node. The first information to be processed may be the information to be processed corresponding to the node to be processed of the specified type, and the information to be processed may be understood as information corresponding to the node, such as a node identifier.

In this embodiment, after receiving the processing request and the target workflow, the orchestration engine module can determine the to-be-processed nodes in the target workflow, and then send the first to-be-processed information corresponding to the to-be-processed nodes of the specified type in the to-be-processed nodes to the processing module for subsequent node processing. Wherein, the process of determining the to-be-processed nodes and sending the first to-be-processed information to the processing module includes, for example, that the orchestration engine module can judge all nodes contained in the target workflow after receiving the processing request and the target workflow to obtain all the to-be-processed nodes, and send the first to-be-processed information corresponding to the to-be-processed nodes of the specified type in the to-be-processed nodes to the processing module; the orchestration engine module can also judge the nodes in the target workflow from left to right in sequence, and after the current node is determined to be the to-be-processed node, judge the type of the to-be-processed node again, and after the judgment, send the first to-be-processed information corresponding to the to-be-processed node of the specified type to the processing module, and then continue to make similar judgments on the next node until all nodes in the target workflow are judged.

In one embodiment, the means for determining the node to be processed in the target workflow may be determined according to the type of the node, or may be determined according to the node identifier of the node, etc.

In one embodiment, the timing of sending the first information to be processed to the processing module may be, for example, directly sending the first information to be processed to the processing module after determining the node to be processed, or when the orchestration engine module receives the task request of the processing module, sending the first information to be processed corresponding to the task request to the processing module, and the task request may be used for the processing module to request whether there is an executable processing task. The number of first information to be processed sent to the processing module may be, for example, one or more first information to be processed may be sent to the processing module at the same time.

S130: Process the first information to be processed by the processing module to obtain a first processing result.

The processing module can be considered as a module that processes the first information to be processed; the first processing result is the result of processing the first information to be processed.

Exemplarily, when the processing module receives the first information to be processed sent by the orchestration engine module, it can process the first information to be processed to obtain a first processing result. In this embodiment, the processing module may include one or more business workers, each of which performs a corresponding type of processing task. After the processing module receives the first information to be processed sent by the orchestration engine module, the service Worker can be determined according to the identification information in the first information to be processed, and then the first information to be processed is processed by the determined service Worker to obtain a first processing result.

Exemplarily, in one embodiment, when a business worker is started, the business worker connects to the orchestration engine module to query whether there are executable tasks. If there are, the orchestration engine module returns the details of the task to be executed (i.e., the first pending information) to the business worker. After receiving the task details, the business worker starts to execute the task. After the task is executed, the task execution result (i.e., the first processing result) can be reported to the orchestration engine module, and the next executable task can be requested at the same time.

In one embodiment, the business worker can flexibly request an indefinite number of executable tasks from the orchestration engine module according to the workload of the server where the business worker is located. For example, when the workload of the server where the business worker is located is light, multiple executable tasks can be requested at a time, otherwise, one executable task can be requested. On this basis, the execution efficiency of the workflow is greatly improved.

In one implementation, in order to avoid the business worker frequently initiating polls to the orchestration engine module when there are no executable tasks, the orchestration engine module may not immediately feedback to the business worker that there are no executable tasks, but instead maintain the connection until an executable task arrives, and then return the executable task to the business worker.

An information processing method provided in the first embodiment of the present application is applied to an information processing system, wherein the information processing system includes a workflow management module, an orchestration engine module and a processing module, wherein the method includes: after receiving a processing request from a client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, wherein the target workflow is the workflow corresponding to the processing request; the orchestration engine module determines the to-be-processed node in the target workflow, and sends the first to-be-processed information to the processing module, wherein the first to-be-processed information is the to-be-processed information corresponding to the to-be-processed node of a specified type; and the processing module processes the first to-be-processed information to obtain a first processing result. By using this method, through the mutual cooperation between the workflow management module, the orchestration engine module and the processing module in the information processing system, the versatility of the information processing method can be improved while ensuring data consistency.

In one embodiment, the information processing system further includes a workflow creation module, and the method further includes:

A workflow is created based on the candidate nodes by the workflow creation module, and the workflow is sent to the workflow management module.

The workflow creation module can be used to create a workflow; the candidate node can be considered as a node for the workflow creation module to create a workflow.

In one embodiment, the candidate nodes include preset nodes and/or custom nodes.

A preset node can refer to a pre-configured general node, which is the basic building block of a workflow. The preset node can include different types of nodes, such as notification nodes or gateway nodes, etc. A custom node can be considered a user-defined node. Different types of nodes can implement different functions, and different business needs can be met by combining different types of nodes.

In this step, the workflow creation module can create a workflow based on the candidate nodes, and then send the created workflow to the workflow management module for subsequent legality verification, etc., wherein the method of creating the workflow is not limited, as long as the workflow can be obtained.

Illustratively, Tables 1-4 are several types of candidate nodes provided in this embodiment. For example, Table 1 is an event/message type node, which may include a start node, an end node, a message node, a timer node, and the like.

Table 2 shows the gateway type nodes, which may include parallel gateway nodes, parallel gateway synchronization nodes, conditional gateway nodes, multi-way selection gateway nodes, multi-way merging gateway nodes, and event gateway nodes.

Table 3 shows the business nodes, which may include script nodes, Service nodes, and various application nodes. The Service node may be a custom node, that is, any node defined by the user.

Table 4 shows notification nodes, which may include email notification nodes, SMS notification nodes, and instant message notification nodes.

In addition to a large number of built-in general nodes, the orchestration engine module also provides a general Service task node that can be used to adapt to any application system. Its business parameters support the use of JSON (JavaScript Object Notation) or YAML format, making it compatible with the business parameters of any application.

Figure 3 is a structural diagram of a node model provided according to Example 1 of the present application. As shown in Figure 3, for a node in a workflow, its node information may include name, type, provider, upstream and downstream nodes, input/output parameters, etc., and the node information is used for subsequent processing.

In one embodiment, the modeler may be responsible for providing users with workflow building services, based on which users may build workflows for different processes according to different business scenarios.

The process of building a workflow may be, for example, building a basic workflow consisting of multiple nodes by dragging and connecting in a modeler, and configuring the nodes in the basic workflow to form a usable automated workflow.

The process of building a workflow can also be, for example, as follows: the user manually writes a JSON/YAMl file defining the workflow according to the workflow definition model; after writing, put it in the directory specified by the modeler, and you can see the defined workflow in the modeler. You can also manually write a file as a custom node and put it into the modeler for calling and use.

In one embodiment, the candidate node includes a resource node, the information processing system further includes a resource management service module, and the method further includes:

The resource information of the resource node is stored through the resource management service module.

The resource management service module can be considered as a module for managing resources. For example, resource information can be stored in the resource management service module. The resource information includes information of each resource platform. The resource platforms include virtualization platforms, cloud platforms, object storage, cloud native platforms and/or physical hosts, etc. The resource information may include: resource type, resource access address, resource authorized users and/or resource type, etc.

In one embodiment, the candidate nodes may include resource nodes. When creating a workflow, the resource nodes may be selected for creation, and then the resource information bound to the resource nodes may be used to process the nodes.

Embodiment 2

Figure 4 is a flow chart of a processing method provided according to Embodiment 2 of the present application, and Embodiment 2 is optimized on the basis of the above embodiments. In this embodiment, determining the to-be-processed node in the target workflow by the orchestration engine module, and sending the first to-be-processed information to the processing module may include: determining the current to-be-processed node in the target workflow according to the node information of the target workflow by the scheduling unit, the current to-be-processed node being the to-be-processed node processed in the current processing cycle; judging by the scheduling unit whether the current to-be-processed node is a to-be-processed node of a specified type; and sending the first to-be-processed information to the processing module when the scheduling unit judges that the current to-be-processed node is a to-be-processed node of a specified type.

At the same time, the orchestration engine module of the second embodiment also includes an execution unit, and the method also includes: determining, by the scheduling unit, a target execution unit corresponding to the current node to be processed when it is determined that the current node to be processed is not a node to be processed of a specified type; and the scheduling unit sends a processing instruction containing the second information to be processed to the target execution unit.

For details not yet provided in this embodiment, please refer to Embodiment 1.

As shown in FIG4 , the method includes:

S210: After receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module.

S220: Determine, by the scheduling unit, a current node to be processed in the target workflow according to the node information of the target workflow.

Among them, the current node to be processed is the node to be processed in the current processing cycle. It can be considered that in this embodiment, the nodes to be processed can be processed in sequence, that is, after the current processing node is processed, the next node to be processed is processed. The current processing cycle is the current processing cycle, and the current node to be processed is the node that needs to be processed in the current processing cycle. The scheduling unit can be considered as a unit that performs scheduling in the orchestration engine module, which is used to determine the current node to be processed and send the first processing information to the processing module. The node information can be understood as the information of the node in the target workflow, such as the upstream and downstream relationship of the node, the workflow execution progress, etc.

In this step, the scheduling unit may determine the current to-be-processed node in the target workflow according to the node information of the target workflow, and the determination process may be determined according to the actual situation of the node.

In one embodiment, determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit includes:

Determine the current node in the target workflow according to the node information of the target workflow;

Determine whether the current node is a node to be processed. If so, determine the current node as the current node to be processed in the target workflow; if not, return to execute the operation of determining the current node in the target workflow according to the node information of the target workflow.

Exemplarily, in the process of determining the current node to be processed in the target workflow according to the node information of the target workflow, the current node in the target workflow can be first determined according to the node information of the target workflow, and then it is determined whether the current node is a node to be processed. If the current node is a node to be processed, it means that the current node needs to be processed, then the current node can be determined as the current node to be processed in the target workflow to execute subsequent steps; if the current node is not a node to be processed, it means that the current node does not need to be processed, then the next node can be continued to be judged, and the operation of determining the current node in the target workflow according to the node information of the target workflow is returned, that is, the next node is taken as the current node, and it is continued to be determined whether the current node is a node to be processed, and the above operation is repeated.

S230. Determine, by the scheduling unit, whether the current node to be processed is a node to be processed of a specified type. If so, execute S240; if not, execute S250-S260.

After the current node to be processed is determined, the type of the current node to be processed may be judged again, for example, whether the current node to be processed is a node to be processed of a specified type, and different steps may be executed according to the result of the judgment.

Exemplarily, if the current node to be processed is a node to be processed of a specified type, indicating that the current node to be processed is a specified node, then steps S240 - S250 may be executed, and the process returns to execute step S220 until the end condition is met.

If the current node to be processed is not a node to be processed of the specified type, it means that the current node to be processed is a node of other types except the specified type, then steps S260-S270 can be executed, and the process returns to step S220 until the end condition is met.

S240. Send the first information to be processed to the processing module through the scheduling unit.

S250: Process the first information to be processed by the processing module to obtain a first processing result.

In one embodiment, after the first information to be processed is processed by the processing module to obtain a first processing result, the method further includes:

Returning the first processing result to the scheduling unit through the processing module;

The scheduling unit receives the first processing result returned by the processing module, and returns to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit until an end condition is met.

It can be considered that after the processing module processes the first information to be processed and obtains the first processing result, the processing module can return the first processing result to the scheduling unit; then, after the scheduling unit receives the first processing result returned by the processing module, it can continue to return to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow through the scheduling unit until the end condition is met. The end condition can be, for example, that the current node is the end node.

S260. Determine, by the scheduling unit, a target execution unit corresponding to the current node to be processed.

In this embodiment, the current node to be backed up is a node of a type other than the specified type, and the second information to be processed can be considered as the information to be processed corresponding to the current node to be backed up. The target execution unit can be considered as the execution unit corresponding to the current node to be processed in the orchestration engine module, which is used to process the second information to be processed.

In this embodiment, after the scheduling unit determines that the current node to be processed is not a node to be processed of the specified type, the target execution unit can be determined, and then a processing instruction containing the second information to be processed is sent to the target execution unit to instruct the target execution unit to perform processing.

S270: The scheduling unit sends a processing instruction including the second information to be processed to the target execution unit.

In one embodiment, the method further comprises:

Processing the second information to be processed by the target execution unit to obtain a second processing result, and returning the second processing result to the scheduling unit;

The second processing result returned by the target execution unit is received by the scheduling unit, and the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit is returned to be executed until an end condition is met.

The second processing result is the result of the target execution unit processing the second information to be processed.

In one embodiment, after receiving the second information to be processed, the target execution unit can process it to obtain a second processing result, and then return the second processing result to the scheduling unit; after receiving the second processing result returned by the target execution unit, the scheduling unit can return to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow until the end condition is met.

A processing method provided in the second embodiment of the present application can provide different processing logics for different nodes to be processed by determining the current node to be processed according to the node information of the target workflow and judging whether the current node to be processed is a node to be processed of a specified type, thereby ensuring data consistency while ensuring data consistency. When processing, improve the efficiency of processing.

FIG. 5 is a structural diagram of a processing method provided according to Embodiment 2 of the present application. As shown in FIG. 5 , the processing method may include the following steps:

(1) Resource registration. First, various resources can be registered in the resource management service module. After successful registration, the resource management service module will monitor the successfully registered resources to update the resource status in real time (i.e., the resource information of the resource node is stored by the resource management service module).

(2) Creating a workflow. The workflow creation module can use a modeler, drag and drop with a mouse, or create a workflow definition JSON/YAML file to create a workflow (ie, create a workflow based on candidate nodes through the workflow creation module).

(3) Publishing workflows. When publishing a workflow, the protocol manager (i.e., protocol management unit) in the workflow management module will verify the validity of the workflow. Only workflows that pass the validity verification will be successfully published and persisted in the storage (i.e., the protocol management unit receives the workflow, verifies the validity of the workflow, and stores the successfully verified workflow). Workflows that fail to pass the validity verification will not be published successfully, but can be published again after modification.

(4) Scheduling workflows. After the workflow is successfully published, you can start running the workflow. At startup, the parser in the orchestration engine module will load the workflow from the storage and parse it to generate a memory map; the scheduler (i.e., the scheduling unit) will generate a workflow instance based on the memory map, and select the nodes to be executed based on the upstream and downstream relationships of the nodes, node flow rules, node execution rules, workflow execution progress and other information (i.e., determine the current node to be processed in the target workflow based on the node information of the target workflow, and the current node to be processed is the node to be processed in the current processing cycle), and add the nodes to be executed to the queue to be executed, and then for different classes of nodes to be executed, The type performs the corresponding operation (i.e., determines whether the current node to be processed is a node to be backed up of the specified type; when the current node to be processed is a node to be backed up of the specified type, the first information to be processed is sent to the processing module through the scheduling unit); otherwise, generates an executor (i.e., an execution unit) of the corresponding node, and hands the node to the executor for execution (i.e., when the current node to be processed is not a node to be backed up of the specified type, the scheduling unit determines the target execution unit corresponding to the current node to be processed; the scheduling unit sends the processing instruction containing the second information to be processed to the target execution unit).

(5) Execute the Service node task. When the business worker is ready to execute the task, it connects to the orchestration engine module to inquire whether there is a task that can be executed; the orchestration engine module will take out the matching node task (i.e., the first information to be processed) from the queue to be executed according to the requested task type in the business worker, and send it to the business worker (i.e., the processing module); after the business worker completes the task, it will report to the orchestration engine module that the task has been completed and the execution result (i.e., the first processing result is returned to the scheduling unit through the processing module).

(6) Perform the next round of scheduling. When the orchestration engine module receives the task completion message reported by the business Worker, it selects the downstream node to be executed based on the execution result of the current node task, the execution status of the upstream node, and the downstream node, and enters a new round of execution process (i.e., the scheduling unit receives the first processing result returned by the processing module, and returns to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit, until the end condition is met).

The following is an exemplary description of the processing method provided in the embodiment of the present application, assuming that the AnyShare application system is composed of three subsystems: OSS gateway, MongoDB and MySQL.

1. Develop application-specific processing recovery nodes.

Since the OSS gateway is a user-specific application system component, it is necessary to develop OSS processing recovery business nodes (i.e. candidate nodes) and business workers (i.e. processing modules). In this embodiment, a general Service node can be used in the modeler and renamed as an OSS node to represent the OSS gateway application. Therefore, there is no need to develop additional specific processing recovery business nodes.

To develop the OSS gateway application's processing and recovery service Worker, you only need to implement two interfaces: cyclically call the interface of the orchestration engine (i.e., the orchestration engine module) to request the currently executable OSS tasks; when the OSS processing or recovery is completed, report the completion to the orchestration engine and report the execution result (i.e., the first processing result).

2. Unified resource management

Usually in the field of disaster recovery, the source of data is defined as the source data end and the destination of data is defined as the destination end. The source data end of the recovery is the processed data generated by various application systems through processing technology; the destination end is the destination of the processed data of the application system.

Different types of application systems require different forms of drill resources for drills, including physical clients, virtualization platforms, cloud platforms, object storage, etc. Therefore, it is necessary to meet the compatibility and universality requirements of application systems and solve the problem of unified management of recovery resources.

First, you can register the virtualization platform, cloud platform, object storage, cloud native platform, and physical host to the resource management module. The registration information includes: resource type, resource access address, resource authorized user, resource type, etc.

Then, after the resource management module monitors that a resource has been successfully registered, it can extract the resource access address and resource authorized user from the resource registration information, and collect resource information on the corresponding resource platform based on this information.

After the initial collection of resource information is completed, the change information of the resource platform is monitored. When the resource platform is changed, new resource information is collected in real time into the information processing system (ie, the resource information of the resource node is stored through the resource management service module).

Finally, the collected resources are tagged with application system labels to indicate that the resources can provide a running environment for those application systems.

3. Workflow Construction

There are many types of application systems in various industries. A single disaster recovery system cannot cover the processing and recovery of all application systems, but it can usually cover the processing and recovery of the components that make up these application systems. For example, the components of an application system: database, middleware, configuration files, etc., can be processed and recovered for application systems that are not supported by such disaster recovery systems through combined orchestration.

First, you can open the modeler and click "New", the modeler will expand an editable canvas; drag candidate nodes from the node list on the right to the canvas; connect multiple nodes with connecting lines; except for the start node and the end node, you can set the parameters of each node. After everything is completed, you can click "Publish", and the modeler will send the definition of the workflow to the workflow management module (that is, create a workflow based on the candidate nodes through the workflow creation module, and send the workflow to the workflow management module).

Figure 6 is a structural diagram of a workflow provided according to Example 2 of the present application. As shown in Figure 6, the workflow may include a start node, a policy node, an OSS gateway node, a MongoDB node, a MySQL node, a message node and an end node. When setting the parameters of the OSS node, the MongoDB node and the MySQL node, the system will automatically adapt according to the application type and resource tag to select the processing destination.

Then, the workflow management module will verify the legitimacy of the workflow, and it will be persistent after the verification (i.e., the workflow is received through the protocol management unit, the workflow is verified for legitimacy, and the workflow that succeeds the verification is stored). After the workflow management module receives the "start" command, it will extract the workflow definition from the database, and then send the command together with the workflow definition to the orchestration engine module (i.e., after receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module).

After receiving the workflow command, the scheduler in the orchestration engine module will generate a workflow instance according to the definition of the workflow; the scheduler finds the start node of the workflow, and searches forward along the connection line from the start node to find the first node to be executed (that is, the scheduling unit determines the current node to be processed in the target workflow according to the node information of the target workflow, and the current node to be processed is the node to be processed in the current processing cycle; the scheduling unit determines whether the current node to be processed is a node to be processed of a specified type; the scheduling unit sends the first information to be processed to the processing module when the current node to be processed is a node to be processed of a specified type). Here, it is the policy node. Since the policy node does not have the business logic of the instance, the scheduler will skip the policy node and continue to search forward to find the node to be executed, which is the OSS node here. The scheduler generates an OSS node instance and adds it to the queue to be executed, waiting for the OSS Worker to take it. Since the OSS node task has not been completed, the scheduler suspends the scheduling of the workflow. After waiting for a period of time, the OSS Worker Ask the orchestration engine whether there is an executable activity. At this time, the orchestration engine will return the OSS task to the OSS Worker (that is, the scheduling unit determines the current node to be processed in the target workflow according to the node information of the target workflow, and the current node to be processed is the node to be processed in the current processing cycle; the scheduling unit determines whether the current node to be processed is a node to be processed of a specified type; when the scheduling unit determines that the current node to be processed is a node to be processed of a specified type, the first to-be-processed information is sent to the processing module).

When the OSS Worker completes the task, it will report to the orchestration engine that the task has been completed and report the execution result (that is, the first processing result is returned to the scheduling unit through the processing module).

After receiving the results reported by the OSS Worker, the orchestration engine scheduler will update the execution status of the OSS node task to completion, and synchronously update the execution result. If the result is successful, the scheduler will continue to look for the downstream node of the OSS node (that is, receive the first processing result returned by the processing module through the scheduling unit, and return to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow through the scheduling unit), that is, the MongoDB and MySQL nodes, because these two nodes are the direct downstream nodes of the OSS node. The scheduler generates node tasks for the two nodes, MongoDB and MySQL, and adds these two tasks to the queue to be executed.

Similarly, when MongoDB Worker and MySQL Worker ask the orchestration engine whether there are any executable activities, the orchestration engine will return the two tasks to MongoDB Worker and MySQL Worker respectively.

Similarly, after MongoDB Worker and MySQL Worker complete their respective tasks, they report the task execution results to the orchestration engine respectively.

After receiving the results reported by MongoDB Worker and MySQL Worker, the orchestration engine scheduler will update the execution status of the MongoDB node task and the MySQL node task to completion, and synchronously update the execution results. If the result is successful, the scheduler will continue to look for downstream nodes, namely message nodes. Since the execution of message nodes does not require the participation of business workers, after the scheduler generates message node tasks, they are directly handed over to the orchestration engine for execution. In this embodiment, the orchestration engine will send the specified message to the OSS Worker.

After receiving the specified message, the OSS Worker will continue to execute the remaining unexecuted business logic until completion, and report the execution result to the orchestration engine (i.e., process the second to-be-processed information through the target execution unit, obtain the second processing result, and return the second processing result to the scheduling unit).

The orchestration engine scheduler continues to search for the downstream node of the message node, which is the end node in this embodiment. At this point, the scheduler ends the operation of the workflow, thereby completing the processing of the entire workflow (i.e., the scheduling unit receives the second processing result returned by the target execution unit and returns the execution result). The operation of the current node to be processed in the target workflow is determined by the scheduling unit according to the node information of the target workflow until an end condition is met).

It can be seen that compared with the related art, the embodiments of the present application include the following beneficial effects:

1. By decomposing various complex application systems, constructing multiple business Worker nodes, and providing the ability to orchestrate these business Worker nodes, a new capability can be formed. This new capability can be used to meet the processing and recovery needs of different users, different application systems, and different business scenarios.

2. The processing method provided in this application has the ability to recover from workflow interruptions. The orchestration engine will persist the process data generated by the workflow during execution. Once any exception occurs and the workflow process is interrupted, when the exception is repaired, the orchestration engine will load the process data of the workflow execution from the persistent storage and recreate the memory mapping of the moment when the workflow execution process was interrupted in the memory, so as to continue to advance the execution of the workflow.

3. The processing method provided by this application is that the business Worker independently decides how many tasks to execute at a time, thereby achieving workload control in a simple and direct way. When the workload of the business Worker server is high, the number of tasks to be executed can be reduced; when the workload is high, the number of tasks to be executed can be increased.

4. The processing technology provided in this application is not bound to any application system, so it has strong compatibility with various application systems.

5. The processing technology provided in this application is not bound to any infrastructure, such as a virtualization platform, and therefore has a strong ability to be compatible with various operating environments.

6. This application has built-in processing and recovery capabilities for commonly used application systems and can be used out of the box.

7. This application abstracts the common parts of different application system processing flows to form independent services for use by different businesses, so that application system processing developers only need to focus on the development of the application system's own processing and recovery capabilities, reducing development difficulty and improving processing stability.

8. The method provided in this application has wide applicability. This application does not limit the implementation method and content of the customized business Worker, such as programming language. Therefore, this method can be used not only in the field of processing recovery, but also in other business fields, such as disaster recovery drills, data desensitization, etc.

Embodiment 3

Fig. 7 is a schematic diagram of the hardware structure of an electronic device according to this embodiment. As shown in Fig. 7, the electronic device includes: one or more processors 710 and a memory 720. Fig. 7 takes one processor 710 as an example.

The electronic device may further include: an input device 730 and an output device 740 .

The processor 710, the memory 720, the input device 730 and the output device 740 in the electronic device may The connection is via a bus or other methods. FIG. 7 takes the connection via a bus as an example.

The input device 730 can receive input digital or character information, and the output device 740 can include a display device such as a display screen.

The memory 720 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules. The processor 710 executes various functional applications and data processing by running the software programs, instructions and modules stored in the memory 720 to implement any of the methods in the above embodiments.

The memory 720 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory may include a volatile memory such as a random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device.

The memory 720 may be a non-transitory computer storage medium or a transient computer storage medium. The non-transitory computer storage medium may be, for example, at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 720 may optionally include a memory remotely disposed relative to the processor 710, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network may include the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The input device 730 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 740 may include a display device such as a display screen.

The electronic device of this embodiment may further include a communication device 750 for transmitting and/or receiving information via a communication network.

All or part of the processes in the above-mentioned embodiment method can be completed by executing related hardware through a computer program. The program can be stored in a non-transitory computer-readable storage medium. When the program is executed, it can include the processes of the embodiment of the above-mentioned method, wherein the non-transitory computer-readable storage medium can be a magnetic disk, an optical disk or a random access memory (Read Only Memory, ROM), etc.

Embodiment 4

This embodiment also provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the above method.

Claims

An information processing method is applied to an information processing system, wherein the information processing system includes a workflow management module, an orchestration engine module, and a processing module, and the method includes:

After receiving the processing request from the client, the workflow management module sends the processing request and the target workflow to the orchestration engine module, where the target workflow is the workflow corresponding to the processing request;

Determine the to-be-processed node in the target workflow by the orchestration engine module, and send the first to-be-processed information to the processing module, where the first to-be-processed information is the to-be-processed information corresponding to the to-be-processed node of a specified type;

The first information to be processed is processed by the processing module to obtain a first processing result.
The method according to claim 1, wherein the workflow management module includes a specification management unit, and before the processing request and the target workflow are sent to the orchestration engine module through the workflow management module, the method further includes:

The workflow is received through the protocol management unit, the workflow is verified for legality, and the workflow that has successfully been verified is stored.
The method according to claim 2, wherein the workflow management module further comprises a workflow management unit, and the workflow management module, after receiving the processing request from the client, sends the processing request and the target workflow to the orchestration engine module, comprising:

After receiving the processing request from the client, the workflow management unit obtains the stored target workflow;

The processing request and the target workflow are sent to the orchestration engine module through the workflow management unit.
The method according to claim 1, wherein the orchestration engine module includes a scheduling unit, and determining the to-be-processed node in the target workflow through the orchestration engine module and sending the first to-be-processed information to the processing module comprises:

Determining, by the scheduling unit, a current node to be processed in the target workflow according to the node information of the target workflow, wherein the current node to be processed is a node to be processed in a current processing cycle;

Determining, by the scheduling unit, whether the current node to be processed is a node to be processed of a specified type;

When the scheduling unit determines that the current to-be-processed node is a to-be-processed node of a specified type, the first to-be-processed information is sent to the processing module.
The method according to claim 4, wherein the first to-be-processed After the processing information is processed to obtain a first processing result, the method further includes:

Returning the first processing result to the scheduling unit through the processing module;

The scheduling unit receives the first processing result returned by the processing module, and returns to execute the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit until an end condition is met.
The method according to claim 4, wherein the orchestration engine module further comprises an execution unit, and the method further comprises:

When the scheduling unit determines that the current node to be processed is not a node to be processed of a specified type, determining a target execution unit corresponding to the current node to be processed;

The scheduling unit sends a processing instruction including second information to be processed to the target execution unit.
The method according to claim 6, further comprising:

Processing the second information to be processed by the target execution unit to obtain a second processing result, and returning the second processing result to the scheduling unit;

The second processing result returned by the target execution unit is received by the scheduling unit, and the operation of determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit is returned to be executed until an end condition is met.
The method according to claim 4, wherein determining the current node to be processed in the target workflow according to the node information of the target workflow by the scheduling unit comprises:

Determine the current node in the target workflow according to the node information of the target workflow;

Determine whether the current node is a node to be processed; in response to the current node being a node to be processed, determine the current node as the current node to be processed in the target workflow; in response to the current node not being a node to be processed, return to execute the operation of determining the current node in the target workflow according to the node information of the target workflow.
The method according to claim 1, wherein the information processing system further comprises a workflow creation module, and the method further comprises:

A workflow is created based on the candidate nodes by the workflow creation module, and the workflow is sent to the workflow management module.
The method according to claim 9, wherein the candidate node includes at least one of a preset node and a custom node.
The method according to claim 9, wherein the candidate node comprises a resource node, The information processing system also includes a resource management service module, and the method further includes:

The resource information of the resource node is stored through the resource management service module.
An electronic device, comprising:

one or more processors;

a memory configured to store one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.
A computer-readable storage medium stores computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 11.