CN115658380A - Distributed flow reverse rollback method and system - Google Patents

Abstract

The invention discloses a distributed process reverse rollback method and a distributed process reverse rollback system, which are applicable to business processes, wherein rollback triggering is carried out according to data definition and a process execution state without sending any instruction, process execution data is obtained by monitoring distributed transceiving data without backing up any data and storing a large number of logs, rollback is realized by an http request, and operations such as log information query and operation frequency recording are not required. The current rollback realization needs to back up data or store a large amount of logs, and relates to that network equipment needs to count equipment operation, and whether rollback exists is determined according to the operation times, so that redundant operation is not needed; the system comprises an environment deployment module, a flow task module, an execution module, a data packaging and sending module and a data monitoring module.

Description

Distributed process reverse rollback method and system

Technical Field

The invention relates to the technical field of networks and computers, in particular to a distributed process reverse rollback method and a distributed process reverse rollback system.

Background

With the rapid development of the internet, many companies complete their projects in a flow-wise manner, and many business flows are driven by performing asynchronous orchestration tasks on microservices, some of which need to run for a long time, for example, netfix developed a microservice orchestrator: conductors, conductors can implement orchestration across services while providing control and visibility of their interactions.

The function of the factor is more perfect as a 'business process engine', but in the process of executing a certain process by the factor, if a certain task is executed abnormally, the execution of the whole process fails. Because some projects are failed to execute and need to be rebuilt, namely, the process needs to be executed again, because the task before the failed task is successfully executed when the process is executed last time, the process is executed again, and the task which is required to be successfully executed returns to the non-executed state. Many current rollback methods are directed to tasks and rollback triggers require active sending of instructions, and some rollback methods require backing up data or storing large amounts of logs, and also requiring querying log information. However, some processes may include multiple nodes, and if each node rolls back and sends a command to trigger, the roll back efficiency is too low. In order to reduce human input and improve rollback efficiency of an abnormal process, the invention provides a distributed process reverse rollback method and a distributed process reverse rollback system.

Disclosure of Invention

Aiming at the defects in the prior art, the invention provides a distributed flow reverse rollback method and a distributed flow reverse rollback system so as to improve the rollback efficiency of an abnormal flow.

A distributed process reverse rollback method comprises the following steps: software environment deployments including, but not limited to, deploying Conductor, mySQL, and Elasticsearch; setting a flow task, designing and generating a flow table and a task table; executing the flow table and the task table, and calling a reducer abnormal flow starting interface to start a flow; data encapsulation, namely encapsulating the data to generate process data in a json format after the process execution is finished; distributed transceiving, namely performing distributed transceiving on the process data; monitoring data, namely monitoring the flow data which is received and transmitted in a distributed mode; acquiring an instance object, and converting the monitored flow data into a workflow; acquiring a flow state, and acquiring the flow state of the workflow; judging whether rollback is needed or not according to the state of the workflow; if the rollback is needed, the completed tasks of the rollback flow are obtained, the task nodes which are successfully executed are obtained from the workflow and stored in a linked list, the linked list is sorted according to the execution time of the task nodes and generated, and the task linked list is traversed to be rolled back; if no rollback is needed, the flow is ended. The invention is suitable for the business process, the rollback trigger is according to the data definition and the process execution state, does not need to send any instruction, obtains the process execution data by monitoring the distributed transceiving data, does not need to back up any data and store a large amount of logs, and the rollback is realized by http request without inquiring log information, recording operation times and other operations. The current rollback implementation needs to backup data or store a large amount of logs, and the network equipment needs to count the equipment operation and determine whether to rollback according to the operation times, so that the redundant operation is not needed.

Netfix Conductor, netfix Conductor is a testing framework for distributed systems. The Netflix Conductor framework is a typical service orchestration framework, workflow and distributed scheduling can be realized through the Conductor, and the performance is excellent. The Netflix Conductor is an open source flow engine written based on JAVA language and is used for constructing a flow based on micro-service. It has the following characteristics: the method is characterized in that a complex business process is allowed to be created, each independent task in the process is realized by one micro-service, a workflow is created based on JSON DSL, the execution of the task is arranged, the workflow is visible and traceable in the execution process, and various control models such as pause, recovery and restart are provided.

Distributed transceiving is accomplished through a distributed publish-subscribe messaging system: the distributed publish-subscribe messaging system may employ an open source streaming platform Kafka developed by the Apache software foundation, written by Scala and Java, which can process all the action stream data of a consumer in a web site.

The Elasticissearch is a distributed, high-expansion and high-real-time search and data analysis engine.

Further, the method for reverse rollback of a distributed process, according to the state of the workflow, determining whether rollback is required includes the specific steps of: and judging according to the parameters of the flow state, if the flow state is failed or abnormal, rolling back, and if the flow state is finished, not rolling back. The flow state, that is, the state (status) of the workflow, is determined according to the state (status) of the workflow, and the state of the workflow generally includes: completion, creation, running, suspension, termination, failure (Completed, created, running, suspended), etc., or may be customized according to the user, where exceptions include conditions other than completion, creation, running. The rollback accuracy can be improved by judging the state, and the misjudgment of subsequent nodes is prevented.

Further, the method for reverse rollback of a distributed process includes the specific steps of obtaining successfully executed task nodes from a workflow and storing the successfully executed task nodes in a linked list: and defining the task class of the workflow, acquiring all tasks of the workflow, and putting the tasks with the states of completion into a linked list. The invention mainly records the nodes in the form of the linked list, the operation of recording the head and the tail is more convenient, the linked list (list) has an important property, the insertion operation and the deletion operation can not cause the failure of the original list iterator, and the dynamic storage allocation is adopted, so that the memory waste and overflow can not be caused.

Further, in the distributed process reverse rollback method, the distributed transceiving is realized and completed by adopting a Kafka system. Kafka is an independent architecture, runs on a cluster consisting of one or more servers, and partitions can be distributed across the cluster nodes, high level API, behavior defined by Kafka itself, mask detail management, ease of use, and is well-suited to the required functionality of the present invention.

Kafka is an open source stream processing platform developed by the Apache software foundation, written by Scala and Java. Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all the action flow data of a consumer in a web site. These data are typically addressed by handling logs and log aggregations due to throughput requirements. The purpose of Kafka is to unify online and offline message processing through the parallel loading mechanism of Hadoop, and also to provide real-time messages through clustering.

Further, in the method for reverse rollback of a distributed process, the specific steps of sorting the linked lists according to the execution time of the task nodes and generating the task linked lists include: acquiring the execution time of all task nodes, and sequencing all task nodes according to the execution time to obtain a sequencing result; and sequencing the linked list according to the sequencing result and generating a task linked list. The rollback result can be clearer through time sequencing, so that the checking is convenient, and careless mistakes are reduced.

Further, the distributed process reverse rollback method includes, but is not limited to, merge sorting, select sorting, insert sorting, hill sorting, and heap sorting.

Further, in the distributed process reverse rollback method, the specific step of traversing the task linked list to rollback includes: inquiring task data, inquiring the task definition table according to a task name, and judging whether to rollback or not according to the task definition table; selecting a rollback mode, if the task type is an http type, acquiring a rollback request method and a URL from the example object task, and completing the rollback of the task by using an http request; and if the task type is the sub workflow type, rolling back the task in an independent flow. Different processing modes are adopted according to different types of tasks, so that the working efficiency can be improved, and the waste of resources is reduced.

Further, the method for reverse rollback of a distributed process further includes: and storing the rollback log, generating a rollback result after the rollback is finished, and storing the rollback result into the rollback log. The integrity of the system is ensured, and error correction is facilitated.

Further, in the distributed process reverse rollback method, the process table includes rollback judgment instructions, which include, but are not limited to, two types. The flow sheet can be customized, and the judgment is carried out according to actual conditions, so that the flexibility is improved.

The utility model provides a reverse rollback system of distributed process, this system includes environment deployment module, process task module, execution module, data encapsulation and sending module, data monitoring module, judgement module, wherein: the environment deployment module is used for deploying a software environment; the flow task module is used for creating, calling and starting a flow; the execution module is used for executing the flow; the data packaging and sending module is used for packaging data and sending the data; the data monitoring module is used for receiving data; the judging module is used for generating a rollback judging instruction, and the rollback judging instruction includes but is not limited to two types.

The invention has the following beneficial effects:

the invention is suitable for the service process, the rollback trigger is according to the data definition and the process execution state, no command is required to be sent, the process execution data is acquired by monitoring kafka data, no data is required to be backed up and a large number of logs are not required to be stored, the rollback is realized by http request, operations such as log information query and operation frequency recording are not required, the whole smoothness and efficiency are high, a large number of manual detections are not required, and the labor input is reduced.

Drawings

In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings used in the detailed description or the prior art description will be briefly described below. Throughout the drawings, like elements or portions are generally identified by like reference numerals. In the drawings, elements or portions are not necessarily drawn to scale.

FIG. 1 is a schematic flow chart of the method of the embodiment;

FIG. 2 is a schematic flow chart of an embodiment;

fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby.

It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.

Example one

As shown in fig. 1, a distributed process reverse rollback method includes: software environment deployments including, but not limited to, deploying Conductor, mySQL, and Elasticsearch; setting a flow task, designing and generating a flow table and a task table, wherein the flow table comprises rollback judgment instructions, and the rollback judgment instructions comprise but are not limited to two types; executing the flow table and the task table, and calling a Conductor abnormal flow starting interface to start a flow; data encapsulation, wherein after the process execution is finished, the data are encapsulated to generate process data in a json format; distributed transceiving, namely performing distributed transceiving on the process data; monitoring data, namely monitoring the flow data which is received and transmitted in a distributed mode; acquiring an instance object, and converting the monitored flow data into a workflow; acquiring a flow state, and acquiring the flow state of the workflow; judging whether rollback is needed or not according to the state of the workflow; if the rollback is needed, the completed tasks of the rollback flow are obtained, including obtaining task nodes which are successfully executed from the workflow and storing the task nodes in a linked list, sequencing the linked list according to the execution time of the task nodes to generate a task linked list, and traversing the task linked list to perform the rollback; if no rollback is needed, the flow is ended. The invention is suitable for the business process, the rollback trigger is according to the data definition and the process execution state, does not need to send any instruction, obtains the process execution data by monitoring the kafka data, does not need to back up any data and store a large amount of logs, and the rollback is realized by http request without inquiring log information, recording operation times and other operations. The present rollback implementation requires data backup or a large amount of log storage, and involves that network devices need to count device operations and determine whether to rollback or not according to the number of operations. The application scenarios of the application include, but are not limited to, network resource arrangement, network service activation, migration, cloud network resource coordination, basic network service deployment and the like in a virtual network; the method comprises the steps of reconstructing a network by using hardware and software of a general industrial standard, and reducing the use of hardware equipment; and the functions of network assembling and disassembling and the like are realized by software, so that the manual construction is reduced.

As shown in fig. 2, the basic principle includes calling the create flow interface create flow of the Conductor, and then calling the start interface of the Conductor flow, and starting the flow. And sending kafka data after the flow execution is successful or failed. And monitoring kafka data and consuming the kafka data, and converting the consumed kafka data into a workflow object. And judging whether the process rolls back or not according to the workflow definition and the workflow execution result, if the process rolls back, acquiring task nodes which are successfully executed from the workflow, storing the task nodes into a linked list, and sequencing the linked list according to the execution time of the nodes. And traversing the task node linked list to call a task rollback interface to realize flow rollback.

In this embodiment, the specific step of determining whether rollback is required according to the state of the workflow includes: and judging according to the parameters of the flow state, if the flow state is failure and abnormal, rolling back, and if the flow state is finished, not rolling back.

In this embodiment, the specific step of obtaining, from a workflow, a task node that is successfully executed and storing the task node in a linked list includes: and defining the task class of the workflow, acquiring all tasks of the workflow, and putting the tasks with the states of completion into a linked list.

In this embodiment, in the distributed process reverse rollback method, the distributed transceiving is implemented and completed by using a Kafka system.

In this embodiment, the method for reverse rollback of a distributed process includes the specific steps of sorting the linked lists according to the execution time of the task nodes and generating the task linked lists: acquiring the execution time of all task nodes, and sequencing all task nodes according to the execution time to obtain a sequencing result; and sequencing the linked list according to the sequencing result and generating a task linked list.

In this embodiment, the method for distributed process reverse rollback includes, but is not limited to, merge sort, select sort, insert sort, hill sort, and heap sort. The most effective sorting method is selected according to actual conditions.

In this embodiment, in the distributed process reverse rollback method, the specific step of traversing the task linked list to rollback includes: inquiring task data, inquiring the task definition table according to a task name, and judging whether to rollback or not according to the task definition table; selecting a rollback mode, if the task type is an http type, acquiring a rollback request method and a URL (uniform resource locator) from the example object task, and completing the rollback of the task by using an http request; and if the task type is the sub workflow type, rolling back the task in an independent flow.

In this embodiment, the method for reverse rollback of a distributed process further includes: and storing the rollback log, generating a rollback result after the rollback is finished, and storing the rollback result into the rollback log.

The utility model provides a reverse rollback system of distributed process, this system of includes environment deployment module, process task module, execution module, data encapsulation and sending module, data monitoring module, judgement module, wherein: the environment deployment module is used for deploying a software environment; the flow task module is used for creating, calling and starting a flow; the execution module is used for executing the flow; the data packaging and sending module is used for packaging data and sending the data; the data monitoring module is used for receiving data; the judging module is used for generating a rollback judging instruction, and the rollback judging instruction includes but is not limited to two types.

The Elasticissearch is a distributed, high-expansion and high-real-time search and data analysis engine. It can conveniently make a large amount of data have the capability of searching, analyzing and exploring.

Kafka is a high throughput distributed publish-subscribe messaging system with the following characteristics: the persistence of messages is provided by the disk data structure of O (1), which can maintain stable performance for a long time even for message storage in TB; even the very common hardware Kafka can support millions of messages per second; supporting partitioning of messages by Kafka server and consumer clusters; and Hadoop parallel data loading is supported.

As shown in fig. 3, the electronic device 130 includes one or more processors 1301 and memory 1302.

The processor 1301 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 130 to perform desired functions.

Memory 1302 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by processor 1301 to implement a distributed flow reverse rollback method of the various embodiments of the present application described above or other desired functionality. Various content such as a distributed flow reverse rollback error parameter may also be stored in the computer readable storage medium.

In one example, the electronic device 130 may further include: an input device 1303 and an output device 1304, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

The input device 1303 may include, for example, a keyboard, a mouse, a joystick, a touch screen, and the like.

The output device 1304 may output various information including the determined exercise data and the like to the outside. The output 1304 may include, for example, a display, a communication network, a remote output device connected thereto, and so forth.

Of course, for simplicity, only some of the components of the electronic device 130 relevant to the present application are shown in fig. 3, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 130 may include any other suitable components depending on the particular application.

In addition to the above methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in a distributed flow reverse rollback method according to various embodiments of the present application described in the present specification.

The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present application may also be a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the steps of a distributed flow reverse rollback method of the present specification according to various embodiments of the present application.

The computer readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is provided for purposes of illustration and understanding only, and is not intended to limit the application to the details which are set forth in order to provide a thorough understanding of the present application.

The block diagrams of devices, apparatuses, devices, systems referred to in this application are only used as illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by one skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably herein. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

It should also be noted that in the devices, apparatuses, and methods of the present application, each component or step can be decomposed and/or re-combined. These decompositions and/or recombinations are to be considered as equivalents of the present application.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The above description is only a preferred embodiment of the present application and should not be taken as limiting the present application, and any modifications, equivalents and the like that are within the spirit and scope of the present application should be included.

Claims (10)

1. A distributed process reverse rollback method is characterized by comprising the following steps:

software environment deployments, including but not limited to deployment Conductor, mySQL, and Elasticsearch;

setting a flow task, designing and generating a flow table and a task table;

executing the flow table and the task table, and calling a Conductor abnormal flow starting interface to start a flow;

data encapsulation, namely encapsulating the data to generate process data in a json format after the process execution is finished;

distributed transceiving, namely performing distributed transceiving on the process data;

monitoring data, namely monitoring the flow data which is received and transmitted in a distributed mode;

acquiring an instance object, and converting the monitored flow data into a workflow;

acquiring a flow state, and acquiring the flow state of the workflow;

judging whether rollback is needed or not according to the state of the workflow;

if the rollback is needed, the completed tasks of the rollback flow are obtained, the task nodes which are successfully executed are obtained from the workflow and stored in a linked list, the linked list is sorted according to the execution time of the task nodes and generated, and the task linked list is traversed to be rolled back;

if no rollback is needed, the flow is ended.

2. The method according to claim 1, wherein the specific step of determining whether rollback is required according to the state of the workflow comprises:

and judging according to the parameters of the flow state, if the flow state is failed or abnormal, rolling back, and if the flow state is finished, not rolling back.

3. The method according to claim 1, wherein the specific step of obtaining task nodes successfully executed from the workflow and storing the task nodes in a linked list comprises:

and defining the task class of the workflow, acquiring all tasks of the workflow, and putting the tasks with the states of completion into a linked list.

4. The method of claim 1, wherein the distributed transceiving is implemented and completed using Kafka system.

5. The distributed process reverse rollback method according to claim 1, wherein the specific step of sorting the linked lists according to the execution time of the task nodes and generating the task linked lists comprises:

acquiring the execution time of all task nodes, and sequencing all task nodes according to the execution time to obtain a sequencing result;

and sequencing the linked list according to the sequencing result and generating a task linked list.

6. The method of claim 5, wherein the sorting method includes but is not limited to merge sorting, select sorting, insert sorting, hill sorting, heap sorting.

7. The method of claim 1, wherein traversing the task chain table for rollback comprises:

inquiring task data, inquiring the task definition table according to a task name, and judging whether to rollback or not according to the task definition table;

selecting a rollback mode, if the task type is an http type, acquiring a rollback request method and a URL (uniform resource locator) from the example object task, and completing the rollback of the task by using an http request; and if the task type is the sub workflow type, rolling back the task in an independent flow.

8. The method of claim 7, further comprising:

and storing the rollback log, generating a rollback result after the rollback is finished, and storing the rollback result into the rollback log.

9. The distributed process reverse rollback method according to claim 1, wherein said flow table includes rollback determination instructions, said rollback determination instructions including, but not limited to, two.

10. A distributed process reverse rollback system, which is used in the distributed process reverse rollback method according to any one of claims 1-9, and which includes an environment deployment module, a process task module, an execution module, a data encapsulation and transmission module, a data monitoring module, and a determination module, wherein:

the environment deployment module is used for deploying a software environment;

the flow task module is used for creating, calling and starting a flow;

the execution module is used for executing the flow;

the data packaging and sending module is used for packaging data and sending the data;

the data monitoring module is used for receiving data;

the judging module is used for generating a rollback judging instruction, and the rollback judging instruction includes but is not limited to two types.

Images