CN113297081B - Execution method and device of continuous integrated pipeline - Google Patents

Execution method and device of continuous integrated pipeline Download PDF

Info

Publication number
CN113297081B
CN113297081B CN202110577765.1A CN202110577765A CN113297081B CN 113297081 B CN113297081 B CN 113297081B CN 202110577765 A CN202110577765 A CN 202110577765A CN 113297081 B CN113297081 B CN 113297081B
Authority
CN
China
Prior art keywords
node
execution
pipeline
nodes
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110577765.1A
Other languages
Chinese (zh)
Other versions
CN113297081A (en
Inventor
樊宇
张广权
李玉娟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Zhenshi Information Technology Co Ltd
Original Assignee
Beijing Jingdong Zhenshi Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Zhenshi Information Technology Co Ltd filed Critical Beijing Jingdong Zhenshi Information Technology Co Ltd
Priority to CN202110577765.1A priority Critical patent/CN113297081B/en
Publication of CN113297081A publication Critical patent/CN113297081A/en
Application granted granted Critical
Publication of CN113297081B publication Critical patent/CN113297081B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management
    • G06F11/3688Test management for test execution, e.g. scheduling of test suites
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3696Methods or tools to render software testable

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Stored Programmes (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The application discloses an execution method and device of a continuous integrated assembly line, and relates to the technical field of computers, wherein a specific implementation mode comprises the steps of executing the assembly line, receiving node execution failure information of the assembly line, acquiring a corresponding node identifier, and further determining a failure node; determining unexecuted nodes, acquiring corresponding configuration information, modifying the configuration information of failed nodes, and updating a test entry identification list; based on the updated test entry identification list, executing unexecuted nodes and updated failed nodes after configuration information modification until all nodes in the pipeline are successfully executed. Therefore, only failed and unexecuted nodes are executed when retrying after the continuous integration of pipeline nodes fails, the overall execution time of the pipeline is reduced, the execution efficiency of the pipeline is improved, and the customer experience is improved.

Description

Execution method and device of continuous integrated pipeline
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and apparatus for executing a continuous integrated pipeline.
Background
Currently, continuous integration (Continuous Integration, CI) is a software development practice, i.e. team development members often integrate their work, usually at least once a day per member, meaning that multiple integration may occur per day. Each integration is verified by automated build-up, so that integration errors are discovered as early as possible. The whole continuous integration comprises three steps: compiling, publishing and automatically testing. After the conventional continuous integrated pipeline fails to execute, the conventional continuous integrated pipeline can only execute from the beginning, and has low efficiency and poor user experience.
In the process of implementing the present application, the inventor finds that at least the following problems exist in the prior art:
after the conventional continuous integrated pipeline fails to execute, the conventional continuous integrated pipeline can only execute from the beginning, and has low efficiency and poor user experience.
Disclosure of Invention
In view of the above, the embodiment of the application provides a method and a device for executing a continuous integrated pipeline, which can solve the problems that the existing continuous integrated pipeline can only be executed from the beginning after the execution fails, and has low efficiency and poor user experience.
To achieve the above object, according to an aspect of an embodiment of the present application, there is provided an execution method of a persistent integrated pipeline, including:
An execution pipeline receives the node execution failure information of the pipeline, acquires a corresponding node identifier, and further determines a failure node;
determining unexecuted nodes, acquiring corresponding configuration information, modifying the configuration information of failed nodes, and updating a test entry identification list;
based on the updated test entry identification list, executing unexecuted nodes and updated failed nodes after configuration information modification until all nodes in the pipeline are successfully executed.
Optionally, before the execution pipeline, the method further comprises:
acquiring source codes of the nodes which are successfully executed, and analyzing the source codes to obtain class information;
and generating a directed graph by taking the class information as a vertex.
Optionally, determining the failure node includes:
Determining a common node based on the directed graph;
in response to determining that execution fails or is not performed on the common nodes, all tree nodes that share the common node are determined to be failed nodes.
Optionally, determining the failure node includes:
determining class information corresponding to each node in the pipeline;
In response to determining that multiple nodes in the pipeline call the same class information together, when one node fails to execute, determining the multiple nodes as failed nodes.
Optionally, determining the unexecuted node includes:
acquiring execution log information of each node in an execution pipeline;
And analyzing the execution log information, and further determining non-execution nodes based on the directed graph.
Optionally, parsing the execution log information, and further determining, based on the directed graph, non-executing nodes includes:
analyzing the execution log information to determine class information and row information in the corresponding codes;
Determining the executed directed graph nodes based on the directed graph, the class information and the row information;
based on the executed directed graph nodes and all nodes of the directed graph, non-executed nodes are determined.
Optionally, before executing the unexecuted node and the updated failed node after modifying the configuration information based on the updated test entry identification list, the method further comprises:
And carrying out retrying approval on each pipeline node corresponding to each entry identifier in the updated test entry identifier list, and executing corresponding reassembled execution node information based on the test entry identifier list in response to determining that the retrying approval passes.
Optionally, retrying and approving each pipeline node corresponding to each entry identifier in the updated test entry identifier list, including:
Determining workflow corresponding to each test entry identifier in the test entry identifier list;
and sending the test entry identification list, the execution configuration information and the corresponding workflow to a preset approval node for retrying approval.
Optionally, before updating the test entry identification list, the method further comprises:
configuration information of the unexecuted node is modified.
Optionally, after all the nodes in the pipeline successfully execute, the method further comprises:
and aggregating the node information in the pipeline after successful execution to generate a new pipeline execution node.
In addition, the application also provides an execution device of the continuous integrated pipeline, which comprises:
The failure node determining unit is configured to execute the pipeline, receive the execution failure information of the pipeline node, acquire the corresponding node identification and further determine the failure node;
The updating unit is configured to determine non-executed nodes, acquire corresponding configuration information, modify the configuration information of failed nodes and update the test entry identification list;
And the execution unit is configured to execute the unexecuted node and the updated failed node after configuration information modification based on the updated test entry identification list until all nodes in the pipeline are successfully executed.
Optionally, the execution apparatus of the persistent integrated pipeline further includes a directed graph generating unit configured to:
acquiring source codes of the nodes which are successfully executed, and analyzing the source codes to obtain class information;
and generating a directed graph by taking the class information as a vertex.
Optionally, the failure node determining unit is further configured to:
Determining a common node based on the directed graph;
in response to determining that execution fails or is not performed on the common nodes, all tree nodes that share the common node are determined to be failed nodes.
Optionally, the failure node determining unit is further configured to:
determining class information corresponding to each node in the pipeline;
In response to determining that multiple nodes in the pipeline call the same class information together, when one node fails to execute, determining the multiple nodes as failed nodes.
Optionally, the updating unit is further configured to:
acquiring execution log information of each node in an execution pipeline;
And analyzing the execution log information, and further determining non-execution nodes based on the directed graph.
Optionally, the updating unit is further configured to:
analyzing the execution log information to determine class information and row information in the corresponding codes;
Determining the executed directed graph nodes based on the directed graph, the class information and the row information;
based on the executed directed graph nodes and all nodes of the directed graph, non-executed nodes are determined.
Optionally, the execution apparatus of the persistent integrated pipeline further comprises an approval unit configured to:
And carrying out retrying approval on each pipeline node corresponding to each entry identifier in the updated test entry identifier list, and executing corresponding reassembled execution node information based on the test entry identifier list in response to determining that the retrying approval passes.
Optionally, the approval unit is further configured to:
Determining workflow corresponding to each test entry identifier in the test entry identifier list;
and sending the test entry identification list, the execution configuration information and the corresponding workflow to a preset approval node for retrying approval.
Optionally, the updating unit is further configured to:
configuration information of the unexecuted node is modified.
Optionally, the execution apparatus of the persistent integrated pipeline further comprises an aggregation unit configured to:
and aggregating the node information in the pipeline after successful execution to generate a new pipeline execution node.
In addition, the application also provides an execution electronic device of the continuous integrated pipeline, which comprises: one or more processors; and a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method of execution of the persistent integrated pipeline as described above.
In addition, the application also provides a computer readable medium, on which a computer program is stored, which when executed by a processor implements the execution method of the continuous integrated pipeline as described above.
One embodiment of the above application has the following advantages or benefits: according to the application, through an execution pipeline, the node execution failure information of the pipeline is received, the corresponding node identification is obtained, and then the failure node is determined; determining unexecuted nodes, acquiring corresponding configuration information, modifying the configuration information of failed nodes, and updating a test entry identification list; based on the updated test entry identification list, executing unexecuted nodes and updated failed nodes after configuration information modification until all nodes in the pipeline are successfully executed. Therefore, the application only executes the failed node and the unexecuted node when retrying after continuously integrating the pipeline node failure, reduces the whole execution time of the pipeline, improves the execution efficiency of the pipeline and improves the customer experience.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the application and are not to be construed as unduly limiting the application. Wherein:
FIG. 1 is a schematic diagram of the main flow of a method of execution of a continuous integrated pipeline according to a first embodiment of the present application;
FIG. 2 is a schematic diagram of the main flow of a method of execution of a continuous integrated pipeline according to a second embodiment of the present application;
FIG. 3 is an application scenario diagram of a method of execution of a continuous integrated pipeline according to a third embodiment of the present application;
FIG. 4 is a schematic diagram of the major modules of an execution device of a continuous integrated pipeline according to an embodiment of the present application;
FIG. 5 is an exemplary system architecture diagram in which embodiments of the present application may be applied;
fig. 6 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the application.
Detailed Description
Exemplary embodiments of the present application will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present application are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of main flow of an execution method of a persistent integrated pipeline according to a first embodiment of the present application, and as shown in fig. 1, the execution method of the persistent integrated pipeline includes:
Step S101, executing the assembly line, receiving the assembly line node execution failure information, obtaining the corresponding node identification, and further determining the failure node.
In this embodiment, an execution body (for example, may be a server) of the execution method of the persistent integrated pipeline may monitor that a user triggers the execution pipeline by means of wired connection or wireless connection, and then the execution body may execute the pipeline. And when the execution body completes execution of each node in the pipeline and monitors that the node with the execution failure exists in the pipeline (for example, the execution body inquires the node with the Fail identifier in the execution log of each node in the pipeline, namely, the failed node), receiving the execution failure information of the node in the pipeline. The pipeline node execution failure information includes a node identifier, which may be an identifier of a corresponding pipeline node, for example, a Fail (i.e., failure) or Pass (i.e., success). The execution body may locate the corresponding failed node by locating the Fail node identification.
In some optional implementations of the present embodiment, determining the failed node further includes: class information corresponding to each node in the pipeline is determined. Specifically, the execution body may determine method class information that needs to be invoked by each node in the pipeline. In response to determining that multiple nodes in the pipeline call the same class information together, when one node fails to execute, determining the multiple nodes as failed nodes.
In this implementation, the class information may include a method class. When there is a situation that multiple nodes in the pipeline need to call the same method class together, if only one node in the multiple nodes fails to execute when the same method class is called, the multiple nodes are all determined to be failed nodes. That is, even if one or more nodes of the plurality of nodes succeed in executing when the same method class is called, the plurality of nodes are determined to fail when the existing node fails in executing when the same method class is called. Therefore, the failed node is retried as much as possible, missing of retried failed nodes is avoided, and the overall execution efficiency and success rate of the continuous integrated assembly line are improved.
Step S102, determining non-executed nodes, acquiring corresponding configuration information, modifying the configuration information of failed nodes, and updating a test entry identification list.
In this embodiment, determining the unexecuted node includes:
and acquiring the execution log information of each node in the execution pipeline.
Specifically, after all nodes in the pipeline of one pass are executed, the execution body may acquire log information of all executed nodes. The nodes with successful execution and failed execution can have the execution log information, and the non-executed nodes can not have the execution log information. The execution log may contain method classes for executing calls.
And analyzing the execution log information, and further determining non-execution nodes based on the directed graph.
Specifically, each vertex of the directed graph is built based on class information. The execution main body can analyze the execution log information to obtain class information corresponding to the nodes which are successfully executed in the current pipeline, and then the execution main body can determine non-executed nodes based on the directed graph containing class information corresponding to all the historical successful nodes and based on the obtained class information corresponding to the nodes which are successfully executed. Namely, the class information existing in the directed graph, but the node corresponding to the class information which does not exist (or does not exist) in the class information corresponding to the node which is successfully executed in the pipeline at this time is determined as the unexecuted node. In the directed graph, nodes are in one-to-one correspondence with class information. By means of the class information in the directed graph, the corresponding node can be determined. Therefore, finally, the unexecuted node of the current pipeline can be accurately determined.
In some optional implementations of the present embodiment, analyzing the execution log information, and further determining, based on the directed graph, non-executed nodes, further includes:
the execution log information is parsed to determine class information and row information in the corresponding code.
In particular, the class information may include a class of methods that the node in the execution pipeline needs to invoke. The line information may include the code lines that the node in the execution pipeline needs to call.
Based on the directed graph and the class information and the row information, the executed directed graph nodes are determined.
Specifically, the directed graph in the implementation mode is built through class information corresponding to the historical execution success node. The execution main body can compare the directed graph established according to the class information corresponding to the historical execution success node (the directed graph can also contain code line information corresponding to the historical execution success node) with the class information and the line information corresponding to the current pipeline execution success node determined according to the execution log information so as to determine the directed graph node executed by the current pipeline.
Based on the executed directed graph nodes and all nodes of the directed graph, non-executed nodes are determined.
Specifically, after determining the nodes of the directed graph executed by the current pipeline, the execution body compares the nodes with all the nodes in the directed graph, so that the nodes which are not executed in the directed graph, namely the nodes which are not executed and correspond to the current pipeline, can be determined.
After determining the unexecuted node, the execution body may query execution configuration information corresponding to the unexecuted node. And modifying configuration information of the corresponding failed node based on the configuration information of the history execution successful node. In this embodiment, relevant nodes that need to be executed by the pipeline task need to be configured before the pipeline is executed, for example, nodes such as code pulling, compiling, unit testing, code scanning, deployment, and automation testing, and metadata information related to applications, for example, information such as a code base address, branch information, grouping information, a node command, and a parent node of the automation test task. The execution pipeline task is then triggered manually or by a setting automatically. In this embodiment, for example, if the failed node of the current persistent integrated pipeline is a unit test node, the execution body may call the configuration information of the unit test node with successful historical execution, and update the configuration information of the unit test node with successful historical execution to the failed node. Then, the execution body may split the acquired configuration information of the unexecuted node and the configuration information of the modified failed node into a set of multiple execution portals, for example, for a unit test node, split the unit test node into each test class, where the test class is a root node of each tree in the forest, and use all the executed test classes as execution portals, and for an automation test node, the execution body may query sub-plans in the automation test plan until test tasks of leaf nodes of all the sub-plans are acquired, and use test tasks of the leaf nodes as execution portals. The execution portal is different from the portal node, and refers to a portal triggering the start of the execution of the automated test. There may be multiple test tasks in an automated test plan, each of which is an entry for an automated test trigger application to execute, and for some data driven classes of automated test tasks, each test scenario is still an entry for the trigger application to execute.
The root node and the child node of each tree in the forest can be obtained through the following steps: the execution body can split the directed graph execution node into a plurality of forest data structures according to the entry node of the execution code (the entry node is on the directed graph and can be the node corresponding to the member method of the class information corresponding to the historical execution success node), the root node of each tree in the forest is the entry node of the execution code, and the child node of each tree is the corresponding code node called in the execution process.
In this embodiment, for an entry node executing code, an exemplary method a, such as a control layer of an application, is an entry node that is triggered to execute by an external http request, i.e., a web request of an automated test case. Method B calls method C of the service layer to handle some business logic, and method C calls other methods D1, D2 of the server to perform different query, modification and deletion logic. The method a and the method B are entry nodes for executing codes, and the method C, the method D1 and the method D2 are child nodes. The execution body may convert a set of a plurality of execution portals obtained by splitting based on the obtained configuration information of the unexecuted node and the modified configuration information of the failed node into a test portal identification list of unexecuted and execution failure, and update the test portal identification list. The updated test entry identification list is used to characterize the nodes of the pipeline that need to be retried.
Step S103, based on the updated test entry identification list, executing unexecuted nodes and updated failed nodes after configuration information modification until all nodes in the pipeline are successfully executed.
In this embodiment, before retrying the execution node, the execution body may acquire the compiled binary folder from the cloud storage, and reassemble the execution node information according to the sub-node information, where the sub-node information includes only sub-nodes that have not been executed and failed to be executed.
The execution body may execute the corresponding unexecuted node and the updated failed node after modifying the configuration information based on the updated test entry identification list, and if the execution body still detects that there is a node that fails in execution, the execution body may retry the failed node with the unexecuted node again until all the sub-nodes (i.e., the unexecuted and executed failed node) execute.
According to the embodiment, through an execution pipeline, the node execution failure information of the pipeline is received, the corresponding node identification is obtained, and then the failure node is determined; determining unexecuted nodes, acquiring corresponding configuration information, modifying the configuration information of failed nodes, and updating a test entry identification list; based on the updated test entry identification list, executing unexecuted nodes and updated failed nodes after configuration information modification until all nodes in the pipeline are successfully executed. Therefore, the application only executes the failed node and the unexecuted node when retrying after continuously integrating the pipeline node failure, reduces the whole execution time of the pipeline, improves the execution efficiency of the pipeline and improves the customer experience.
Fig. 2 is a main flow diagram of an execution method of a persistent integrated pipeline according to a second embodiment of the present application, and as shown in fig. 2, the execution method of the persistent integrated pipeline includes:
step S201, acquiring source codes of the successful execution nodes, and analyzing the source codes to obtain class information.
Before the execution body generates the directed graph, the corresponding source code can be pulled according to the code base version and the code base address of all the successful execution nodes in the historical continuous integrated pipeline when the execution is successful. Source code (also called a source program) refers to an uncompiled text file written according to a certain programming language specification, and is a series of human-readable computer language instructions. In modern programming languages, the source code may be in the form of a book or tape, but the most common format is a text file, the purpose of which is to compile a computer program. The ultimate goal of computer source code is to translate human-readable text into binary instructions that a computer can execute, a process called compilation, performed by a compiler. The source code is relative to the object code and the executable code. The source code is code written in assembly language and high-level language. Target code refers to binary code generated by a source code through a compiler and directly recognized by a cpu. Executable code is an executable file formed by concatenating object code, and is of course binary. After pulling the corresponding source code, the executing body can analyze the source code to obtain the called method class information.
Step S202, generating a directed graph by taking class information as a vertex.
After the execution main body pulls the source codes, all the source codes can be analyzed, all the class information is obtained according to the source codes, the class information is established in the directed graph, the member methods and the member variables in the class are further analyzed, each member establishes a vertex, the sides of the class vertices are established, the member methods of the class are further analyzed, the codes are analyzed line by line, if the methods called in the code lines already exist in the directed graph, the sides of the member methods and the sides of the called method vertices are established, the sides have attribute information, the attribute is a code line number and an actual parameter list in the code lines, for some code methods, the situation that different code lines in one method call one method exists, therefore, the attribute is added for distinguishing until the member methods and the member variables in all the classes are analyzed, and the directed graph is generated. Parsing of source code is a relatively static analysis process.
Step S203, the assembly line is executed, the assembly line node execution failure information is received, the corresponding node identification is obtained, and then the failure node is determined.
The principle of step S203 is similar to that of step S101, and will not be described here.
Specifically, step S203 may also be implemented by steps S2031 to S2032:
Step S2031, determining a common node based on the directed graph.
The execution body can split the directed graph execution node into a plurality of forest data structures according to the entry node of the execution code (the entry node is on the directed graph and can be the node corresponding to the member method of the class information corresponding to the historical execution success node), the root node of each tree in the forest is the entry node of the execution code, and the child node of each tree is the corresponding code node called in the execution process. The execution body can determine the nodes shared in the data structures of the forests according to the data structures of the forests obtained by splitting the directed graph. For example, in a data structure of multiple forests, the first tree involves five nodes a1-b1-c-d-e, the second tree involves five nodes a2-b2-c-d-e, where c, d, e are shown in the directed graph as three coincident common nodes, if d fails to execute in the first tree because d is a common node for both trees, the second tree will not be considered successful, and the root node of the tree will be considered to be successful only if c, d, e in both trees execute successfully.
In step S2032, in response to determining that the execution of the common node fails or is not performed, all the tree nodes sharing the common node are determined as failed nodes.
In this embodiment, if the common node fails or is not executed in one tree, all the trees sharing the node will be identified as failed, and only if all the nodes in one tree are successfully executed and all the other nodes sharing the node with the tree node are successfully executed, the root node of the tree is considered to be successfully executed, so that the failed node can be retried as far as possible, and the situation that omission can be applied to case calling among cross applications is avoided.
Step S204, determining non-executed nodes, obtaining corresponding configuration information, modifying the configuration information of failed nodes, and updating the test entry identification list.
Before updating the test entry identification list, the method further comprises:
configuration information of the unexecuted node is modified. The execution main body can correspondingly modify and update the configuration information of the unexecuted node according to the configuration information of the node successfully executed by the historical pipeline, so that the retry result of the execution node is more accurate, the pipeline execution efficiency is improved, and the customer experience is improved.
The principle of step S204 is similar to that of step S102, and will not be described here again.
Step S205, retrying and approving each pipeline node corresponding to each entry identifier in the updated test entry identifier list, and executing corresponding reassembled execution node information based on the test entry identifier list in response to determining that the retrying and approving pass.
Specifically, retrying and approving each pipeline node corresponding to each entry identifier in the updated test entry identifier list, including:
Determining workflow corresponding to each test entry identifier in the test entry identifier list;
and sending the test entry identification list, the execution configuration information and the corresponding workflow to a preset approval node for retrying approval.
Specifically, the preset approval node may be all code submitters (corresponding code submitter information can be obtained according to the code library version) between the code library version executed successfully in history and the code library version. The code submitting personnel, the assembly line executing personnel and the application responsible personnel corresponding to the preset approval nodes can determine whether the workflow is retried to be executed according to the pushed execution result, can be approved by multiple persons at the same time, and finish the workflow when one person refuses to retry. The workflow corresponding to each test entry identifier in the test entry identifier list may be a workflow corresponding to the pipeline is determined, and the workflow may include a code submitter node, a pipeline executor node, and an application responsible person node.
After determining the corresponding workflow, the execution body can send the test entry identification list, the execution configuration information and the corresponding workflow to preset approval nodes (such as code submitting personnel nodes, pipeline executing personnel nodes and application responsible personnel nodes) of the workflow to perform dynamic retry approval so as to discard some pipeline nodes without retry, thereby achieving the effect of dynamically releasing system resources, improving the execution efficiency of pipeline retry and improving customer experience. It can be understood that each preset approval node on the workflow can perform approval at the same time, or enter the approval process of the next approval node after the approval of one approval node passes according to the preset approval sequence. The meaning of retry is that the code does not need to be submitted again, the configuration of failed nodes and unexecuted nodes, the environmental problem information such as data and the like can be modified to be re-executed, in addition to determining whether to re-execute, in the workflow, the workflow state can be modified to be a solving or assigning a solving person, when the workflow state is in the solving or assigned state, other approval nodes cannot be re-executed, after the problem is solved (such as data problem, environmental problem, automatic test case problem and the like), the solved state can initiate re-execution, when one approval node in the workflow initiates re-execution, other approval nodes cannot operate the workflow within the time of a preset threshold, and after the workflow state exceeds the preset threshold time, the integrated pipeline continues to start executing pipeline re-execution. When all code submitterers do not reject the retry, the persistent integration pipeline begins performing the retry operation, wherein sub-nodes in the node may identify information for entries of multiple tests.
Step S206, based on the updated test entry identification list, executing unexecuted nodes and updated failed nodes after configuration information modification until all nodes in the pipeline are successfully executed.
The principle of step S206 is similar to that of step S103, and will not be described here again.
After all the nodes in the pipeline successfully execute, the method further comprises:
and aggregating the node information in the pipeline after successful execution to generate a new pipeline execution node.
Specifically, when all the sub-nodes are successfully executed, that is, no execution failure and no execution node exist, the continuous integrated pipeline can aggregate the sub-node data after each retry of the successful execution, generate a final complete pipeline execution node, push the final complete pipeline execution node to a code submitting personnel node, a pipeline execution personnel node and an application responsible personnel node in the workflow, and normally finish the workflow executed by the version.
The initial trigger condition of the application is that a user manually executes the pipeline, and when all nodes of the pipeline are successfully executed, analysis of source codes and execution logs is started. When the subsequent pipeline execution failure occurs, triggering the approval process of the workflow re-execution. After the approval is passed, the execution failure and the nodes which are not operated are greatly improved in the re-execution efficiency.
The execution method of the continuous integrated pipeline can shorten the re-execution time after the continuous integrated pipeline node fails, improve the operation efficiency, reduce the repeated execution and improve the overall operation capability of the pipeline. The execution failure triggers the workflow approval, which can greatly reduce unnecessary pipeline execution. Besides being suitable for pipeline reduction retry, the method can be used for measuring the coverage rate of automatic test and unit test, can also be combined with approval information of workflow to trace back the root cause of the execution failure of a daily pipeline, can reduce the retry times of the follow-up pipeline, and further improves the overall execution efficiency and success rate of a team.
Fig. 3 is an application scenario diagram of an execution method of a persistent integrated pipeline according to a third embodiment of the present application. The execution method of the continuous integrated pipeline is applied to the scene in the field of software quality measurement. As shown in fig. 3, a server 303 executes a pipeline, receives pipeline node execution failure information 301, acquires a corresponding node identifier 302, and further determines a failed node 304. The server 303 determines that the node 305 is not executed, obtains the corresponding configuration information 306, modifies the configuration information of the failed node 304, obtains the modified configuration information 307 of the failed node, and further updates the test entry identifier list to obtain an updated test entry identifier list 308. The server 303 executes the unexecuted node and the updated failed node 309 after modifying the configuration information based on the updated test entry identification list 308 until all the nodes in the pipeline have executed successfully. In summary, the execution method of the continuous integrated pipeline of the present application can be implemented by the following steps: configuring pipeline node information, executing pipelines, analyzing source codes and executing information, analyzing the successful execution of partial nodes, analyzing the failed node information, re-executing failed nodes or sub-nodes after failure, aggregating the data of the nodes and sub-nodes after successful retry and executing successful nodes.
FIG. 4 is a schematic diagram of the main modules of an execution device of a continuous integrated pipeline according to an embodiment of the present application. As shown in fig. 4, the execution apparatus of the persistent integrated pipeline includes a failure node determination unit 401, an update unit 402, and an execution unit 403.
The failure node determining unit 401 is configured to execute a pipeline, receive the execution failure information of the pipeline node, obtain the corresponding node identifier, and further determine the failure node.
And an updating unit 402 configured to determine that the node is not executed, acquire corresponding configuration information, modify configuration information of the failed node, and further update the test entry identification list.
An execution unit 403 configured to execute the unexecuted node and the updated failed node after modifying the configuration information based on the updated test entry identification list until all the nodes in the pipeline are successfully executed.
In some embodiments, the execution apparatus of the persistent integrated pipeline further comprises a directed graph generation unit configured to: acquiring source codes of the nodes which are successfully executed, and analyzing the source codes to obtain class information; and generating a directed graph by taking the class information as a vertex.
In some embodiments, the failed node determination unit 401 is further configured to: determining a common node based on the directed graph; in response to determining that execution fails or is not performed on the common nodes, all tree nodes that share the common node are determined to be failed nodes.
In some embodiments, the failed node determination unit 401 is further configured to: determining class information corresponding to each node in the pipeline; in response to determining that multiple nodes in the pipeline call the same class information together, when one node fails to execute, determining the multiple nodes as failed nodes.
In some embodiments, the updating unit 402 is further configured to: acquiring execution log information of each node in an execution pipeline; and analyzing the execution log information, and further determining non-execution nodes based on the directed graph.
In some embodiments, the updating unit 402 is further configured to: analyzing the execution log information to determine class information and row information in the corresponding codes; determining the executed directed graph nodes based on the directed graph, the class information and the row information; based on the executed directed graph nodes and all nodes of the directed graph, non-executed nodes are determined.
In some embodiments, the execution apparatus of the persistent integrated pipeline further comprises an approval unit, not shown in fig. 4, configured to: and carrying out retrying approval on each pipeline node corresponding to each entry identifier in the updated test entry identifier list, and executing corresponding reassembled execution node information based on the test entry identifier list in response to determining that the retrying approval passes.
In some embodiments, the approval unit is further configured to: determining workflow corresponding to each test entry identifier in the test entry identifier list; and sending the test entry identification list, the execution configuration information and the corresponding workflow to a preset approval node for retrying approval.
In some embodiments, the updating unit 402 is further configured to: configuration information of the unexecuted node is modified.
In some embodiments, the execution apparatus of the persistent integrated pipeline further comprises an aggregation unit, not shown in fig. 4, configured to: and aggregating the node information in the pipeline after successful execution to generate a new pipeline execution node.
It should be noted that, in the execution method of the continuous integrated pipeline and the execution device of the continuous integrated pipeline of the present application, there is a corresponding relationship in the implementation content, so the repetitive content will not be described.
FIG. 5 illustrates an exemplary system architecture 500 of an execution method of a persistent integrated pipeline or an execution device of a persistent integrated pipeline to which embodiments of the present application may be applied.
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505. The network 504 is used as a medium to provide communication links between the terminal devices 501, 502, 503 and the server 505. The network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 505 via the network 504 using the terminal devices 501, 502, 503 to receive or send messages or the like. Various communication client applications may be installed on the terminal devices 501, 502, 503, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 501, 502, 503 may be a variety of electronic devices with continuously integrated pipeline processing screens and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 505 may be a server providing various services, such as a background management server (by way of example only) that provides support for the execution failure information of pipeline nodes submitted by users using the terminal devices 501, 502, 503. The background management server can execute the pipeline, receive the node execution failure information of the pipeline, acquire the corresponding node identification, and further determine the failure node; determining unexecuted nodes, acquiring corresponding configuration information, modifying the configuration information of failed nodes, and updating a test entry identification list; based on the updated test entry identification list, executing unexecuted nodes and updated failed nodes after configuration information modification until all nodes in the pipeline are successfully executed. Therefore, the application only executes the failed node and the unexecuted node when retrying after continuously integrating the pipeline node failure, reduces the whole execution time of the pipeline, improves the execution efficiency of the pipeline and improves the customer experience.
It should be noted that, the execution method of the persistent integrated pipeline provided in the embodiment of the present application is generally executed by the server 505, and accordingly, the execution device of the persistent integrated pipeline is generally disposed in the server 505.
It should be understood that the number of terminal devices, networks and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 6, there is illustrated a schematic diagram of a computer system 600 suitable for use in implementing an embodiment of the present application. The terminal device shown in fig. 6 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present application.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the computer system 600 are also stored. The CPU601, ROM602, and RAM603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a liquid crystal credit authorization query processor (LCD), and the like, and a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the present application are performed when the computer program is executed by a Central Processing Unit (CPU) 601.
The computer readable medium shown in the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present application may be implemented in software or in hardware. The described units may also be provided in a processor, for example, described as: a processor includes a failed node determination unit, an update unit, and an execution unit. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
As another aspect, the present application also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs, and when the one or more programs are executed by the device, the device is caused to execute the pipeline, receive the node execution failure information of the pipeline, acquire the corresponding node identification, and further determine the failure node; determining unexecuted nodes, acquiring corresponding configuration information, modifying the configuration information of failed nodes, and updating a test entry identification list; based on the updated test entry identification list, executing unexecuted nodes and updated failed nodes after configuration information modification until all nodes in the pipeline are successfully executed. Therefore, the application only executes the failed node and the unexecuted node when retrying after continuously integrating the pipeline node failure, reduces the whole execution time of the pipeline, improves the execution efficiency of the pipeline and improves the customer experience.
According to the technical scheme provided by the embodiment of the application, only failed and unexecuted nodes are executed when retrying after continuously integrating pipeline nodes, so that the overall execution time of the pipeline is reduced, the execution efficiency of the pipeline is improved, and the customer experience is improved.
The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.

Claims (13)

1. A method of execution of a persistent integrated pipeline, comprising:
An execution pipeline receives the node execution failure information of the pipeline, acquires a corresponding node identifier, and further determines a failure node;
Determining a non-executed node, determining the type of the failed node, acquiring configuration information of a corresponding node with successful historical execution according to the type, updating the configuration information of the node with successful historical execution to the configuration information of the failed node, and further updating a test entry identification list;
and executing the unexecuted node and the updated failed node after configuration information modification based on the updated test entry identification list until all nodes in the pipeline are successfully executed.
2. The method of claim 1, wherein prior to the execution pipeline, the method further comprises:
acquiring source codes of nodes which are successfully executed, and analyzing the source codes to obtain class information;
and generating a directed graph by taking the class information as a vertex.
3. The method of claim 2, wherein the determining a failed node comprises:
Determining a common node based on the directed graph;
In response to determining that execution fails or is not performed on the common node, all tree nodes sharing the common node are determined to be failed nodes.
4. The method of claim 2, wherein the determining a failed node comprises:
Determining class information corresponding to each node in the pipeline;
In response to determining that multiple nodes in the pipeline call the same class information together, when one node fails to execute, the multiple nodes are determined to be failed nodes.
5. The method of claim 2, wherein the determining that the node is not executing comprises:
acquiring execution log information after each node in the pipeline is executed;
And analyzing the execution log information, and further determining non-execution nodes based on the directed graph.
6. The method of claim 5, wherein the parsing the execution log information to determine non-executing nodes based on the directed graph comprises:
analyzing the execution log information to determine class information and row information in the corresponding codes;
Determining the executed directed graph nodes based on the directed graph, the class information and the row information;
And determining non-executed nodes based on the executed directed graph nodes and all nodes of the directed graph.
7. The method of claim 1, wherein prior to the executing the unexecuted node and the updated failed node after modifying configuration information based on the updated list of test entry identifications, the method further comprises:
and carrying out retrying approval on each pipeline node corresponding to each entry identifier in the updated test entry identifier list, and executing corresponding reassembled execution node information based on the test entry identifier list in response to determining that retrying approval passes.
8. The method of claim 7, wherein retrying each pipeline node corresponding to each entry identifier in the updated list of test entry identifiers comprises:
determining workflow corresponding to each test entry identifier in the test entry identifier list;
and sending the test entry identification list, the execution configuration information and the corresponding workflow to a preset approval node for retrying approval.
9. The method of claim 1, wherein prior to the updating the test entry identification list, the method further comprises:
configuration information of the unexecuted node is modified.
10. The method of claim 1, wherein after all of the nodes in the pipeline have successfully executed, the method further comprises:
and aggregating the node information in the pipeline after successful execution to generate a new pipeline execution node.
11. An execution apparatus for a continuous integrated pipeline, comprising:
The failure node determining unit is configured to execute the pipeline, receive the execution failure information of the pipeline node, acquire the corresponding node identification and further determine the failure node;
The updating unit is configured to determine a non-executed node, determine the type of the failed node, acquire the configuration information of the corresponding node with successful historical execution according to the type, update the configuration information of the node with successful historical execution to the configuration information of the failed node, and update a test entry identification list;
And the execution unit is configured to execute the unexecuted node and the updated failed node after configuration information modification based on the updated test entry identification list until all nodes in the pipeline are successfully executed.
12. An execution electronic device of a continuous integrated pipeline, comprising:
One or more processors;
Storage means for storing one or more programs,
When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-10.
13. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-10.
CN202110577765.1A 2021-05-26 2021-05-26 Execution method and device of continuous integrated pipeline Active CN113297081B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110577765.1A CN113297081B (en) 2021-05-26 2021-05-26 Execution method and device of continuous integrated pipeline

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110577765.1A CN113297081B (en) 2021-05-26 2021-05-26 Execution method and device of continuous integrated pipeline

Publications (2)

Publication Number Publication Date
CN113297081A CN113297081A (en) 2021-08-24
CN113297081B true CN113297081B (en) 2024-06-18

Family

ID=77325271

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110577765.1A Active CN113297081B (en) 2021-05-26 2021-05-26 Execution method and device of continuous integrated pipeline

Country Status (1)

Country Link
CN (1) CN113297081B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116521134A (en) * 2023-03-13 2023-08-01 广州嘉为科技有限公司 Method, device and medium for modifying parameters of CI pipeline in execution process
CN118151948A (en) * 2024-05-13 2024-06-07 恒生电子股份有限公司 Compilation processing method, device, apparatus, storage medium, and program product

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110673936A (en) * 2019-09-18 2020-01-10 平安科技(深圳)有限公司 Breakpoint continuous operation method and device for arranging service, storage medium and electronic equipment

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
BRPI0615761A2 (en) * 2005-09-09 2011-05-24 Microsoft Corp programmability for xml document store
CN101599145A (en) * 2009-06-30 2009-12-09 中兴通讯股份有限公司 A kind of Workflow management method and system
US10120722B2 (en) * 2014-01-31 2018-11-06 Telefonaktiebolaget Lm Ericsson (Publ) Reordering workflows for network provisioning by using blocking penalty and compounded failure risk values for tasks
US9916235B2 (en) * 2016-08-09 2018-03-13 Seagate Technology Llc Code failure locator
CN108804209A (en) * 2017-05-05 2018-11-13 中思博安科技(北京)有限公司 The dispatching method and device of intelligent contract
US10310969B2 (en) * 2017-05-31 2019-06-04 Oracle International Corporation Systems and methods for test prediction in continuous integration environments
US10423523B2 (en) * 2018-02-02 2019-09-24 Ca, Inc. Automated selection of test cases for regression testing
US11258704B2 (en) * 2018-06-29 2022-02-22 Intel Corporation Technologies for managing network traffic through heterogeneous networks
US11061808B2 (en) * 2019-10-03 2021-07-13 Red Hat, Inc. Troubleshooting test failures that occurred during a testing phase of a continuous integration pipeline
CN110837373A (en) * 2019-10-17 2020-02-25 深圳市基石协作科技有限公司 Continuous integration and continuous delivery method, device, computer equipment and storage medium
CN111930624B (en) * 2020-08-12 2024-04-05 中国工商银行股份有限公司 Test link message data processing method and device
CN112394949B (en) * 2020-12-03 2022-04-22 中国科学院软件研究所 Service version dynamic configuration method for continuous integration

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110673936A (en) * 2019-09-18 2020-01-10 平安科技(深圳)有限公司 Breakpoint continuous operation method and device for arranging service, storage medium and electronic equipment

Also Published As

Publication number Publication date
CN113297081A (en) 2021-08-24

Similar Documents

Publication Publication Date Title
US20220253298A1 (en) Systems and methods for transformation of reporting schema
US20240045850A1 (en) Systems and methods for database orientation transformation
JP2018530070A (en) System and method for building, optimizing and implementing a platform on a cloud-based computing environment
CN113297081B (en) Execution method and device of continuous integrated pipeline
CN110865889A (en) Method and apparatus for transferring events between components
CN111126948A (en) Processing method and device for approval process
CN111125064A (en) Method and device for generating database mode definition statement
CN112559024A (en) Method and device for generating transaction code change list
US20230153081A1 (en) Methods and systems for correlating source code commitments and model result records during model development
US10409573B1 (en) Batch script decomposer
US11928627B2 (en) Workflow manager
CN110806967A (en) Unit testing method and device
CN113360365B (en) Flow test method and flow test system
US11030087B2 (en) Systems and methods for automated invocation of accessibility validations in accessibility scripts
CN113032004B (en) Method, apparatus and program product for managing development jobs in a development environment
CN114527993A (en) Application deployment method, device, equipment and medium
CN113448852A (en) Test case obtaining method and device, electronic equipment and storage medium
CN113268417B (en) Task execution method and device
CN113031960B (en) Code compiling method, device, server and storage medium
US11700177B2 (en) Cloud computing infrastructure pipelines with run triggers
CN118071310B (en) Business processing method and system based on flow engine
CN115934151A (en) Method and device for building multi-terminal small program
CN113326188A (en) Browser testing method and device
CN117785205A (en) Data evaluation method, device, electronic equipment and computer readable medium
CN117891732A (en) Code testing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant