CA3159788A1

CA3159788A1 - Version releasing method and device based on multi-environment offline task

Info

Publication number: CA3159788A1
Application number: CA3159788A
Authority: CA
Inventors: Haiyang Cao; Wei Xu; Zhenzhen Wang; Cheng Li
Original assignee: 10353744 Canada Ltd
Current assignee: 10353744 Canada Ltd
Priority date: 2019-11-06
Filing date: 2020-07-30
Publication date: 2021-05-14
Also published as: WO2021088437A1; CN110941446B; CN110941446A

Abstract

Disclosed in the present invention are a version releasing method and device based on a multi-environment offline task, for use in improving the task flow cross-environment release efficiency while reducing the task flow cross-environment release cost. The method comprises: creating a new task flow in a development environment and creating multiple task sub-nodes, and obtaining the dependencies between the task sub-nodes; obtaining a task flow reference version from a target environment, and when the task sub-nodes and/or the dependencies in the new task flow are changed compared with the task flow reference version, updating the version number of the new task flow and then submitting same; and obtaining task metadata information corresponding to the task flow having the new version number, releasing the task metadata information to the target environment, obtaining a task node file resource corresponding to the task flow having the new version number, and when the task node file resource is changed, triggering a file asynchronous upload service to release the task node file resource to the target environment, so as to implement release of the task flow having the new version number in the target environment.

Description

VERSION RELEASING METHOD AND DEVICE BASED ON MULTI-ENVIRONMENTS OFFLINE TASK
BACKGROUND OF THE INVENTION
Technical Field [0001] The present invention relates to the field of internet technology, and more particularly to a version releasing method and a version releasing device based on a multi-environments offline task.
Description of Related Art

[0002] Offline calculation is a calculation pattern in which all data to be calculated has been obtained before the calculating task starts and the data volume does not change, and the calculation result can be immediately obtained after one problem has been solved. Offline calculation is usually employed to perform complicated batch operations on a great deal of data, and exhibits such advantages as large computational amount, quick computational speed, and the capability to quickly enquire calculation results in batches.
With the rapid development and application of information technology, the scale of application systems in the industry has been rapidly enlarging itself, and data generated thereby has been increased abruptly in a crazy speed to such an extent that the processing capabilities of the traditional computing technology and information system can no longer satisfy. In order to address the present situation, offline calculation technology based on big data processing and the related offline calculation platforms have been vigorously developing.

[0003] In an age of data processing technology, the big data processing technology needs to solve the scalability and complexity of the system, in a big data platform are carried everyday Date recue/date received 2022-05-02 thousands upon ten thousands of ETL task schedules, these tasks are variegated in forms, tasks in a single environment can no longer meet the business requirements of multiple environments, and this necessitates the user to repetitively perform such works as developing, testing, releasing, operating and maintaining the offline task in different environments, whereby it tends to cause wastage of machine resources and manpower resources, thus finally rendering the offline task unduly high in development cost and relatively low in efficiency.
SUMMARY OF THE INVENTION

[0004] An objective of the present invention is to provide a version releasing method and a version releasing device based on a multi-environments offline task capable of enhancing efficiency in cross-environments release of the task flow at the same time of reducing the cost in cross-environments release of the task flow, as well as economizing on machine resources of various environments and reducing cost.

[0005] In order to achieve the above objective, according to the first aspect, the present invention provides a version releasing method based on a multi-environments offline task, and the method comprises:

[0006] newly creating a task flow in a development environment and creating a plurality of task child nodes, simultaneously obtaining dependency relations among the task child nodes, wherein the task child nodes contain task metadata information and task node file resources;

[0007] uploading the task metadata information to a database in the development environment for storage, and uploading the task node file resources to a distributed file system in the development environment for storage;

[0008] obtaining a task flow reference version from a target environment, and submitting the newly created task flow after having updated a version number therewith when change occurs to the task child nodes and/or the dependency relations in the newly created task Date recue/date received 2022-05-02 flow relative to the task flow reference version; and

[0009] obtaining task metadata information to which a task flow with a new version number corresponds from the database in the development environment, simultaneously releasing the task metadata information in the target environment, obtaining a task node file resource to which the task flow with the new version number corresponds from the distributed file system in the development environment, and triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version, to realize release of the task flow with the new version number in the target environment.

[0010] Preferably, prior to obtaining a task flow reference version from a target environment, the method further comprises:

[0011] performing presence detection on a data source that corresponds to the newly created task flow in the target environment via a releasing module, feeding risk reminder information and/or modification reminder information back to a user if the detection does not pass, obtaining the task flow reference version from the target environment and starting to perform a change calculation operation on the newly created task flow if the detection passes.

[0012] Preferably, the step of obtaining a task flow reference version from a target environment, and submitting the newly created task flow after having updated a version number therewith when change occurs to the task child nodes and/or the dependency relations in the newly created task flow relative to the task flow reference version includes:

[0013] obtaining a task event list to which a current version corresponds on the basis of the newly created task flow in the development environment, and obtaining a task event list to which a reference version corresponds on the basis of a reference version of a task flow in the target environment, wherein the reference version of the task flow is a task flow with the newest version number already released in the target environment;

Date recue/date received 2022-05-02

[0014] comparing the task event list to which the current version corresponds with the task event list to which the reference version corresponds, recording change information in a change list when change occurs to the task child nodes and/or the dependency relations in the newly created task flow, and simultaneously updating the version number of the newly created task flow;

[0015] storing change information of the task flow with the new version number in a history table, and marking deleted task child nodes in the history table;

[0016] duplicating task child node information and/or event information of the task flow with the new version number entirely in a total record sheet for recordation, and incrementally duplicating changed task child node information and/or event information of the task flow with the new version number in an incremental record sheet; and

[0017] submitting, by a developing module, the task flow with the new version number to the releasing module, and simultaneously notifying a version update status of the task flow with the new version number to the releasing module.

[0018] Further, prior to the step of obtaining task metadata information to which a task flow with a new version number corresponds from the database in the development environment, simultaneously releasing the task metadata information in the target environment, the method further comprises:

[0019] judging whether the currently released task flow with the new version number is consecutive to the version number of the task flow reference version, if yes, directly employing an incrementing pattern for release, if not, employing a merging-before-incrementing pattern for release.

[0020] Preferably, the step of obtaining a task node file resource to which the task flow with the new version number corresponds from the distributed file system in the development environment, and triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version includes:

Date recue/date received 2022-05-02

[0021] receiving the task flow with the new version number to be released, obtaining corresponding task metadata information from the database and releasing the same in the target environment;

[0022] necessarily obtaining a corresponding task node file resource from the distributed file system when the task metadata information is successfully released in the target environment, and triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version;

[0023] feeding where the problem lies back to the user via a running log and reminding the user to implement a retry operation when the task metadata information is not successfully released in the target environment; and

[0024] changing, by the releasing module, a release status of the task flow with the new version number via a real-time consumption message after the task flow with the new version number has been successfully released across environments in the target environment.

[0025] Preferably, after the task flow with the new version number has been successfully released in the target environment, the method further comprises:

[0026] reminding the user to upgrade and run the task flow with the new version number already released in the target environment, to replace running of the task flow with the old version number in the target environment.

[0027] Preferably, after the step of realizing release of the task flow with the new version number in the target environment, the method further comprises:

[0028] scheduling and executing the task flow with the new version number through a running module and presenting a running result, and simultaneously presenting to the user in the releasing module total metadata information and dependency relations among various task child nodes of the task flow with the new version number, so as to enable the user to select a task flow with a corresponding version number to execute a rollback edit operation.
Date recue/date received 2022-05-02

[0029] Exemplarily, the target environment includes one or more of a test environment, a quasi-production environment, and a production environment.

[0030] Exemplarily, the task metadata information includes one or more of task basic information, a task name, task description, task frequency, a task instance, a task priority, task validation time, a task type, task configuration information, event information, task dependency information, task parameter information, task data source information, and file dependency information, and the node file resource includes one or more of a task-dependent sql file, a task running Jar packet or a task running zip file, and an xml file configured by the task.

[0031] In comparison with prior-art technology, the version releasing method based on a multi-environments offline task provided by the present invention achieves the following advantageous effects.

[0032] In the version releasing method based on a multi-environments offline task provided by the present invention, task flow information is created or edited to construct task metadata information of a plurality of task child nodes, dependency relations among the plural task child nodes are simultaneously generated, a task node file resource of each task child node is correspondingly added, the constructed task metadata information and dependency relations are thereafter stored in a database, and the task node file resources are stored in a distributed file system; by extracting a task flow reference version from the target environment and comparing and verifying the same with the newly created task flow, if change occurs to the task child nodes and/or the dependency relations of the newly created task flow, the newly created task flow is assigned with a new version number, and an operation is executed to submit the task flow with the new version number to the target environment, what corresponds after the submission is an operation of releasing the task flow with the new version number from the source environment to the target environment, Date recue/date received 2022-05-02 and the specific steps are: firstly obtaining task metadata information to which the task flow with the new version number corresponds from the database, and releasing the task metadata information from the source environment to the target environment;
after the task metadata information has been released in the target environment to completion, it is further required to verify whether change occurs to the task node file resource to which the task flow with the new version number corresponds relative to the task flow reference version in the target environment, and a file asynchronous transmission service is triggered when change occurs to release the task node file resource in the target environment, to finally realize release of the task flow with the new version number in the target environment.

[0033] Seen as such, the present invention makes it possible to realize cross-environmental release of task flows, to reduce consumption of machine resources of operations in different environments at the same time of reducing such operations as developing, testing, releasing, operating and maintaining the task flow in different environments, and to achieve the technical effects of enhancing efficiency in cross-environments release of the task flow and reducing the cost in cross-environments release of the task flow.

[0034] According to the second aspect, the present invention provides a version releasing device based on a multi-environments offline task, the device is applied to the version releasing method based on a multi-environments offline task as recited in the above technical solution, and comprises:

[0035] a developing unit, for newly creating a task flow in a development environment and creating a plurality of task child nodes, simultaneously obtaining dependency relations among the task child nodes, wherein the task child nodes contain task metadata information and task node file resources;

[0036] an uploading unit, for uploading the task metadata information to a database in the development environment for storage, and uploading the task node file resources to a distributed file system in the development environment for storage;

Date recue/date received 2022-05-02

[0037] a comparing unit, for obtaining a task flow reference version from a target environment, and submitting the newly created task flow after having updated a version number therewith when change occurs to the task child nodes and/or the dependency relations in the newly created task flow relative to the task flow reference version;

[0038] a releasing unit, for obtaining task metadata information to which a task flow with a new version number corresponds from the database in the development environment, simultaneously releasing the task metadata information in the target environment, obtaining a task node file resource to which the task flow with the new version number corresponds from the distributed file system in the development environment, and triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version, to realize release of the task flow with the new version number in the target environment; and

[0039] a running unit, for initiating a task scheduling request to execute a version releasing task of task flows.

[0040] Preferably, the developing unit is further capable of providing visualized canvas presentation manipulatable by dragging, and an exclusive lock function is introduced in the development process to ensure consistency in developments of the task flow in various environments.

[0041] In comparison with prior-art technology, the advantageous effects achieved by the version releasing device based on a multi-environments offline task provided by the present invention are identical with the advantageous effects achievable by the version releasing method based on a multi-environments offline task as provided by the foregoing technical solution, so no repetition is redundantly made in this context.

[0042] According to the third aspect, the present invention provides a computer-readable storage medium storing thereon a computer program that executes steps of the aforementioned Date recue/date received 2022-05-02 version releasing method based on a multi-environments offline task when it is run by a processor.

[0043] In comparison with prior-art technology, the advantageous effects achieved by the computer-readable storage medium provided by the present invention are identical with the advantageous effects achievable by the version releasing method based on a multi-environments offline task as provided by the foregoing technical solution, so no repetition is redundantly made in this context.
BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described here are employed to provide further understanding to the present invention, and constitute a part of the present invention. The schematic embodiments of the present invention and descriptions thereof are meant to explain the present invention, rather than to inadequately restrict the present invention. In the drawings,

[0045] Fig. 1 is a flowchart schematically illustrating the version releasing method based on a multi-environments offline task in an embodiment;

[0046] Fig. 2 is a flowchart schematically illustrating the release of a task flow in an embodiment;

[0047] Fig. 3 is a functional framework view illustrating the version releasing method based on a multi-environments offline task in an embodiment;

[0048] Fig. 4 is a view exemplarily illustrating an application scenario for the release of a task flow in an embodiment; and

[0049] Fig. 5 is a view exemplarily illustrating an application scenario for the release of another task flow in an embodiment.

Date recue/date received 2022-05-02 DETAILED DESCRIPTION OF THE INVENTION

[0050] In order to make more lucid and clear the aforementioned objectives, features and advantages of the present invention, the technical solutions in the embodiments of the present invention will be more clearly and comprehensively described below with reference to the accompanying drawings in the embodiments of the present invention.
Apparently, the embodiments as described are merely partial, rather than the entire, embodiments of the present invention. All other embodiments obtainable by persons ordinarily skilled in the art on the basis of the embodiments in the present invention without spending creative effort shall all fall within the protection scope of the present invention.

[0051] Embodiment 1

[0052] Please refer to Figs. 1 to 3, this embodiment provides a version releasing method based on a multi-environments offline task, and the method comprises:

[0053] newly creating a task flow in a development environment and creating a plurality of task child nodes, simultaneously obtaining dependency relations among the task child nodes, wherein the task child nodes contain task metadata information and task node file resources; uploading the task metadata information to a database in the development environment for storage, and uploading the task node file resources to a distributed file system in the development environment for storage; obtaining a task flow reference version from a target environment, and submitting the newly created task flow after having updated a version number therewith when change occurs to the task child nodes and/or the dependency relations in the newly created task flow relative to the task flow reference version; and obtaining task metadata information to which a task flow with a new version number corresponds from the database in the development environment, simultaneously releasing the task metadata information in the target environment, Date recue/date received 2022-05-02 obtaining a task node file resource to which the task flow with the new version number corresponds from the distributed file system in the development environment, and triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version, to realize release of the task flow with the new version number in the target environment.

[0054] In the version releasing method based on a multi-environments offline task provided by this embodiment, task flow information is created or edited to construct task metadata information of a plurality of task child nodes, dependency relations among the plural task child nodes are simultaneously generated, a task node file resource of each task child node is correspondingly added, the constructed task metadata information and dependency relations are thereafter stored in a database, and the task node file resources are stored in a distributed file system; by extracting a task flow reference version from the target environment and comparing and verifying the same with the newly created task flow, if change occurs to the task child nodes and/or the dependency relations of the newly created task flow, the newly created task flow is assigned with a new version number, and an operation is executed to submit the task flow with the new version number to the target environment, what corresponds after the submission is an operation of releasing the task flow with the new version number from the source environment to the target environment, and the specific steps are: firstly obtaining task metadata information to which the task flow with the new version number corresponds from the database, and releasing the task metadata information from the source environment to the target environment;
after the task metadata information has been released in the target environment to completion, it is further required to verify whether change occurs to the task node file resource to which the task flow with the new version number corresponds relative to the task flow reference version in the target environment, and a file asynchronous transmission service is triggered when change occurs to release the task node file resource in the target environment, to finally realize release of the task flow with the new version number in Date recue/date received 2022-05-02 the target environment.

[0055] Seen as such, this embodiment makes it possible to realize cross-environmental release of task flows, to reduce consumption of machine resources of operations in different environments at the same time of reducing such operations as developing, testing, releasing, operating and maintaining the task flow in different environments, and to achieve the technical effects of enhancing efficiency in cross-environments release of the task flow and reducing the cost in cross-environments release of the task flow. Seen as such, this embodiment makes it possible to realize cross-environmental release of newly created task flows, to reduce consumption of machine resources of operations in different environments at the same time of reducing such operations as developing, testing, releasing, operating and maintaining the task flow in different environments, and to achieve the technical effects of enhancing efficiency in cross-environments release of the task flow and reducing the cost in cross-environments release of the task flow.

[0056] In addition, through this embodiment it can be easy to release the newly created task flow in different environments for running, application of the file asynchronous transmission service makes it possible to verify the idempotency of the node file resource and to feed back the migration status of the task node file resource in real time. At the same time, the various links of the release can be controlled, and better monitoring means is provided.

[0057] As should be noted, the file asynchronous transmission service in this embodiment is realized via a file service asynchronous device. The file service asynchronous device can provide the file asynchronous transmission service to resource files of different environments, and the realization principle thereof is as follows: various bottom cluster environments are opened through a proxy server, the releasing module sends out a service request to the file service asynchronous device via the mode of HTTP or RPC, message consumption is carried out via message middleware Kaffka, asynchronous information concerning succeeded transmission or failed transmission of the resource file of the task Date recue/date received 2022-05-02 flow to the target environment is received and the release status of the task flow is replaced, so that the releasing module timely traces and changes the release status of the task flow, so as to facilitate to perform other operations. Through application of the asynchronous transmission service, it is made possible to shield different bottom environments, it is not required for the user to pay more attention to the difference of the release environments, and it suffices to put the emphasis merely on the development of the offline task.

[0058] To facilitate comprehension, the file service asynchronous device is introduced in greater detail now: the concept of thread pool is introduced to realize the application of a highly concurrent scenario, a corePoolSize parameter is set, new threads are created to execute the task in the case corePoolSize is not exceeded, while the task is added to BlockingQueue for execution in the case corePoolSize is equalized or exceeded, and the task will be denied if the number of created new threads exceeds maximumPoolSize; the file service asynchronous device supports dynamic addition of threads, when there are great quantities of task flows to be processed in the system, corePoolSize maximumPoolSize parameters can be set to realize concurrent processing, and when there are great many tasks in a single task flow, a full load pattern can be started to equip the single task flow with sub-threads and to allocate the remaining threads to process the single task flow. The file service asynchronous device also functions to release the node file resource from the development environment to the target environment, to detect whether the node file resource in the target environment is present and whether the configuration switch is covered or passed over, to return the size of the file after successful release and to push the status information after success to kaffa for consumption by the releasing center. The file service asynchronous device also includes a reverse copying function, with respect to a scenario initially deployed by the system, if the task flow is present before the production environment (target environment), at this time it is only required for the user to export the task flow and subsequently import it to the development environment (source environment), the corresponding node file resource Date recue/date received 2022-05-02 will be obtained when release of the node file resource is involved, and it is then duplicated to the development environment, thus providing the user with an imperceptible using experience.

[0059] Exemplarily, the task metadata information includes one or more of task basic information, a task name, task description, task frequency, a task instance, a task priority, task validation time, a task type, task configuration information, event information, task dependency information, task parameter information, task data source information, and file dependency information, and the node file resource includes one or more of a task-dependent sql file, a task running Jar packet or a task running zip file, and an xml file configured by the task. The target environment includes one or more of a test environment, a quasi-production environment, and a production environment.

[0060] In this embodiment, prior to obtaining a task flow reference version from a target environment, the method further comprises: performing presence detection on a data source that corresponds to the newly created task flow in the target environment via a releasing module, feeding risk reminder information and/or modification reminder information back to a user if the detection does not pass, obtaining the task flow reference version from the target environment and starting to perform a change calculation operation on the newly created task flow if the detection passes.

[0061] In this embodiment, the step of obtaining a task flow reference version from a target environment, and submitting the newly created task flow after having updated a version number therewith when change occurs to the task child nodes and/or the dependency relations in the newly created task flow relative to the task flow reference version includes:

[0062] obtaining a task event list to which a current version corresponds on the basis of the newly created task flow in the development environment, and obtaining a task event list to which a reference version corresponds on the basis of a reference version of a task flow in the target environment, wherein the reference version of the task flow is a task flow with the Date recue/date received 2022-05-02 newest version number already released in the target environment; comparing the task event list to which the current version corresponds with the task event list to which the reference version corresponds, recording change information in a change list when change occurs to the task child nodes and/or the dependency relations in the newly created task flow, and simultaneously updating the version number of the newly created task flow;
storing change information of the task flow with the new version number in a history table, and marking deleted task child nodes in the history table; duplicating task child node information and/or event information of the task flow with the new version number entirely in a total record sheet for recordation, and incrementally duplicating changed task child node information and/or event information of the task flow with the new version number in an incremental record sheet; and submitting, by a developing module, the task flow with the new version number to the releasing module, and simultaneously notifying a version update status of the task flow with the new version number to the releasing module.

[0063] During specific implementation, after the presence detection of the data source in the target environment has passed, it is now required to start the change calculation operation of the newly created task flow, and the process is as follows: a task event list to which the newly created task flow corresponds in the development environment is extracted, and a task event list to which the reference version of the task flow corresponds in the target environment is extracted, since the task event lists store therein the task child node information and the dependency relation information among the nodes, it can be derived whether change occurs to the newly created task flow relative to the reference version of the task flow after comparing the two task event lists, a new version number is assigned anew to the newly created task flow when the comparison result shows that the change occurs, change information of the task flow with the new version number is simultaneously stored in a history table, the task child nodes deleted relative to the task flow with the old version number in the development environment are thereafter marked, such marking aims to facilitate the user to select a task flow with a corresponding version Date recue/date received 2022-05-02 number to execute a rollback edit operation, the history table is simultaneously modified on the basis of jobid of the deleted task child nodes to update the newest version information in the history table, subsequently the task child node information and/or event information of the task flow with the new version number within the canvas are entirely duplicated to a total record sheet for recordation, the changed task child node information and/or event information of the task flow with the new version number are incrementally duplicated to an incremental record sheet, the task flow nodes are simultaneously written in a change list, the developing module thereafter submits the task flow with the new version number to the releasing module, the developing module notifies the releasing module after submission, and simultaneously transmits the basic task flow information, development environment information and target environment information as released to the releasing module.

[0064] In this embodiment, the step of obtaining a task node file resource to which the task flow with the new version number corresponds from the distributed file system in the development environment, and triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version includes:

[0065] receiving the task flow with the new version number to be released, obtaining corresponding task metadata information from the database and releasing the same in the target environment; necessarily obtaining a corresponding task node file resource from the distributed file system when the task metadata information is successfully released in the target environment, and triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version; feeding where the problem lies back to the user via a running log and reminding the user to implement a retry operation when the task metadata information is not successfully released in the target environment;
and changing, by the releasing module, a release status of the task flow with the new version number via a real-time consumption message after the task flow with the new Date recue/date received 2022-05-02 version number has been successfully released across environments in the target environment.

[0066] During specific implementation, the task flow with the new version number to be released is received, the status of the task flow with the new version number is marked as to be released, a release instruction manipulated by the user at the releasing module is received, the corresponding task metadata information in the development environment is released to the target environment, and the file asynchronous transmission service is triggered after release is completed; as should be noted, the file asynchronous transmission service is not started if there is no relevant sub-task dependent resource in the target environment or no change occurs to the task node file resource, if any problem occurs in the release of the task metadata information, the cause of the problem can be presented to the user via a running log, and the user is reminded to click again to manipulate the release instruction for renewed release, until the task metadata information is successfully released; the file asynchronous transmission service can support the concurrent request service of multiple task flow resources, and can generate uniquely identifying primary key identification file resource and idempotency of the task flow for verification; the file asynchronous service starts to operate when the releasing module submits the service request, new releasing requests can be incessantly received during the process, asynchronous concurrent service is realized via the thread pool technique, after the single task flow resource has been successfully released across environments, the message is notified to the releasing module via kaffka, and the releasing module changes the corresponding status of the task flow via a real-time consumption message. Besides, during the process of releasing the task flow, if the task metadata information of the task flow with the new version number is released for the first time, release is performed on the basis of the total record sheet, if the task metadata information of the task flow with the new version number is not submitted for the first time, incremental release is performed on the basis of the incremental record sheet.

Date recue/date received 2022-05-02

[0067] In this embodiment, after the task flow with the new version number has been successfully released in the target environment, the method further comprises:

[0068] reminding the user to upgrade and run the task flow with the new version number already released in the target environment, to replace running of the task flow with the old version number in the target environment.

[0069] In this embodiment, after the step of realizing release of the task flow with the new version number in the target environment, the method further comprises:

[0070] scheduling and executing the task flow with the new version number through a running module and presenting a running result, and simultaneously presenting to the user in the releasing module total metadata information and dependency relations among various task child nodes of the task flow with the new version number, so as to enable the user to select a task flow with a corresponding version number to execute a rollback edit operation. For instance, the rollback edit operation is executed on task flows of previous version numbers according to the markings in the history table.

[0071] In this embodiment, prior to the step of obtaining task metadata information to which a task flow with a new version number corresponds from the database in the development environment, simultaneously releasing the task metadata information in the target environment, the method further comprises:

[0072] judging whether the currently released task flow with the new version number is consecutive to the version number of the task flow reference version, if yes, directly employing an incrementing pattern for release, if not, employing a merging-before-incrementing pattern for release.

[0073] During specific implementation, if the version number of the new version of task flow (the newly created task flow) is consecutive to the task flow reference version number, an incrementing pattern is employed for release, namely only releasing the added task child nodes in the target environment on the basis of the dependency relations, if the Date recue/date received 2022-05-02 version number of the currently released task flow is not consecutive to the task flow reference version number, a merging-before-incrementing pattern is employed for release, in which the merging-before-incrementing pattern means that, if there are n number of versions developed for the task flows, but only the first version is released to the production environment, whereas the second to the n- lth versions are not released to the production environment, in this case if it is desired to release the nth version to the production environment, it will be required to find out the change records of the task flows of the second to the nth versions and the newest task flow reference version in the production environment, a final releasing result is derived after comparing and merging, and the relevant task child nodes are released to the target environment in accordance with the final releasing result.

[0074] To facilitate comprehension, the merging-before-incrementing releasing pattern in the present embodiment is exemplarily explained below.

[0075] If the newest production version of the task flow is V1.1, while the version of the development environment has been developed to task flow V1.4 at this time, an incrementing concept with merging versions will be involved in this case;

[0076] V1.0 version contains altogether three task child nodes A, B, C, and the dependency relations are A¨>B. A¨C. the V1.0 version is released from the development environment to the test environment, and subsequently released to the production environment;

[0077] V1.1 version is further developed as it has been found that requirement was changed, the version contains three task child nodes A, B, C. and the dependency relations are A¨>B.
A,C, in which the configuration information of task child node A is modified, the version is released from the development environment to the test environment, and subsequently incrementally released to the production environment, the incrementing pattern is selected for release at this time, so it suffices to only release the changed task child node A;

Date recue/date received 2022-05-02

[0078] V1.2 version is further developed in the duration, the version contains two task child nodes A, B, and the dependency relation is A¨d3, task child node C and dependency thereof are deleted, and the version is only released from the development environment to the test environment at this time;

[0079] V1.3 version contains three task child nodes A, B, D, and the dependency relations are A,B,D, in which node D is newly added this time, and the version is only released from the development environment to the test environment at this time;

[0080] finally, V1.4 version is further developed, the version contains three task child nodes A, B, D, and the dependency relations are A¨d3¨>13, in which task child node A is again modified, and the version is thereafter released from the development environment to the test environment, and finally released to the production environment;

[0081] till this time, three version adjustments have been undergone from production version V1.1 to production version V1.4, in which task child node C is deleted from V1.2 version, task child node D is added to V1.3 version, task child node A is modified in V1.4 version, and in the merged version V1.4 as finally obtained is modified task child node A, remains unchanged task child node B, is deleted task child node C, and is newly added task child node D, if the merging-before-incrementing pattern was not used, the developing personnel would not know the changes occurred between V1.1 version and V1.4 version, in which case if task child nodes A, B, D are merely simply released to the production environment, the releasing result possibly presented would contain task child nodes A, B, C, D, rather than the final exact releasing result that should contain task child nodes A, B, D.

[0082] Fig. 4 is a view exemplarily illustrating an application scenario for the release of a task flow in this embodiment, and this scenario is correspondingly explained below.

[0083] Task flow S is developed in the development environment (source environment), and contains task child nodes A, B, C, V1.0 version is only released to a grayscale environment, the version of the production environment (target environment) is selected Date recue/date received 2022-05-02 at this time to serve as the task flow reference version, as can be seen from Fig. 4, the task flow reference version is empty, and the total releasing pattern is employed for release this time; V1.1 version contains task child nodes A, C, D, it is released to the production environment, the version of the production environment is still selected at this time to serve as the task flow reference version, as can be seen from Fig. 4, the task flow reference version is still empty, the total releasing pattern is employed for release to the grayscale environment this time, and the total releasing pattern is also employed for release to the production environment.

[0084] Fig. 5 is a view exemplarily illustrating an application scenario for the release of another task flow in this embodiment, and this scenario is correspondingly explained below.

[0085] Task flow S is developed in the development environment, and contains task child nodes A, B, C; V1.0 version is released to the grayscale environment and the production environment, the version of the production environment is selected at this time to serve as the task flow reference version, as can be seen from Fig. 5, the task flow reference version is empty, and the total releasing pattern is employed for release this time; task child node D is newly added to V1.1 version, task child nodes A, B, C, D are released to the grayscale environment, the version of the production environment is still selected at this time to serve as the task flow reference version, as can be seen from Fig. 5, the task flow reference version is V1.0 containing task child nodes A, B, C, the incrementing release pattern is employed for release to the grayscale environment this time, and it suffices at this time to only release the relevant information of the added task child node D, while the other task child nodes are not changed and not released; task child node E is newly added to V1.2 version, task child nodes A, B, C, D, E are contained, these are released to the grayscale environment and the version of the production environment is selected to serve as the task flow reference version, as can be seen from Fig.
5, the task flow reference version is still V1.0 containing task child nodes A, B, C, the incrementing release pattern is employed for release to the grayscale environment this time, and it Date recue/date received 2022-05-02 suffices to only release the relevant information of the added task child node D and task child node E to the production environment.

[0086] The above two scenarios are only exemplarily described, the task flow reference version can as well be selected from various versions in the development environment or the grayscale environment, and the present embodiment makes no restriction thereto.

[0087] Exemplarily, the task flow statuses in the releasing module can be classified as: to be released, to be upgraded, release failed, release terminated, release succeeded. By to be released is meant that triggering is effected when the task flow metadata information is submitted from the developing module to the releasing module; by to be updated is meant that the task flow metadata information has been successfully released to the corresponding target environment, and the old version is replaced with the newly released version for running; by release failed is meant that abnormality occurs in the releasing process or the releasing period has reached the triggering status; by release terminated is meant that the user can terminate the releasing procedure, and it is possible at this time to develop again at the developing module, and to thereafter submit to the releasing module for renewed release; by release succeeded is meant that the user has successfully released the task flow metadata information of the new version number together with the dependent resources to the target environment. In addition, the aforementioned release statuses are only exemplarily described, and are not specifically restricted, the developing personnel can correspondingly circulate and expand the statuses based on business requirements, to which no description is redundantly made in this context.

[0088] The aforementioned developing module can be further employed for uploading offline resources and visualized canvas presentation of offline task flows, and realizing self-defined configuration of parameters through page-clicking or dragging edition;
a cooperative developing function by multiple persons is also supported, the concept of exclusive lock is introduced in the development, whereby the current developing Date recue/date received 2022-05-02 personnel proactively obtains a development lock during task development, while the other users cannot develop the task flow, and the other developing personnel are enabled for development only after the current developing personnel proactively releases the task lock; through the setup of the exclusive lock is ensured the consistency in development of the task flow in various environments, and the exclusive lock is visible to all the developing personnel, each developing personnel can see by whom the current task flow is locked, and the user can negotiate with the occupier for unlocking, so that is guaranteed the substantiveness of task flow development.

[0089] Embodiment 2

[0090] This embodiment provides a version releasing device based on a multi-environments offline task, and the device comprises:

[0091] a developing unit, for newly creating a task flow in a development environment and creating a plurality of task child nodes, simultaneously obtaining dependency relations among the task child nodes, wherein the task child nodes contain task metadata information and task node file resources;

[0092] an uploading unit, for uploading the task metadata information to a database in the development environment for storage, and uploading the task node file resources to a distributed file system in the development environment for storage;

[0093] a comparing unit, for obtaining a task flow reference version from a target environment, and submitting the newly created task flow after having updated a version number therewith when change occurs to the task child nodes and/or the dependency relations in the newly created task flow relative to the task flow reference version;

[0094] a releasing unit, for obtaining task metadata information to which a task flow with a new version number corresponds from the database in the development environment, simultaneously releasing the task metadata information in the target environment, obtaining a task node file resource to which the task flow with the new version number corresponds from the distributed file system in the development environment, and Date recue/date received 2022-05-02 triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version, to realize release of the task flow with the new version number in the target environment; and

[0095] a running unit, for initiating a task scheduling request to execute a version releasing task of task flows.

[0096] Preferably, the developing unit is further capable of providing visualized canvas presentation manipulatable by dragging, and an exclusive lock function is introduced in the development process to ensure consistency in developments of the task flow in various environments.

[0097] In comparison with prior-art technology, the advantageous effects achieved by the version releasing device based on a multi-environments offline task provided by this embodiment are identical with the advantageous effects achievable by the version releasing method based on a multi-environments offline task as provided by the foregoing embodiment, so no repetition is redundantly made in this context.

[0098] Embodiment 3

[0099] This embodiment provides a computer-readable storage medium storing thereon a computer program that executes steps of the aforementioned version releasing method based on a multi-environments offline task when it is run by a processor.

[0100] In comparison with prior-art technology, the advantageous effects achieved by the computer-readable storage medium provided by this embodiment are identical with the advantageous effects achievable by the version releasing method based on a multi-environments offline task as provided by the foregoing technical solution, so no repetition is redundantly made in this context.

Date recue/date received 2022-05-02

[0101] As understandable by persons ordinarily skilled in the art, the entire or partial steps that realize the method of the present invention can be completed via a program that instructs relevant hardware, the program can be stored in a computer-readable storage medium, and subsumes the various steps of the method in the aforementioned embodiment when it is executed, and the storage medium can be ROM/RAM, a magnetic disk, an optical disk, a memory card, etc.

[0102] The above description is merely directed to specific modes of execution of the present invention, but the protection scope of the present invention is not restricted thereby. Any change or replacement easily conceivable to persons skilled in the art within the technical range disclosed by the present invention shall be covered by the protection scope of the present invention. Accordingly, the protection scope of the present invention shall be based on the protection scope as claimed in the Claims.
Date recue/date received 2022-05-02

Claims

What is claimed is:

1. A
version releasing method based on a multi-environments offline task, characterized in that the method comprises:
newly creating a task flow in a development environment and creating a plurality of task child nodes, simultaneously obtaining dependency relations among the task child nodes, wherein the task child nodes contain task metadata information and task node file resources;
uploading the task metadata information to a database in the development environment for storage, and uploading the task node file resources to a distributed file system in the development environment for storage;
obtaining a task flow reference version from a target environment, and submitting the newly created task flow after having updated a version number therewith when change occurs to the task child nodes and/or the dependency relations in the newly created task flow relative to the task flow reference version; and obtaining task metadata information to which a task flow with a new version number corresponds from the database in the development environment, simultaneously releasing the task metadata information in the target environment, obtaining a task node file resource to which the task flow with the new version number corresponds from the distributed file system in the development environment, and triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version, to realize release of the task flow with the new version number in the target environment.
the task metadata information includes one or more of task basic information, a task name, task description, task frequency, a task instance, a task priority, task validation time, a task type, task configuration information, event information, task dependency information, task parameter information, task data source information, and file dependency information, and the task node file resource includes one or more of a task-dependent sql file, a task running Jar packet or a task running zip file, and an xml file configured by the task. The target environment includes one or more of a test environment, a quasi-production environment, and a production environment.

2. The method according to Claim 1, characterized in further comprising, prior to obtaining a task flow reference version from a target environment:
performing presence detection on a data source that corresponds to the newly created task flow in the target environment via a releasing module, feeding risk reminder information and/or modification reminder information back to a user if the detection does not pass, obtaining the task flow reference version from the target environment and starting to perform a change calculation operation on the newly created task flow if the detection passes.

3. The method according to Claim 1 or 2, characterized in that the step of obtaining a task flow reference version from a target environment, and submitting the newly created task flow after having updated a version number therewith when change occurs to the task child nodes and/or the dependency relations in the newly created task flow relative to the task flow reference version includes:
obtaining a task event list to which a current version corresponds on the basis of the newly created task flow in the development environment, and obtaining a task event list to which a reference version corresponds on the basis of a reference version of a task flow in the target environment, wherein the reference version of the task flow is a task flow with the newest version number already released in the target environment;
comparing the task event list to which the current version corresponds with the task event list to which the reference version corresponds, recording change information in a change list when change occurs to the task child nodes and/or the dependency relations in the newly created task flow, and simultaneously updating the version number of the newly created task flow;
storing change information of the task flow with the new version number in a history table, and marking deleted task child nodes in the history table;
duplicating task child node information and/or event information of the task flow with the new version number entirely in a total record sheet for recordation, and incrementally duplicating changed task child node information and/or event information of the task flow with the new version number in an incremental record sheet; and submitting, by a developing module, the task flow with the new version number to the releasing module, and simultaneously notifying a version update status of the task flow with the new version number to the releasing module.

4. The method according to Claim 3, characterized in further comprising, prior to the step of obtaining task metadata information to which a task flow with a new version number corresponds from the database in the development environment, simultaneously releasing the task metadata information in the target environment:
judging whether the currently released task flow with the new version number is consecutive to the version number of the task flow reference version, if yes, directly employing an incrementing pattern for release, if not, employing a merging-before-incrementing pattern for release.

5. The method according to Claim 3, characterized in that the step of obtaining a task node file resource to which the task flow with the new version number corresponds from the distributed file system in the development environment, and triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version includes:
receiving the task flow with the new version number to be released, obtaining corresponding task metadata information from the database and releasing the same in the target environment;
necessarily obtaining a corresponding task node file resource from the distributed file system when the task metadata information is successfully released in the target environment, and triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version;
feeding where the problem lies back to the user via a running log and reminding the user to implement a retry operation when the task metadata information is not successfully released in the target environment; and changing, by the releasing module, a release status of the task flow with the new version number via a real-time consumption message after the task flow with the new version number has been successfully released across environments in the target environment.

6. The method according to Claim 5, characterized in further comprising, after the task flow with the new version number has been successfully released in the target environment:
reminding the user to upgrade and run the task flow with the new version number already released in the target environment, to replace running of the task flow with the old version number in the target environment.

7. The method according to Claim 5, characterized in further comprising, after the step of realizing release of the task flow with the new version number in the target environment:
scheduling and executing the task flow with the new version number through a running module and presenting a running result, and simultaneously presenting to the user in the releasing module total metadata information and dependency relations among various task child nodes of the task flow with the new version number, so as to enable the user to select a task flowwith a corresponding version number to execute a rollback edit operation.

8. The method according to Claim 1, characterized in that the target environment includes one or more of a test environment, a quasi-production environment, and a production environment.

9. A
version releasing device based on a multi-environments offline task, characterized in that the device comprises:
a developing unit, for newly creating a task flow in a development environment and creating a plurality of task child nodes, simultaneously obtaining dependency relations among the task child nodes, wherein the task child nodes contain task metadata information and task node file resources;
an uploading unit, for uploading the task metadata information to a database in the development environment for storage, and uploading the task node file resources to a distributed file system in the development environment for storage;
a comparing unit, for obtaining a task flow reference version from a target environment, and submitting the newly created task flow after having updated a version number therewith when change occurs to the task child nodes and/or the dependency relations in the newly created task flow relative to the task flow reference version;
a releasing unit, for obtaining task metadata information to which a task flow with a new version number corresponds from the database in the development environment, simultaneously releasing the task metadata information in the target environment, obtaining a task node file resource to which the task flow with the new version number corresponds from the distributed file system in the development environment, and triggering a file asynchronous transmission service to release the task node file resource in the target environment when change occurs to the task node file resource relative to the task flow reference version, to realize release of the task flow with the new version number in the target environment; and a running unit, for initiating a task scheduling request to execute a version releasing task of task flows.
the task metadata information includes one or more of task basic information, a task name, task description, task frequency, a task instance, a task priority, task validation time, a task type, task configuration information, event information, task dependency information, task parameter information, task data source information, and file dependency information, and the task node file resource includes one or more of a task-dependent sql file, a task running Jar packet or a task running zip file, and an xml file configured by the task. The target environment includes one or more of a test environment, a quasi-production environment, and a production environment.

10. The device according to Claim 9, characterized in that the developing unit is further capable of providing visualized canvas presentation manipulatable by dragging, and that an exclusive lock function is introduced in the development process to ensure consistency in developments of the task flow in various environments.