WO2023093200A1

WO2023093200A1 - Method and apparatus for asynchronously processing tasks, and storage medium and electronic apparatus

Info

Publication number: WO2023093200A1
Application number: PCT/CN2022/117071
Authority: WO
Inventors: 韩林
Original assignee: 中兴通讯股份有限公司
Priority date: 2021-11-24
Filing date: 2022-09-05
Publication date: 2023-06-01
Also published as: CN116166722A

Abstract

Provided in the embodiments of the present disclosure are a method and apparatus for asynchronously processing tasks, and a storage medium and an electronic apparatus. The method comprises: stripping a starting state and a waiting state of a data ETL task from a running state, so as to obtain a processed running state; transforming the data ETL task into data ETL tasks a and data ETL tasks b; and controlling the data ETL tasks a which are in the starting state and the waiting state to be executed on a slave ETL server, and controlling the data ETL tasks b which are in the processed running state to be asynchronously executed on a master ETL server, wherein the tasks executed on the slave ETL server are not limited by a threshold value of the number of concurrent running tasks, and the tasks executed on the master ETL server are limited by the threshold value of the number of concurrent running tasks. The problems in the related art of the efficiency of processing distributed data ETL tasks being low and a resource management policy being inflexible and irrational can be solved, thereby improving the processing capability of the data ETL tasks.

Description

Task asynchronous processing method, device, storage medium and electronic device

Cross References to Related Applications

This disclosure is based on the Chinese patent application CN202111407069.2 filed on November 24, 2021 with the title of "task asynchronous processing method, device, storage medium and electronic device", and claims the priority of this patent application, which is incorporated by reference The disclosed content is incorporated in this disclosure in its entirety.

technical field

Embodiments of the present disclosure relate to the communication field, and in particular, relate to a task asynchronous processing method, device, storage medium, and electronic device.

Background technique

Data Extract-Transform-Load (ETL) task processing is the core functional module of the shared data exchange platform, which provides data support services for upper-level government and enterprise applications. In the original technical solution, the data ETL task processing system is also based on the enterprise service bus technology and is distributed, but it only distinguishes the execution status of the task from the arrangement status, the ready status, the running status, and the ending status, among which the running status It is an indivisible state unit, and the resource management strategy of the ETL task in the data ETL task processing system in the data ETL task processing system is consistent: if a data ETL task is scheduled and executed as expected at a certain moment, and The scheduling module judges that the parameter of the number of tasks running concurrently has reached the set threshold, then the task can only be submitted to the execution engine module by the scheduling module after the resources are released after other tasks are executed. However, although the data ETL task being executed by the execution engine module of the system is in the running state, there are actually multiple processing steps in this state due to business characteristics, which is equivalent to multiple sub-states. The data ETL tasks in different sub-states in the running state have different characteristics on the consumption of server resources. The early sub-states such as the waiting state do not consume a lot of server resources but take more time. The later sub-states such as the one after stripping other sub-states The running state consumes a lot of computing resources such as memory and CPU. Therefore, in the original technical solution, due to the unreasonable division of the execution status of the data ETL task and the corresponding execution strategy, the computing resources of the system are not fully and rationally utilized, resulting in low processing efficiency.

Aiming at the low efficiency of distributed data ETL task processing and one-size-fits-all and unreasonable resource management strategies in related technologies, no solution has been proposed yet.

Contents of the invention

Embodiments of the present disclosure provide a task asynchronous processing method, device, storage medium, and electronic device, so as to at least solve the problems of low efficiency of distributed data ETL task processing and one-size-fits-all and unreasonable resource management strategies in the related art.

According to an embodiment of the present disclosure, a task asynchronous processing method is provided, including:

The start state and waiting state of the data ETL task are separated from the running state to obtain the processed running state;

Converting the data ETL task into a data ETL task a and a data ETL task b, wherein the data ETL task a corresponds to the data ETL task b one by one;

Controlling the data ETL task a in the startup state and the waiting state to be executed on the slave ETL server, and controlling the data ETL task b in the running state after the processing to be executed asynchronously on the master ETL server, wherein The tasks executed on the slave ETL server are not limited by the threshold of the concurrent number of tasks running, and the tasks executed on the master ETL server are limited by the threshold of the number of concurrent tasks running.

According to another embodiment of the present disclosure, there is also provided an apparatus for task asynchronous processing, including:

The stripping module is configured to strip the start state and the waiting state of the data ETL task from the running state to obtain the processed running state;

A conversion module configured to convert the data ETL task into a data ETL task a and a data ETL task b, wherein the data ETL task a corresponds to the data ETL task b one by one;

The control module is configured to control the data ETL task a in the starting state and the waiting state to be executed on the slave ETL server, and control the data ETL task b in the running state after the processing to be asynchronous back and forth on the master ETL server Executing, wherein, the tasks executed on the slave ETL server are not limited by the threshold of the number of concurrent tasks running, and the tasks executed on the master ETL server are limited by the threshold of the number of concurrent tasks running on the master ETL server.

According to yet another embodiment of the present disclosure, there is also provided a computer-readable storage medium, where a computer program is stored in the storage medium, wherein the computer program is set to execute any one of the above method embodiments when running in the steps.

According to yet another embodiment of the present disclosure, there is also provided an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to perform any of the above Steps in the method examples.

Description of drawings

FIG. 1 is a block diagram of a hardware structure of a mobile terminal of a task asynchronous processing method according to an embodiment of the present disclosure;

FIG. 2 is a flowchart of a task asynchronous processing method according to an embodiment of the present disclosure;

3 is a schematic diagram of an execution state of a data ETL task according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of node composition of a distributed data ETL processing system according to an embodiment of the present disclosure;

Fig. 5 is a block diagram of an apparatus for task asynchronous processing according to an embodiment of the present disclosure.

Detailed ways

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings and in combination with the embodiments.

It should be noted that the terms "first" and "second" in the specification and claims of the present disclosure and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence.

The method embodiments provided in the embodiments of the present disclosure may be executed in mobile terminals, computer terminals or similar computing devices. Taking running on a mobile terminal as an example, FIG. 1 is a block diagram of the hardware structure of the mobile terminal according to the task asynchronous processing method of the embodiment of the present disclosure. As shown in FIG. 1 , the mobile terminal may include one or more (only shown in FIG. 1 1) Processor 102 (processor 102 may include but not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned mobile terminal may also include a communication function The transmission device 106 and the input and output device 108. Those skilled in the art can understand that the structure shown in FIG. 1 is only for illustration, and it does not limit the structure of the above mobile terminal. For example, the mobile terminal may also include more or fewer components than those shown in FIG. 1 , or have a different configuration from that shown in FIG. 1 .

The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the task asynchronous processing method in the embodiment of the present disclosure, and the processor 102 executes the computer program stored in the memory 104 by running the computer program Various functional applications and service chain address pool slicing processing realize the above-mentioned method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory that is remotely located relative to the processor 102, and these remote memories may be connected to the mobile terminal through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 106 is used to receive or transmit data via a network. The specific example of the above network may include a wireless network provided by the communication provider of the mobile terminal. In one example, the transmission device 106 includes a network interface controller (NIC for short), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet in a wireless manner.

In this embodiment, a task asynchronous processing method running on the above-mentioned mobile terminal or network architecture is provided. FIG. 2 is a flow chart of the task asynchronous processing method according to an embodiment of the present disclosure. As shown in FIG. 2 , the process includes at least But not limited to the following steps:

Step S202, stripping the start state and waiting state of the data ETL task from the running state to obtain the processed running state;

Fig. 3 is a schematic diagram of the execution state of a data ETL task according to an embodiment of the present disclosure. As shown in Fig. 3, the execution state in the related art includes an orchestration state, a ready state, a running state and an end state, as shown in the left process in Fig. 3, That is, the startup state and the waiting state belong to a part of the running state in related technologies. In this embodiment, the starting state and the waiting state are separated from the running state, which is equivalent to re-dividing the execution state. Redefine the execution state of the data ETL task, and improve the execution strategy and resource management strategy according to the re-division of the state. The execution state of the redefined data ETL task includes orchestration state, ready state, start state, waiting state, running state, and end state, as shown in the flow on the right side of Figure 3.

Step S204, converting the data ETL task into data ETL task a and data ETL task b, wherein the data ETL task a corresponds to the data ETL task b one by one, and optionally, the data ETL task can be associated by sharing the same task ID a and data ETL task b.

Step S206, control the data ETL task a in the startup state and the waiting state to be executed on the slave ETL server, and control the processed data ETL task b in the running state to be executed asynchronously on the master ETL server, wherein, in the slave ETL server The tasks executed on the server are not limited by the threshold of the number of tasks running concurrently, and the tasks executed on the main ETL server are limited by the threshold of the number of tasks running concurrently.

The task exception handling in the embodiments of the present disclosure can be applied to a distributed data ETL processing system. The system includes a master ETL server and multiple slave ETL servers. Since tasks in the startup state and waiting state occupy resources for a long time, the startup state and Tasks in the waiting state are executed in the slave ETL server, and the execution of tasks in the slave server will not be limited by the concurrent number threshold of the main ETL server's tasks, so that during the concurrent execution of a large number of data ETL tasks, the computing resources are reduced. invalid occupancy.

Through the above steps S202 to S206, the problems of low processing efficiency of distributed data ETL tasks and one-size-fits-all and unreasonable resource management strategies in related technologies can be solved. Since tasks in the startup state and waiting state occupy resources for a long time, it is difficult to control the startup state and waiting state. The data ETL task a in the state is executed on the slave ETL server. The execution process does not need to limit computing resources. The data ETL task b is executed on the master ETL server. The execution process needs to limit computing resources, which can greatly improve server resources. In the process of concurrent execution of a large number of data ETL tasks, it reduces the invalid occupation of computing resources and improves the processing capacity of data ETL tasks.

In this embodiment, the above step S206 may specifically include: controlling the execution of the data ETL task a on the slave ETL server, and after the execution of the data ETL task a, controlling the execution of the corresponding data ETL task b on the master ETL server, configuring the data The task information of the ETL task, the task information at least includes task priority information, task running node information, data collection information source, data processing flow, data storage information; the task information of the data ETL task is compressed and packaged into task files A and Task file B; transfer task file A to the slave ETL server, and transfer task file B to the master ETL server, correspondingly, notify the slave ETL server corresponding to task file A to run the data ETL task a corresponding to task file A; in the ETL After the task a runs, notify the main ETL server corresponding to the task file B to run the data ETL task b corresponding to the task file B, and control the data ETL task b to enter the end state after the operation is completed, specifically, the running information of the statistical data ETL task; Determine whether exception handling is required according to the running information, and if the determination result is yes, execute exception handling; calculate the next execution time; release computing resources to prepare for the execution of the next data ETL task.

The slave ETL server corresponding to the task file A in this embodiment is used to run the following operations: data query, data persistence, compression and packaging, and notification nodes; the main ETL server corresponding to the above-mentioned task file B is used to run the following operations: data transmission , data decompression, data conversion, and data loading to the destination.

Fig. 4 is a schematic diagram of the node composition of the distributed data ETL processing system according to an embodiment of the present disclosure. As shown in Fig. 4, the change of the execution strategy of the corresponding scheduling module after re-dividing according to the state is: the execution control of the data ETL task requires According to the task file, there is a one-to-one correspondence between the data ETL task and the task file. The original data ETL task and its corresponding task file are improved into two tasks (data ETL task a and data ETL task b) and their corresponding task file A and task file B respectively. The above changes are changes in execution logic, but from the perspective of user perception, this is still a complete ETL task being executed: the execution process of executing logical task file A and data ETL task a (processing steps: data query , data persistence, compression and packaging), the user perceives that the data ETL task is in the start state and waiting state; the execution process of the logical task file B and data ETL task b (processing steps: data transmission, data decompression, Data conversion, data loading to the destination), the user perceives that the data ETL task is running. Data ETL task a and data ETL task b correspond one-to-one, and are executed asynchronously. The execution control process relies on the system service bus technology to complete the communication.

The change in the resource management strategy of the scheduling module is: for the data ETL task after the execution state is redefined, when its execution state is in the start state or waiting state, the scheduling module will not include it in the configuration parameter of the number of concurrent tasks running This expression is equivalent to the fact that data ETL task a is not limited by this parameter and data ETL task b is limited by this parameter.

The above improvement process is formed based on the execution characteristics of the data ETL task: the logical data ETL task a is executed on the execution module of the slave ETL server, and the logical data ETL task b is executed on the execution module of the master ETL server. Since there is a one-to-many correspondence between the master ETL server and the slave servers, the number of slave ETL servers is large and the business load is balanced, so the execution process of the data ETL task a does not need to limit the processing of computing resources, and the data The execution process of ETL task b needs to be processed with limited computing resources; therefore, after improvement, the utilization rate of server resources can be greatly improved and costs can be saved. Since the execution process of the data ETL task a includes data query, data persistence, and compression and packaging operations, these time-consuming operations are stripped out of the links sensitive to computing resources after the improvement, so during the concurrent execution of a large number of data ETL tasks, the need for Ineffective occupancy of computing resources improves data ETL task processing capabilities. Data ETL task a and data ETL task b correspond one-to-one, and are executed asynchronously before and after. The execution control process relies on the system service bus technology to complete the communication, so its process control has the characteristics of automation, which increases the flexibility of system scheduling management, and the system and ETL operation Human-computer friendly interaction of maintenance personnel.

Enter the orchestration state after the orchestration task operation: relevant personnel design and configure the relevant information of the ETL task through the distribution module and the visual orchestration module. The relevant information of the ETL task includes, but is not limited to: configuration task priority information, configuration task running node information, configuration data collection information source, design data processing flow, and configuration data storage information.

Enter the ready state after saving and synchronizing the task: the distribution module of the main ETL server compresses and packages the relevant information of the ETL task in the orchestration state into two ETL task files, task file A and task file B respectively, and the two task files share the same A task ID, the execution process has a contextual relationship, and the compressed and packaged two ETL task files are transmitted to the corresponding ETL server using the SFTP protocol.

After starting the task, enter the starting state of the figure: the task file A is submitted to the execution engine module by the scheduling module on the corresponding slave ETL server, and then decompressed, loaded, verified, parsed, and run respectively. The main logic of its operation is to use the remote call technology Web The asynchronous method of Service notifies the corresponding slave ETL server, and the data ETL task a represented by task file A can end after the notification is successfully sent. It should be noted that the data ETL task can only be executed after the task file is decompressed, loaded, verified, and parsed by the execution engine.

The data ETL task a represented by task file A enters the waiting state after running: task file A performs three processes of data query, data persistence, and compression and packaging after the corresponding slave ETL server receives the notification. The above process Control relies on the Enterprise Service Bus technology ESB. After the data completes the three processes of data query, data persistence, and compression and packaging in sequence, the system service bus technology ESB will notify the task file B to be in the corresponding main ETL server, and the notification message includes the resource path SFTP of the data, and then enter the operation Status: Task file B is submitted to the execution engine module by the scheduling module on the corresponding main ETL server after the corresponding main ETL server receives the notification from the enterprise service bus technology ESB, and then decompresses, loads, and verifies respectively , analysis, and operation. The main logic of its operation is data transmission, data decompression, data conversion, and data loading to the destination. It should be noted that the data ETL task can only be executed after the task file is decompressed, loaded, verified, and parsed by the execution engine.

The data ETL task b represented by the task file B enters the end state after running: the process status monitoring module checks the verification result, notifies the log module to count the relevant running information of the ETL task, notifies the scheduling module whether exception handling is required, calculates the next execution time, and releases associated computing resources.

The embodiment of the present disclosure divides the execution state of the above-mentioned ETL task at a fine-grained level and improves the resource management strategy of the scheduling module, and separates the link of the ETL task's running state, the start state and the waiting state, which are more waiting for a long time. Tasks in this state are not limited by the threshold of the concurrent number of tasks managed by the scheduling module; only other operations that consume more memory resources and computing resources, such as data transmission, data decompression, data conversion, and data loading to the destination, are counted. Into the running state, unified into the control and management of the task running concurrent number threshold of the scheduling module. The ETL task asynchronous processing system reduces the invalid occupation of computing resources by data waiting behavior, and improves the data ETL task processing capability. The switching between states depends on the enterprise service bus technology, and the process control has the characteristics of automation. The execution process of the original task is actually split by two successively running data ETL tasks, which increases the flexibility of system scheduling management. Through the visual display of the process status monitoring module, the interaction between the system and ETL operation and maintenance personnel is human-computer friendly.

In this embodiment, the distributed data ETL processing system includes a master ETL server and a slave ETL server. The slave ETL server and the data source storage device are in the same local area network, and the main ETL server often cannot directly obtain the desired data from the data source storage device in the local area network of the commission office, so the slave ETL server acts as the agent of the main ETL server As a horizontal expansion of the main ETL server, it has the same execution engine module and realizes the ability of distributed data processing. The control interaction between the master ETL server and the slave ETL server is through the remote call technology Web Service, and the data interaction is through the SFTP communication protocol.

According to another embodiment of the present disclosure, a task asynchronous processing device is also provided. FIG. 5 is a block diagram of a task asynchronous processing device according to an embodiment of the present disclosure. As shown in FIG. 5 , it includes:

The stripping module 52 is configured to strip the start state and the waiting state of the data ETL task from the running state to obtain the processed running state;

The conversion module 54 is configured to convert the data ETL task into a data ETL task a and a data ETL task b, wherein the data ETL task a corresponds to the data ETL task b;

The control module 56 is configured to control the data ETL task a in the startup state and the waiting state to be executed on the slave ETL server, and control the processed data ETL task b in the running state to be executed asynchronously on the master ETL server, wherein, in The tasks executed on the slave ETL server are not limited by the threshold of the number of tasks running concurrently, and the tasks executed on the master ETL server are limited by the threshold of the number of concurrent tasks running on the master ETL server.

In an exemplary embodiment, the control module 56 is also configured to:

The control data ETL task a is executed on the slave ETL server, and after the execution of the data ETL task a is completed, the corresponding data ETL task b is controlled to be executed on the master ETL server.

In an exemplary embodiment, the above-mentioned device also includes:

The configuration module is configured to configure the task information of the data ETL task, wherein the task information at least includes task priority information, node information of task operation, data collection information source, data processing flow, and data storage information;

The compression and packaging module is configured to compress and package the task information of the data ETL task into task file A and task file B, the task file A corresponds to the data ETL task a one by one, and the task file B corresponds to the data ETL task b one by one correspond;

The transfer module is configured to transfer task file A to the slave ETL server, and transfer task file B to the master ETL server.

In an exemplary embodiment, the control module 56 includes:

The first notification submodule is configured to notify the corresponding data ETL task a of task file A from the ETL server running task file A;

The second notification submodule is configured to notify the corresponding main ETL server of the task file B to run the data ETL task b corresponding to the task file B after the ETL task a runs;

The control sub-module is set to enter the end state after the control data ETL task b finishes running.

In an exemplary embodiment, the slave ETL server corresponding to the above-mentioned task file A is used to run the following operations: data query, data persistence, compression and packaging, and notification nodes;

The main ETL server corresponding to the above task file B is used to run the following operations: data transmission, data decompression, data conversion, and data loading to the destination.

In an exemplary embodiment, the above-mentioned control submodule is also set to:

Statistical data ETL task running information;

Determine whether exception handling is required according to the running information, and execute exception handling if the determination result is yes;

Calculate the next execution time;

Free up computing resources.

Embodiments of the present disclosure also provide a computer-readable storage medium, in which a computer program is stored, wherein the computer program is set to execute the steps in any one of the above method embodiments when running.

In an exemplary embodiment, the above-mentioned computer-readable storage medium may include but not limited to: U disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM) , mobile hard disk, magnetic disk or optical disk and other media that can store computer programs.

Embodiments of the present disclosure also provide an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

In an exemplary embodiment, the electronic device may further include a transmission device and an input and output device, wherein the transmission device is connected to the processor, and the input and output device is connected to the processor.

For specific examples in this embodiment, reference may be made to the examples described in the foregoing embodiments and exemplary implementation manners, and details will not be repeated here in this embodiment.

Obviously, those skilled in the art should understand that each module or each step of the above-mentioned disclosure can be realized by a general-purpose computing device, and they can be concentrated on a single computing device, or distributed in a network composed of multiple computing devices In fact, they can be implemented in program code executable by a computing device, and thus, they can be stored in a storage device to be executed by a computing device, and in some cases, can be executed in an order different from that shown here. Or described steps, or they are fabricated into individual integrated circuit modules, or multiple modules or steps among them are fabricated into a single integrated circuit module for implementation. As such, the present disclosure is not limited to any specific combination of hardware and software.

The above descriptions are only preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

A task asynchronous processing method, comprising:

The start state and waiting state of the data ETL task are separated from the running state to obtain the processed running state;

Converting the data ETL task into a data ETL task a and a data ETL task b, wherein the data ETL task a corresponds to the data ETL task b one by one;

Controlling the data ETL task a in the startup state and the waiting state to be executed on the slave ETL server, and controlling the data ETL task b in the running state after the processing to be executed asynchronously on the master ETL server, wherein The tasks executed on the slave ETL server are not limited by the threshold of the concurrent number of tasks running, and the tasks executed on the master ETL server are limited by the threshold of the number of concurrent tasks running.
The method according to claim 1, wherein the data ETL task a under the control of the starting state and the waiting state is executed on the slave ETL server, and controls the data ETL task a under the running state after the processing b is executed asynchronously back and forth on the main ETL server, including:

The data ETL task a is controlled to be executed on the slave ETL server, and after the execution of the data ETL task a is completed, the corresponding data ETL task b is controlled to be executed on the master ETL server.
The method according to claim 2, wherein, before converting the data ETL task into data ETL task a and data ETL task b sharing the same task ID, the method further comprises:

Configure the task information of the data ETL task, wherein the task information at least includes task priority information, node information for task operation, data collection information source, data processing flow, and data storage information;

The task information of the data ETL task is compressed and packaged into a task file A and a task file B, the task file A corresponds to the data ETL task a one by one, and the task file B corresponds to the data ETL task b one by one correspond;

The task file A is transferred to the slave ETL server, and the task file B is transferred to the master ETL server.
The method according to claim 3, wherein the data ETL task a is controlled to be executed on the slave ETL server, and after the execution of the data ETL task a is completed, the data ETL task b is controlled to be executed on the master ETL server. Execution on the server includes:

Notifying the slave ETL server corresponding to the task file A to run the data ETL task a corresponding to the task file A;

After the ETL task a finishes running, notify the main ETL server corresponding to the task file B to run the data ETL task b corresponding to the task file B;

Control the data ETL task b to enter the end state after running.
The method according to claim 4, wherein,

The slave ETL server corresponding to the task file A is used to run the following operations: data query, data persistence, compression and packaging, and notification nodes;

The main ETL server corresponding to the task file B is used to run the following operations: data transmission, data decompression, data conversion, and data loading to the destination.
The method according to claim 4, wherein controlling the data ETL task b to enter the end state after running comprises:

Statize the running information of the data ETL task;

Determine whether exception handling is required according to the running information, and execute exception handling if the determination result is yes;

Calculate the next execution time;

Free up computing resources.
A task asynchronous processing device, comprising:

The stripping module is configured to strip the start state and the waiting state of the data ETL task from the running state to obtain the processed running state;

A conversion module configured to convert the data ETL task into a data ETL task a and a data ETL task b, wherein the data ETL task a corresponds to the data ETL task b one by one;

The control module is configured to control the data ETL task a in the starting state and the waiting state to be executed on the slave ETL server, and control the data ETL task b in the running state after the processing to be asynchronous back and forth on the master ETL server Executing, wherein, the tasks executed on the slave ETL server are not limited by the threshold of the number of concurrent tasks running, and the tasks executed on the master ETL server are limited by the threshold of the number of concurrent tasks running on the master ETL server.
The device according to claim 7, wherein the control module is further configured to:

The data ETL task a is controlled to be executed on the slave ETL server, and after the execution of the data ETL task a is completed, the data ETL task b is controlled to be executed on the master ETL server.
A computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to perform the method described in any one of claims 1 to 6 when running.
An electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to perform the method described in any one of claims 1 to 6.