CN114760312A - Distributed task coordination method, device, equipment and medium - Google Patents
Distributed task coordination method, device, equipment and medium Download PDFInfo
- Publication number
- CN114760312A CN114760312A CN202210452053.1A CN202210452053A CN114760312A CN 114760312 A CN114760312 A CN 114760312A CN 202210452053 A CN202210452053 A CN 202210452053A CN 114760312 A CN114760312 A CN 114760312A
- Authority
- CN
- China
- Prior art keywords
- distributed
- task
- tasks
- remote
- framework
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 230000004044 response Effects 0.000 claims abstract description 40
- 230000008569 process Effects 0.000 claims abstract description 25
- 238000005516 engineering process Methods 0.000 claims abstract description 19
- 238000004590 computer program Methods 0.000 claims description 11
- 230000001360 synchronised effect Effects 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 6
- 230000000903 blocking effect Effects 0.000 description 3
- 230000000875 corresponding effect Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000005538 encapsulation Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a distributed task coordination method, a device, equipment and a medium, wherein the method comprises the following steps: when a distributed task issued by a front end is acquired, the distributed task is asynchronously initialized internally, and the initialized distributed task is issued to a waiting queue by a notification issuing frame to wait for response. The cluster management component then asynchronously dispatches the distributed tasks in the wait queue to the plurality of coroutines by notifying the issuing framework. And finally, synchronously acquiring responses fed back by the connected equipment nodes through a plurality of coroutines so as to respond to the distributed tasks. The remote process calling technology adopts a 'request-response' mode, the mode has a notification protocol of 2-way handshake, whether the downstream service can successfully receive the related information or not can be accurately sensed, response feedback is obtained, and therefore the actual requirement of high-throughput distributed task coordination is met.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a distributed task coordination method, apparatus, device, and medium.
Background
For a distributed task coordination scenario, a distributed notification coordination system of distributed application program coordination service software (zookeeper) is often adopted in the existing scheme. The system is generally strongly dependent on zookeeper's notification mechanism in implementation, which may function well as a broadcast message. Referring to fig. 1, when a sub-process issues a message in the form of writing a data, if other process modules have a WATCH (walk) behavior on the written data, a notification of data change issued by the zookeeper is received, and a callback function defined in advance is executed to execute relevant logic. That is, the zookeeper realizes the issue and notification of events through an abstract observation Interface (watch Application Programming Interface), and executes a customized callback function.
However, the watch mechanism has the following disadvantages: the watchdog mechanism is disposable, and the zookeeper does not care about whether the message is acquired by the receiving end and does corresponding action. This requires the sub-process to wait for the same resource again after obtaining an event notification, so that a permanent wait behavior can be realized. It is also because of these disadvantages that the practical requirements for high-throughput distributed task coordination cannot be met.
Disclosure of Invention
Based on this, it is necessary to provide a distributed task coordination method, apparatus, device and medium to solve the problem that high-throughput distributed task coordination cannot be satisfied.
A distributed task coordination method is applied to a distributed database cluster containing a plurality of equipment nodes, and comprises the following steps:
when the cluster management component acquires the distributed tasks issued by the front end, the cluster management component asynchronously performs internal initialization on the distributed tasks, and issues the initialized distributed tasks to a waiting queue through a notification issuing frame to wait for response; the notification issuing framework is used for establishing a basic framework based on a remote process calling technology;
and the cluster management component asynchronously and underground transmits the distributed tasks in the waiting queue to a plurality of coroutines through the notification issuing framework, and synchronously acquires responses fed back by the connected equipment nodes through the plurality of coroutines so as to respond to the distributed tasks.
In one embodiment, the method further comprises:
caching internal events associated with the distributed task.
In one embodiment, the method further comprises:
And expanding the equipment node through a domain name resolution protocol or a load balancing strategy based on the notification issuing framework.
In one embodiment, before asynchronously internally initializing the distributed task, the method further includes:
and synchronously carrying out validity check on the logic of the distributed tasks, and sending a response to the front end based on a check result.
In one embodiment, the method further comprises:
generating a corresponding request entity when a parameter transmission request is acquired;
the requesting entity is executed through a preparation phase of serial synchronous execution, an execution phase of synchronous waiting response and a global-oriented ending phase.
In one embodiment, the method further comprises:
acquiring a preset channel for each equipment node, adding a channel matched with a remote task and executing the channel when the remote task is realized based on a single-equipment remote procedure calling technology, and initializing parameters related to the remote task;
when the remote task is realized based on the remote process calling technology of the multiple devices, a channel matched with the remote task is added for multiple times and executed, and parameters related to the remote task are initialized.
In one embodiment, the method further comprises:
the processing process of the request entity and the processing process of the remote task are arranged to execute the target tasks related to a plurality of stages or related to a plurality of node devices to execute related instructions simultaneously.
A distributed task coordination device applied to a distributed database cluster containing a plurality of equipment nodes comprises the following steps:
the waiting response module is used for asynchronously initializing the distributed tasks when the cluster management component acquires the distributed tasks issued by the front end, and issuing the initialized distributed tasks to a waiting queue through a notification issuing framework to wait for response; the notification issuing framework is used for establishing a basic framework based on a remote process calling technology;
and the task coordination module is used for asynchronously and underground transmitting the distributed tasks in the waiting queue to a plurality of coroutines by the cluster management component through the notification issuing frame, and synchronously acquiring responses fed back by the connected equipment nodes through the plurality of coroutines so as to respond to the distributed tasks.
A computer-readable storage medium, in which a computer program is stored which, when executed by a processor, causes the processor to carry out the steps of the above-mentioned distributed task orchestration method.
A distributed task orchestration device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the distributed task orchestration method.
The invention provides a distributed task coordination method, a distributed task coordination device, distributed task coordination equipment and a distributed task coordination medium, wherein the coordination process is mainly based on a notification issuing framework constructed by a remote process calling technology. And then the cluster management component asynchronously and underground sends the distributed tasks in the waiting queue to a plurality of coroutines by informing the sending framework, so that the distributed tasks can be sent in time, and the subsequent service blockage of the distributed database cluster caused by the fact that a certain distributed task is currently executed for a long time can be avoided. And finally, synchronously acquiring responses fed back by the connected equipment nodes through a plurality of coroutines so as to respond to the distributed tasks. Therefore, the risk of single handshake can be avoided, the remote process calling technology adopts a request-response mode, the mode has a notification protocol of 2-time handshake, whether the downstream service can successfully receive the related information can be accurately sensed, response feedback can be obtained, and therefore the actual requirement of high-throughput distributed task coordination is met.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Wherein:
FIG. 1 is a flow diagram that illustrates a distributed task orchestration method according to an embodiment;
FIG. 2 is a flow diagram of a distributed task coordination method in one embodiment;
FIG. 3 is a diagram illustrating an exemplary architecture of a distributed task orchestration device according to one embodiment;
FIG. 4 is a block diagram of a distributed task orchestration device according to an embodiment.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
For convenience of explanation, the following terms of department proper names are expressed in english and abbreviations, and are now explained: RPC: remote procedure call; DBMS, distributed database management system; cluster: a distributed database cluster; ClusterManager: a cluster management component; node: a device node; NodeManager: a Node management component; POSTquery: a parameter transmission request; REQUEST entity: a requesting entity; channel: a channel; REMOTATASK: a remote task; HTTP: a hypertext transfer protocol; coroutine: coordinating the program.
As shown in fig. 1, fig. 1 is a flowchart illustrating a distributed task coordination method in an embodiment, and is applied to a Cluster including a plurality of nodes.
The DBMS is used for uniformly managing and controlling the database, and originally has the appeal that a plurality of physical devices work together, so that an independent component is required for managing the whole Cluster. Its core tasks include, but are not limited to: maintaining cluster state, managing distributed tasks, issuing and executing multi-node cooperative tasks, inquiring and displaying cluster state and the like.
In this embodiment, the ClusterManager serves as a cluster management component of the DBMS and undertakes the above core tasks. ClusterManager is an independent functional module and can be operated on any physical device in the cluster without state.
The steps provided by the distributed task coordination method in this embodiment include:
and 102, when the cluster management component acquires the distributed task issued by the front end, asynchronously performing internal initialization on the distributed task, and issuing the initialized distributed task to a waiting queue through a notification issuing frame to wait for response.
In the stage, the receiving of the task, the internal initialization of the task and the issuing of the task are executed asynchronously, so that the blocking is avoided.
Specifically, as shown in fig. 2, firstly, the validity of the logic of the distributed task is synchronously checked, that is, the RequestCheck and Get RequestID are performed, and then an HTTP response is sent to the front end based on the check result. When the verification result is legal, the response is used for feeding back the distributed tasks which are currently received to the front end and are being processed. Otherwise, no processing is performed. Since the logic of validity check is simple, there is no long-time blocking even if synchronous waiting is adopted.
In this embodiment, the notification issuing framework constructs a basic framework based on the RPC technology. RPC refers to a process on computer A that calls another process on computer B, where the calling process on A is suspended and the called process on B begins execution, and when the value is returned to A, the A process continues execution. The caller can pass information to the callee by using parameters and then get information through the returned results. This process is transparent to the developer. Therefore, the risk of single handshake of zookeeper can be avoided, and a notification protocol of 2-way handshake such as a request-response mode of an RPC technology can accurately sense whether the downstream service can successfully receive related information and obtain response feedback, so that the actual requirement of high-throughput distributed task coordination is met.
The distributed task is then internally initialized asynchronously, including creating an object of an external class and then using the object to create an internal class object. Since each inner class can independently inherit from one implementation (of an interface), there is no impact on the inner class whether or not the outer class has inherited a certain implementation (of an interface), thus better compatibility with distributed tasks. And finally, issuing the initialized distributed tasks to a waiting queue through a notification issuing framework so as to wait for response.
And 104, the cluster management component asynchronously and underground transmits the distributed tasks in the waiting queue to a plurality of coroutines through the notification issuing framework, and synchronously acquires responses fed back by the connected equipment nodes through the plurality of coroutines so as to respond to the distributed tasks.
In this embodiment, as shown in fig. 2, the ClusterManager issues the distributed tasks in the waiting queue to multiple coroutines asynchronously by notifying the issuing framework. The issued coroutines are currently idle coroutines, the coroutines are completely controlled by programs, and one coroutine is connected with one Node. The responses fed back by the connected nodes can be synchronously acquired through the plurality of coroutines. At this stage, although the rpc of the individual NodeManagers are still synchronous and like calls, the overall process flow is asynchronous, so the entire HTTP service is not blocked.
As can be seen, the task processing of the ClusterManager is a pure asynchronous framework facing the front end and the back end. The purpose of this implementation is to ensure that the ClusterManager can continuously provide Cluster management service to the outside, and subsequent service blocking of the Cluster due to a certain distributed task being executed for a long time currently will not occur.
Further, considering that in the notification mechanism of zookeeper, if the data of the node to be held is updated frequently in a short time, based on the disadvantages mentioned in the background, the loss of the message, that is, the loss of the hold event, may also occur in theory.
To address this issue, in one particular embodiment, internal events associated with the distributed tasks are also cached. For example, memory events must be cached internally before they are distributed. Thus, the event is not lost, and high throughput is ensured.
Further, in a zookeeper notification mechanism, the zookeeper service is a master-standby topology, the number of watchers capable of bearing is limited, potential performance bottleneck risks exist, and the lateral expansion capability is limited.
To address such a problem, in a specific embodiment, the Node is further expanded horizontally by a domain name resolution protocol or a load balancing policy based on a notification issuing framework, and a uniform external service capability is formed, so that a potential risk of performance bottleneck is eliminated.
According to the distributed task coordination method, a notification issuing framework constructed mainly based on a remote process calling technology is used, firstly, when the cluster management component acquires the distributed task issued by the front end, the distributed task is asynchronously initialized, so that the distributed task is better compatible, and then the initialized distributed task is issued to a waiting queue through the notification issuing framework to wait for response. And then the cluster management component asynchronously and underground sends the distributed tasks in the waiting queue to a plurality of coroutines by informing the sending framework, so that the distributed tasks can be sent in time, and the subsequent service blockage of the distributed database cluster caused by the fact that a certain distributed task is currently executed for a long time can be avoided. And finally, synchronously acquiring responses fed back by the connected equipment nodes through a plurality of coroutines so as to respond to the distributed tasks. Therefore, the risk of single handshake can be avoided, the remote process calling technology adopts a request-response mode, the mode has a notification protocol of 2-time handshake, whether the downstream service can successfully receive the related information can be accurately sensed, response feedback is obtained, and therefore the actual requirement of high-throughput distributed task coordination is met.
The following describes a distributed task coordination method with reference to different specific application scenarios:
first type of application scenario, REQUEST-API remote traffic handling abstraction:
each time the ClusterManager acquires the POSTquery, a Request entity is generated in the ClusterManager. The entity is used for bearing the functions of executing subtasks, persisting information and the like.
For a Request entity, the flow of processing this Request is fixed, and includes three stages: a SetUp phase SetUp (), an execute phase DealRequest (), and an end phase TearDown ().
When internal initialization is carried out, for different distributed tasks, the ClusterRequest class is required to be inherited, and SetUpImpl/DelalRequest/TearDownImpl methods are filled, and the self-determined logics are called in the task execution flow.
1. Preparation stage SetUp ()
SetUp needs to be lightweight, so this phase is performed serially, single threaded. This is also for the purpose of facilitating the handling of e.g. exclusive tasks. For example, a task that can only exist and be executed globally needs to be executed, and the task can be logically lifted to the SetUp phase.
2. Execute stage DelalRequest ()
This phase executes the main task logic and will be deployed in coroutine, but it will not respond asynchronously at rpc and still be in a synchronous manner. Since synchronization at this stage, etc., may affect the main logic, synchronization may be used to simplify the programming difficulty.
3. Ending stage TearDown ()
This phase does global epilogue work, such as notifying compute nodes, updating metadata clusters, and so on. This phase will also be expanded in coroutine.
Second class of application scenarios, ReMOTETASK-Abstract of APIREMETTASK:
1. single device RPC
Each Node is represented by a Channel in the ClusterManager, and the object can be understood as an encapsulation of a remote hypertext transfer service (http server), so when the single-device-based RPC technology is implemented for REMOTETASK, the preparation work required includes: selecting a Channel matched with REMOTATASK, executing AllSubChannelSuccess (), and initializing parameters related to tasks.
2. Multi-device RPC
Some scenarios require that commands be executed on multiple machines simultaneously, and wait for all devices to succeed or for any one device to fail, at which point an error is declared. Here, corresponding to the operation of the RemoteTask instantiation, when the RemoteTask is implemented by the multi-device based RPC technology, the required preparation work includes: adding channels matched with REMOTATASK for multiple times, executing AllSubChannelSuccess (), and initializing parameters related to tasks. Tags may be used to distinguish between different machines performing tasks if the parameters differ.
The third type of application scenario, MissionRequest object-complex task arrangement and execution:
a complex target task may involve multiple phases, or involve multiple node devices executing simultaneously. There is therefore a need for a function that can implement complex REMOTATASK choreography.
The MissionRequest is inherited from the ClusterRequest, combines the functions of REMOTATASK, and realizes complex target tasks by arranging the processing procedures of the request entities and the REMOTATASK.
For example, in a capacity expansion scene, only one Expand class needs to be implemented, and the class inherits from the MissionRequest, and then SetUpMission ()/DealRequest ()/TearDownImpl ()/arrageremotettask () is implemented, so that the whole capacity expansion task flow can be unified into the existing task frame.
The logic that the arragerremotetask () needs to implement is to instantiate RemoteTask one by one and then put it into a queue as needed. When the execution is actually performed, the execution is taken out from the queue one by one.
As can be seen, for the above application scenarios, a uniform abstract interface is provided. The distributed task coordination framework realized based on the abstract interface shields the specific logic of complex repeated RPC interaction of multiple physical devices, so that the distributed management task of the cross-physical device can be realized only by realizing service logic without paying attention to the complex multi-device interaction logic. The uniform abstract model provides a stable, controllable and observable subtask coordination mechanism for the whole cluster, and simultaneously, the good encapsulation enables the task management logic and the task implementation logic to be well separated, and provides a foundation for the benign expansion of the system.
In one embodiment, as shown in fig. 3, a distributed task coordination device is provided, the device comprising:
a waiting response module 302, configured to asynchronously perform internal initialization on the distributed task when the cluster management component acquires the distributed task issued by the front end, and issue the initialized distributed task to a waiting queue through a notification issuing frame to wait for a response; the notification issuing framework builds a basic framework based on a remote process calling technology;
and the task coordination module 304 is configured to enable the cluster management component to asynchronously and underground send the distributed tasks in the waiting queue to multiple coroutines through the notification issuing frame, and to synchronously acquire responses fed back by the connected device nodes through the multiple coroutines so as to respond to the distributed tasks.
FIG. 4 is a diagram that illustrates an internal structure of a distributed task orchestration device in one embodiment. As shown in FIG. 4, the distributed task orchestration device comprises a processor, a memory, and a network interface, which are connected through a system bus. The memory comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium of the distributed task coordination device stores an operating system and may also store a computer program that, when executed by the processor, causes the processor to implement the distributed task coordination method. The internal memory may also have a computer program stored thereon that, when executed by the processor, causes the processor to perform a distributed task orchestration method. It will be appreciated by those skilled in the art that the configuration shown in fig. 4 is a block diagram of only a portion of the configuration associated with the present application, and does not constitute a limitation on the distributed task orchestration device to which the present application applies, and that a particular distributed task orchestration device may include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.
A distributed task orchestration device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the computer program when executing the computer program: when the cluster management component acquires the distributed tasks issued by the front end, the cluster management component asynchronously initializes the distributed tasks internally and issues the initialized distributed tasks to a waiting queue through a notification issuing framework to wait for response; the cluster management component asynchronously and underground sends the distributed tasks in the waiting queue to a plurality of coroutines through informing the issuing framework, and synchronously obtains responses fed back by the connected equipment nodes through the plurality of coroutines so as to respond to the distributed tasks.
A computer-readable storage medium, storing a computer program which, when executed by a processor, performs the steps of: when the cluster management component acquires the distributed tasks issued by the front end, the cluster management component asynchronously initializes the distributed tasks internally and issues the initialized distributed tasks to a waiting queue through a notification issuing framework to wait for response; the cluster management component asynchronously and underground sends the distributed tasks in the waiting queue to a plurality of coroutines through informing the issuing framework, and synchronously obtains responses fed back by the connected equipment nodes through the plurality of coroutines so as to respond to the distributed tasks.
It should be noted that, the distributed task coordination method, apparatus, device and computer-readable storage medium described above belong to a general inventive concept, and the contents in the embodiments of the distributed task coordination method, apparatus, device and computer-readable storage medium may be mutually applicable.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not to be construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present patent application shall be subject to the appended claims.
Claims (10)
1. A distributed task coordination method is applied to a distributed database cluster containing a plurality of equipment nodes, and comprises the following steps:
when the cluster management component acquires the distributed tasks issued by the front end, the cluster management component asynchronously performs internal initialization on the distributed tasks, and issues the initialized distributed tasks to a waiting queue through a notification issuing frame to wait for response; the notification issuing framework is used for establishing a basic framework based on a remote process calling technology;
And the cluster management component asynchronously and underground sends the distributed tasks in the waiting queue to a plurality of coroutines through the notification issuing framework, and synchronously acquires responses fed back by the connected equipment nodes through the plurality of coroutines so as to respond to the distributed tasks.
2. The method of claim 1, further comprising:
caching internal events associated with the distributed task.
3. The method of claim 1, further comprising:
and expanding the equipment node through a domain name resolution protocol or a load balancing strategy based on the notification issuing framework.
4. The method of claim 1, wherein prior to asynchronously internally initializing the distributed task, further comprising:
and synchronously carrying out validity check on the logic of the distributed tasks, and sending a response to the front end based on a check result.
5. The method of claim 1, further comprising:
generating a corresponding request entity when a parameter transmission request is acquired;
the requesting entity is executed through a preparation phase of serial synchronous execution, an execution phase of synchronous waiting response and a global-oriented ending phase.
6. The method of claim 5, further comprising:
acquiring a preset channel for each equipment node, adding a channel matched with a remote task and executing the channel when the remote task is realized based on a single-equipment remote procedure calling technology, and initializing parameters related to the remote task;
when the remote task is realized based on the remote process calling technology of the multiple devices, a channel matched with the remote task is added for multiple times and executed, and parameters related to the remote task are initialized.
7. The method of claim 6, further comprising:
the processing procedure of the request entity and the processing procedure of the remote task are arranged to execute the target task which involves a plurality of stages or a plurality of node devices to execute related instructions simultaneously.
8. A distributed task coordination apparatus applied to a distributed database cluster including a plurality of device nodes, the apparatus comprising:
the waiting response module is used for asynchronously carrying out internal initialization on the distributed tasks when the cluster management component acquires the distributed tasks issued by the front end, and issuing the initialized distributed tasks to a waiting queue through a notification issuing frame to wait for response; the notification issuing framework is used for establishing a basic framework based on a remote process calling technology;
And the task coordination module is used for asynchronously and underground transmitting the distributed tasks in the waiting queue to a plurality of coroutines by the cluster management component through the notification issuing framework and synchronously acquiring responses fed back by the connected equipment nodes through the plurality of coroutines so as to respond to the distributed tasks.
9. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 7.
10. A distributed task orchestration device comprising a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210452053.1A CN114760312A (en) | 2022-04-27 | 2022-04-27 | Distributed task coordination method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210452053.1A CN114760312A (en) | 2022-04-27 | 2022-04-27 | Distributed task coordination method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114760312A true CN114760312A (en) | 2022-07-15 |
Family
ID=82332838
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210452053.1A Pending CN114760312A (en) | 2022-04-27 | 2022-04-27 | Distributed task coordination method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114760312A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106713226A (en) * | 2015-11-12 | 2017-05-24 | 卓望数码技术（深圳）有限公司 | Remote procedure call processing method used for distributed system and remote procedure call processing system thereof |
| WO2020024469A1 (en) * | 2018-07-31 | 2020-02-06 | 平安科技（深圳）有限公司 | Service processing method, calling management system and computer device |
| CN111385255A (en) * | 2018-12-28 | 2020-07-07 | 北京金山云网络技术有限公司 | Asynchronous call implementation method and device, server and server cluster |
| CN111414266A (en) * | 2020-03-23 | 2020-07-14 | 山东浪潮通软信息科技有限公司 | Synchronous and asynchronous communication method and device for distributed transactions |
| WO2020233077A1 (en) * | 2019-05-21 | 2020-11-26 | 深圳壹账通智能科技有限公司 | System service monitoring method, device, and apparatus, and storage medium |
-
2022
- 2022-04-27 CN CN202210452053.1A patent/CN114760312A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106713226A (en) * | 2015-11-12 | 2017-05-24 | 卓望数码技术（深圳）有限公司 | Remote procedure call processing method used for distributed system and remote procedure call processing system thereof |
| WO2020024469A1 (en) * | 2018-07-31 | 2020-02-06 | 平安科技（深圳）有限公司 | Service processing method, calling management system and computer device |
| CN111385255A (en) * | 2018-12-28 | 2020-07-07 | 北京金山云网络技术有限公司 | Asynchronous call implementation method and device, server and server cluster |
| WO2020233077A1 (en) * | 2019-05-21 | 2020-11-26 | 深圳壹账通智能科技有限公司 | System service monitoring method, device, and apparatus, and storage medium |
| CN111414266A (en) * | 2020-03-23 | 2020-07-14 | 山东浪潮通软信息科技有限公司 | Synchronous and asynchronous communication method and device for distributed transactions |
Non-Patent Citations (2)
| Title |
|---|
| 予亭: "zookeeper学习心得二：同步与异步", Retrieved from the Internet <URL:https://blog.csdn.net/randompeople/article/details/70500136> * |
| 胡喜明: "基于响应式的高性能远程过程调用框架的设计与实现", 中国优秀硕士学位论文全文数据库 (信息科技辑), no. 2022, pages 3 - 5 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021109735A1 (en) | Cross-chain-network-based resource processing method, and device | |
| US9904585B1 (en) | Error handling in executing workflow state machines | |
| CN112035228A (en) | Resource scheduling method and device | |
| CN111026430B (en) | Method and system for flexible node composition on local or distributed computer system | |
| EP3837604B1 (en) | In situ triggered function as a service within a service mesh | |
| CN115328663A (en) | Method, device, equipment and storage medium for scheduling resources based on PaaS platform | |
| US9231995B2 (en) | System and method for providing asynchrony in web services | |
| US10866841B2 (en) | Communication system and method for accessing and deploying temporary microservices on a heterogeneous platform | |
| US11595299B2 (en) | System and method of suppressing inbound payload to an integration flow of an orchestration based application integration | |
| CN112433863A (en) | Micro-service calling method and device, terminal equipment and storage medium | |
| CN111580793A (en) | Method, apparatus, computer device and storage medium for executing service | |
| US20200310828A1 (en) | Method, function manager and arrangement for handling function calls | |
| CN114237628B (en) | Microservice arranging and calling method and device, computer equipment and storage medium | |
| CN110457132B (en) | Method and device for creating functional object and terminal equipment | |
| CN115248692A (en) | Device and method for supporting cloud deployment of multiple deep learning framework models | |
| CN114924858A (en) | Task scheduling method and device, storage medium and electronic equipment | |
| CN114168297A (en) | Method, device, equipment and medium for scheduling collection tasks | |
| CN108664343B (en) | State calling method and device for micro-service | |
| CN111198753A (en) | Task scheduling method and device | |
| US20230283695A1 (en) | Communication Protocol for Knative Eventing's Kafka components | |
| US9323509B2 (en) | Method and system for automated process distribution | |
| CN114760312A (en) | Distributed task coordination method, device, equipment and medium | |
| CN114979286B (en) | Access control method, device, equipment and computer storage medium for container service | |
| CN111338775B (en) | Method and equipment for executing timing task | |
| CN114840221A (en) | Cloud server deployment method, device, system, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |