CN117827402A - Method and device for executing timing task in remote double-activity system - Google Patents

Method and device for executing timing task in remote double-activity system Download PDF

Info

Publication number
CN117827402A
CN117827402A CN202311708831.XA CN202311708831A CN117827402A CN 117827402 A CN117827402 A CN 117827402A CN 202311708831 A CN202311708831 A CN 202311708831A CN 117827402 A CN117827402 A CN 117827402A
Authority
CN
China
Prior art keywords
slave
task
host
service
database
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311708831.XA
Other languages
Chinese (zh)
Inventor
饶梦达
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianyi Cloud Technology Co Ltd
Original Assignee
Tianyi Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianyi Cloud Technology Co Ltd filed Critical Tianyi Cloud Technology Co Ltd
Priority to CN202311708831.XA priority Critical patent/CN117827402A/en
Publication of CN117827402A publication Critical patent/CN117827402A/en
Pending legal-status Critical Current

Links

Landscapes

  • Hardware Redundancy (AREA)

Abstract

The invention belongs to the technical field of ex-situ dual-activity architecture, and particularly relates to a method and a device for executing timing tasks in an ex-situ dual-activity system. According to the invention, different ZKs are adopted as registration centers in different environments, the problem of repeated execution of timing tasks of a different-place double-activity system is solved, the system can be used in two different-place machine rooms, main and standby switching can be carried out along with a database, delay is reduced, independent ZK clusters are used, the problems of network and cluster election brain cracking generated by crossing city ZK clusters are avoided, the number of executed fragments can be adjusted by a slave machine and a host machine according to task read-write types, the execution efficiency is ensured, netty is used as a heartbeat interaction mechanism between the timing tasks, priority scheduling of the timing tasks is realized through a custom fragmentation algorithm, and automatic main and standby switching can be realized under the condition that a fault occurs in a certain place.

Description

Method and device for executing timing task in remote double-activity system
Technical Field
The invention belongs to the technical field of ex-situ dual-activity architecture, and particularly relates to a method and a device for executing timing tasks in an ex-situ dual-activity system.
Background
For enterprises in cloud environment, the importance of constructing a dual-activity data center is self-evident, on one hand, the dual-activity mode of providing business production service can be provided for the outside, and the utilization rate of resources and the working efficiency and performance of the system are greatly improved; on the other hand, the disaster recovery in different places can be realized, and the system can obtain greater value.
Many developing teams are direct deployments of single points when using elastic jobs for timing tasks, which may not be critical for some asynchronous non-core services (e.g., reconciliation, monitoring, etc.), but are extremely prone to problems for the above similar single point deployment of services for some highly available compensation, core data timing modifications (e.g., interest updates for financial scenarios, etc.).
There are a number of limitations to performing timed tasks in ex-situ dual activity systems. Because the dual-active system needs multi-place multi-machine deployment, in general projects, dozens of timing tasks are available, once the multi-machine deployment is performed, the timing tasks can be repeatedly executed, and both fixed ip and staggered time schemes have larger defects.
The present system supports ElasticJob.
Disclosure of Invention
The invention aims to provide a method for executing a timing task in a remote dual-activity system, which can realize priority scheduling of the timing task through a custom slicing algorithm and realize automatic active-standby switching under the condition that a fault occurs in a certain place.
The technical scheme adopted by the invention is as follows:
a method of performing timed tasks in a ex-situ dual activity system, comprising:
acquiring master-slave information of a database, and distinguishing a host from a slave;
the host and the slave establish a channel through Netty Server Channel Initializer to realize the exchange of periodical heartbeat data and the time task fragmentation information, and the host returns an ACK message after receiving the information;
if the slave does not sense the heartbeat of the host in the set time or detects the primary-backup switching of the database in the process of regularly accessing the database, the slave indicates that the service of the timing task of the host is abnormal or the primary-backup switching of the database occurs, and triggers the primary-backup switching of the timing task management module;
acquiring a service for creating an elastic job timing task, and marking the read-write type of the task;
the host acquires a JobInstance list of the service fragment instance from the ZooKeeper;
the slave sends heartbeat data;
and acquiring the service type, and adjusting the fragment duty ratio according to the service type.
In a preferred embodiment, the step of obtaining master-slave information of the database to distinguish the master from the slave includes:
designating connection information of a master database and a slave database in a database configuration file;
initializing the system when starting, and obtaining configuration information of a database by a management module;
according to the obtained configuration information of the database, the management module judges the role of the current node, wherein the role comprises a host and a slave, the host is a master node responsible for executing timing task management, and the slave is a standby node capable of being switched to the host when the master is switched;
if the master-slave switching of the database occurs, the management module synchronously updates master-slave information in time.
In a preferred solution, the step of enabling the master and the slave to establish a channel through Netty Server Channel Initializer to realize the exchange of periodic heartbeat data and the time task fragmentation information, and returning an ACK message after the master receives the information includes:
acquiring Netty Server Channel Initializer;
causing the master and the slave to establish a channel through Netty Server Channel Initializer;
the slave machine periodically sends heartbeat data and timing task segmentation information to the host machine;
after receiving the heartbeat data and the timed task fragmentation information, the host performs task management and scheduling, and returns an ACK message to confirm the reception.
In a preferred solution, the step of enabling the master and the slave to establish a channel through Netty Server Channel Initializer to realize the exchange of periodic heartbeat data and the time task fragmentation information, and after the step of returning an ACK message after receiving the information, the method further includes:
if the slave does not sense the heartbeat of the host in the set time or detects the primary-backup switching of the database in the process of regularly accessing the database, the slave indicates that the service of the timing task of the host is abnormal or the primary-backup switching of the database occurs, and triggers the primary-backup switching of the timing task management module;
if the slave is switched to the state of the host, namely, the slave is converted to a master node capable of executing timing task management, the slave sends a switching message to the host to inform the host that the master and the slave are switched, and the host checks the state of the database after receiving the switching message;
if the database is found to be changed into the slave, the host sends an ACK message for agreeing to switch;
if the database is not found to be changed into the slave database, sending an ACK message for refusing the switching;
if the slave machine can not receive the reply of the host machine for a long time, namely the service of the timing task of the host machine is abnormal and can not be communicated, the slave machine is actively switched to the host machine state;
if the timing task service corresponding to the main database is recovered to be normal, the slave is switched back under a certain condition.
In a preferred embodiment, the step of creating an Elasticjob timing task service and marking a read-write type of a task includes:
acquiring an elastic Job timing task service created by a system;
acquiring an interface of task attribute setting in an elastic Job framework;
marking the elasticJob timing task for acquiring the service according to the interface set by the task attribute in the elasticJob framework, wherein the mark comprises a writing task and a writing task.
In a preferred embodiment, the step of obtaining, by the host, a list of service fragment instances jobsistance from a ZooKeeper includes:
acquiring a ZooKeeper path;
the host is connected to the ZooKeeper;
the host acquires information of the service fragment instance from the ZooKeeper path, and the acquired information of the service fragment instance comprises a list of the service fragment instance.
In a preferred embodiment, the step of sending heartbeat data by the slave includes:
the slave acquires the heartbeat data format and generates a heartbeat data packet
Adding information of a slave on-board service fragment instance JobInstance into a heartbeat data packet;
and sending the generated heartbeat data packet to a host computer by using the established Netty communication channel.
In a preferred embodiment, the step of obtaining the service type and adjusting the fragment duty ratio according to the service type includes:
acquiring a fragmentation manager;
the segmentation manager receives the information of the task and judges the service type of the task, wherein the service type comprises a reading task and a writing task;
and setting the fragmentation duty ratio according to the service type.
In a preferred embodiment, the step of setting the fragmentation duty ratio according to the service type includes:
acquiring a service type;
if the service type is a reading task, the number of fragments and the occupation proportion of the service executed by the host are larger than those of the slave, and the number of fragments and the occupation proportion of the service executed by the host are improved;
if the service type is a writing task, the number of fragments and the occupation ratio of the slave machine executing the service are larger than those of the master machine, and the number of fragments and the occupation ratio of the slave machine executing the service are improved.
And, an apparatus for performing timed tasks in a ex-situ dual activity system, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement a method of performing timed tasks in the ex-situ dual activity system.
The invention has the technical effects that:
the invention adopts different ZKs as registration centers in different environments, solves the problem of repeated execution of timing tasks of a two-active system in different places, can be used in two machine rooms in different places, can be switched between the master and the slave together with a database, reduces delay, uses independent ZK clusters, avoids the problems of network and cluster election brain cracking generated by the cross-city ZK clusters, and can adjust the number of executed fragments according to the task read-write types by a slave and a host, thereby ensuring the execution efficiency, uses Netty as a heartbeat interaction mechanism between the timing tasks, realizes the priority scheduling of the timing tasks through a self-defined fragmentation algorithm, and can realize automatic master-slave switching under the condition of fault occurrence in a certain place.
Drawings
FIG. 1 is a flow chart of a method provided by the present invention;
FIG. 2 is a schematic framework diagram in a timed task architecture of the present invention;
FIG. 3 is a schematic diagram of a timed task manager heartbeat management module of the present invention;
FIG. 4 is a schematic diagram of a timed task manager partition management module of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present invention is not limited to the specific embodiments disclosed below.
Further, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic can be included in at least one implementation of the invention. The appearances of the phrase "in one preferred embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
Further, the present invention will be described in detail with reference to the drawings, which are only examples for convenience of illustration, and should not limit the scope of the present invention.
Referring to fig. 1 to 4, a method for executing a timing task in a ex-situ dual-activity system is provided, which includes:
s1, obtaining master-slave information of a database, and distinguishing a host from a slave;
s2, the host and the slave establish a channel through Netty Server Channel Initializer to realize the exchange of periodic heartbeat data and the time task fragmentation information, and the host returns an ACK message after receiving the information;
if the slave does not sense the heartbeat of the host in the set time or detects the primary-backup switching of the database in the process of regularly accessing the database, the slave indicates that the service of the timing task of the host is abnormal or the primary-backup switching of the database occurs, and triggers the primary-backup switching of the timing task management module;
s3, acquiring a service for creating an elastic job timing task, and marking the read-write type of the task;
s4, the host acquires a JobInstance list of the service fragment instance from the ZooKeeper;
s5, the slave sends heartbeat data;
s6, obtaining the service type, and adjusting the fragment duty ratio according to the service type.
In the above steps S1 to S6, in the system initialization stage, the management module identifies the master database and the slave database by reading the configuration information of the database, so as to distinguish whether the current node is the master or the slave, the Netty framework is used, a communication channel between the master and the slave is established through Netty Server Channel Initializer, so as to realize periodic exchange of heartbeat data and time task fragmentation information, when the slave does not sense the heartbeat of the master or detects the master-slave switching of the database within a specified time, the logic of the master-slave switching is triggered, the slave switches to the state of the master, when the elastic job timing task is created, the read-write type of the task is marked, the information is embedded into the configuration of the task for subsequent task scheduling and fragmentation management, the host acquires a service fragmentation instance btance list from a specified path through connection to the zoojokeeper, including the state of each fragment and related information, the slave machine periodically generates a heartbeat data packet, wherein the heartbeat data packet comprises the information of the current state of the slave machine and the service slicing instance, the heartbeat data packet is sent to the host machine through a Netty communication channel, a slicing manager adjusts the slicing duty ratio of a master node and a slave node according to the service types of tasks so as to meet the execution demands of different service types, the system can ensure that the timing tasks can be flexibly managed in a different-place double-active environment by effectively distinguishing the host machine from the slave machine and adapting to the scenes such as the switching of the master machine and the slave machine, the availability of the system is improved, an efficient communication channel is established to help the real-time synchronization of the system state, the communication delay is reduced, the host machine and the slave machine can effectively cooperate, the instantaneity and the stability of the system are improved, the switching of the master machine and the slave machine is timely triggered to help the system to quickly adapt to the change under the condition that the service of the timing tasks of the host machine is abnormal or the switching of the database occurs, the continuous execution of the timing tasks is ensured, by marking the read-write type of the task, the system can carry out reasonable fragment management and scheduling according to the characteristics of the task, improves the efficiency and performance of task execution, can better carry out task scheduling and fragment management by acquiring real-time service fragment instance information from a ZooKeeper, ensures the efficient execution of the task, periodically sends heartbeat data to help to monitor the state of the slave computer in real time, ensures the capability of timely acquiring the information of the slave computer, ensures the real-time performance of communication, adjusts the fragment occupation ratio according to the service type, ensures that the system is more flexible to adapt to the execution characteristics of different task types, and improves the processing capability of the system on the read-write task.
The method comprises the steps of obtaining master-slave information of a database and distinguishing a host from a slave, and comprises the following steps:
s101, designating connection information of a master database and a slave database in a database configuration file;
s102, initializing a system when starting, and acquiring configuration information of a database by a management module;
s103, judging the role of the current node by the management module according to the acquired configuration information of the database, wherein the role comprises a host and a slave, the host is a master node responsible for executing timing task management, and the slave is a standby node capable of being switched to the host when the master is switched;
if the master-slave switching of the database occurs, the management module synchronously updates master-slave information in time.
In the above steps S101 to S103, in the database configuration file of the system, by designating the connection information of the master database and the slave database, including the database address, the user name, the password, etc., the management module is responsible for performing the initialization operation when the system is started, including obtaining the configuration information of the database, the management module determines the role of the current node according to the obtained database configuration information, and divides the current node into two roles of the master and the slave, if the master and the slave of the database switch in the system operation process, the management module monitors the change and timely updates the change, the connection information of the master and the slave database can be easily acquired through the configuration file designation connection information, so as to perform role determination and initialization, the configuration information acquisition in the initialization stage enables the system to accurately identify the master and slave relation of the database in the operation, provides a necessary information basis for the subsequent master and slave switching and task management, and the system can clearly distinguish the role of the current node through role determination, and the master is responsible for performing the timing task management, while the slave is used as a standby node, and the real-time synchronization master and slave information is helpful for rapidly adapting to the system to the change of the master and the slave, thereby ensuring the reliability of the data and the task management module.
The method comprises the steps of enabling a host and a slave to establish a channel through Netty Server Channel Initializer to realize the exchange of periodical heartbeat data and timing task fragmentation information, and returning an ACK message after the host receives the information, wherein the method comprises the following steps:
s201, acquiring Netty Server Channel Initializer;
s202, enabling a master machine and a slave machine to establish a channel through Netty Server Channel Initializer;
s203, the slave machine periodically transmits heartbeat data and timing task segmentation information to the host machine;
and S204, after receiving the heartbeat data and the timed task fragmentation information, the host performs task management and scheduling, and returns an ACK message to confirm the reception.
In the steps S201 to S204, netty Server Channel Initializer is a key component in the Netty framework, the processing pipelines for initializing the Netty communication channel are respectively initialized in the system starting process of the host and the slave, so that the host and the slave use the same processing pipelines, the slave generates heartbeat data and timing task fragmentation information at a specified time interval and sends the heartbeat data and the timing task fragmentation information to the host through the established Netty channel, the host receives the heartbeat data and the timing task fragmentation information sent by the slave through the Netty channel, performs task management and scheduling, then generates an ACK message and returns the ACK message to the slave, the system can flexibly configure and customize the initialization process of the Netty communication channel to meet specific requirements, the host and the slave can establish a unified communication channel through the same Netty Server Channel Initializer, the consistency and maintainability of the system communication are improved, the regular heartbeat data and task information exchange are helpful for maintaining the real-time communication between the host, the state and task execution condition of the slave can be timely known, the host can receive the appropriate ACK message according to the information of the slave, and the reliability of the slave is ensured, and the reliability of the communication is ensured.
The method comprises the steps that a channel is established between a host and a slave through Netty Server Channel Initializer, periodic heartbeat data and timed task fragmentation information exchange is achieved, and after the step that the host returns an ACK message after receiving the information, the method further comprises the following steps:
if the slave does not sense the heartbeat of the host in the set time or detects the primary-backup switching of the database in the process of regularly accessing the database, the slave indicates that the service of the timing task of the host is abnormal or the primary-backup switching of the database occurs, and triggers the primary-backup switching of the timing task management module;
if the slave is switched to the state of the host, namely, the slave is converted to a master node capable of executing timing task management, the slave sends a switching message to the host to inform the host that the master and the slave are switched, and the host checks the state of the database after receiving the switching message;
if the database is found to be changed into the slave, the host sends an ACK message for agreeing to switch;
if the database is not found to be changed into the slave database, sending an ACK message for refusing the switching;
if the slave machine can not receive the reply of the host machine for a long time, namely the service of the timing task of the host machine is abnormal and can not be communicated, the slave machine is actively switched to the host machine state;
if the timing task service corresponding to the main database is recovered to be normal, the slave is switched back under a certain condition.
As described above, the timing task management module monitors the state of the slave, if the heartbeat of the slave is not received or the switching of the database master and slave is detected within a specified time, the slave actively monitors the state of the master, timely discovers the switching event of the master and slave, ensures the quick response of the system to the switching of the master and slave, ensures the usability of the system, after the slave is switched into the state of the master, the slave sends a switching message to the master through a Netty channel, informs the master that the switching of the master occurs, the master timely acquires the information of the switching of the master, can timely adjust task management and fragmentation strategy, ensures the stable transition of the switching of the master, checks the state of the database after the master receives the switching message sent by the slave, judges whether the database has become a slave, can accurately judge the switching result of the master by checking the database state, provides a basis for the subsequent processing, if the state of the master checks the database has become a slave, the master sends an ACK message agreeing to the slave, the switching role is ensured after the slave receives the message, if the master timely acquires the information of the switching of the master, the master is refused to switch from the master, and the master is not normally operated, if the state is refused to be switched from the master, and the system is normally has been switched, if the state is not normally, the state is refused to the master is normally, and the switching of the master is normally has been continuously, and the state is ensured, if the state is normally has been normally, and the state is not been switched, and the state is ensured, and the state is normally has been normally, and the state is continuously had the state is ensured, and the state is constantly has been normally and has been changed to be a service state, according to the state of the timing task service of the main database, the system can dynamically adjust the master-slave relationship, and the task execution efficiency of the whole system is ensured to be improved to the maximum extent under the normal running state.
Creating an elastijob timing task service, and marking the read-write type of a task, wherein the step comprises the following steps:
s301, acquiring an elastic Job timing task service created by a system;
s302, acquiring an interface of task attribute setting in an elastic Job framework;
s303, marking the elasticJob timing task for acquiring the service according to the interface set by the task attribute in the elasticJob framework, wherein the mark comprises a writing task and a writing task.
In the above steps S301 to S303, in the system, the ElasticJob timing task service should be created by the service module and registered in the task scheduling framework, the ElasticJob framework provides an interface for setting task attributes, and is used for configuring each attribute of the task, the obtained service ElasticJob timing task is marked by using the task attribute setting interface provided by the ElasticJob framework, the read-write type of the task is marked, the system can accurately know which timing tasks currently exist by obtaining the ElasticJob timing task service created by the system, a basis is provided for the subsequent task marking, and the task attribute setting interface is obtained, so that the system can dynamically configure various attributes of the task, including the execution mode, the scheduling policy, and the like, thereby flexibly adapting to different service requirements, and the read-write type of the marking task is helpful for the task scheduling framework to better understand the properties of the task, for example, the marking can include information such as a read task, a write task, and the like, so that the task scheduling framework can more intelligently allocate the execution nodes of the task when the task is fragmented.
The step of obtaining a service fragment instance jobsistance list from the ZooKeeper by the host comprises the following steps:
s401, acquiring a ZooKeeper path;
s402, connecting a host to a ZooKeeper;
s403, the host acquires information of the service fragment instance from the ZooKeeper path, wherein the acquired information of the service fragment instance comprises a list of the service fragment instance.
In the above steps S401 to S403, in the distributed system, the service fragment information is usually stored in the ZooKeeper under a specific path, and the host uses the ZooKeeper client library to establish a connection with the ZooKeeper server, and the host obtains the stored service fragment instance information from the specific path through the ZooKeeper client, which may include the IP address and the execution state of the instance, and the like, through obtaining the ZooKeeper path, the system can clearly store the position of the service fragment information in the ZooKeeper, provide an accurate path for subsequent connection and information acquisition of the host, and through connecting to the ZooKeeper, the host can realize reading and monitoring of the service fragment information stored in the ZooKeeper, so as to keep real-time fragment information synchronization, obtain the information of the service fragment instance, so that the host can know the current service fragment state, including which instances are running, and the execution conditions of the instances, and provide necessary information for subsequent task scheduling, thereby helping the system flexibly adapt to service change, and ensuring that the system is distributed in a balanced and balanced environment.
The step of sending heartbeat data from the slave comprises:
s501, acquiring a heartbeat data format from the slave machine and generating a heartbeat data packet
S502, adding information of a slave on-board service fragment instance JobInstance into a heartbeat data packet;
s503, using the established Netty communication channel, sending the generated heartbeat data packet to the host.
In the steps S501 to S503, the slave defines the format of the heartbeat data in advance, including necessary information, such as the state and the running condition of the slave, and the slave generates a heartbeat data packet conforming to the defined format in the heartbeat period, and when the slave generates the heartbeat data packet, the slave adds the information (such as the example ID and the execution state) of the service fragment instance into the heartbeat data packet to provide a more detailed system state of the host, and the slave sends the generated heartbeat data packet to the host by using the established Netty communication channel, so that the Netty provides an efficient asynchronous communication framework, ensures the stability of communication, ensures the structural consistency of the heartbeat data by defining the heartbeat data format, enables the host to accurately analyze and process the heartbeat information, adds the information of the service fragment instance into the heartbeat data packet, enables the host to know the execution condition of each fragment, provides more comprehensive information for task scheduling, and realizes reliable and real-time communication between the slave and the host by using the Netty communication channel, thereby facilitating the acquisition of the information of the slave to the state of the host and the cooperative real-time monitoring of the system.
The method comprises the steps of obtaining the service type and adjusting the fragment duty ratio according to the service type, and comprises the following steps:
s601, acquiring a fragment manager;
s602, the fragment manager receives the information of the task and judges the service type of the task, wherein the service type comprises a reading task and a writing task;
s603, setting the slicing duty ratio according to the service type.
In the steps S601 to S603, a slicing manager exists in the system and is responsible for scheduling and managing the slicing logic of the task, the slicing manager is generally based on the slicing management mechanism of the elastjob, when receiving the task information, the slicing manager analyzes the service type of the task, determines whether the task is a read task or a write task, and dynamically adjusts the slicing duty ratio according to the service type of the task, for example, for the read task, the slicing duty ratio of a master node can be improved, for the write task, the slicing duty ratio of a slave computer can be improved, the slicing strategy of the slicing manager can be obtained, so that the system can dynamically adjust the task, flexible control over task execution is realized, the slicing manager can know the access property of the task to the database by judging the service type of the task, a basis is provided for subsequent slicing adjustment, the dynamic slicing duty ratio is beneficial to the system to better adapt to the requirements of different service types, and the execution efficiency of the task and the utilization rate of the database resource are improved by optimizing the slicing duty ratio.
The step of setting the fragment duty ratio according to the service type comprises the following steps:
s6031, obtaining a service type;
if the service type is a reading task, the number of fragments and the occupation proportion of the service executed by the host are larger than those of the slave, and the number of fragments and the occupation proportion of the service executed by the host are improved;
if the service type is a writing task, the number of fragments and the occupation ratio of the slave machine executing the service are larger than those of the master machine, and the number of fragments and the occupation ratio of the slave machine executing the service are improved.
In step S6031, the system obtains the task type of the task from the task information, for example, the task is marked as a read task or a write task, the obtaining of the task type is to perform corresponding slice adjustment according to the task property, so as to optimize the task execution efficiency, when the task type is the read task, the system considers that the host is more suitable for executing the task, thus increasing the slice number and occupation ratio of the host to execute the task, improving the slice number and occupation ratio of the host to execute the service, helping to optimize the execution efficiency of the multi-task, fully utilizing the performance of the host, improving the overall reading capability of the system, when the task type is the write task, the system considers that the slave is more suitable for executing the task, thus increasing the slice number and occupation ratio of the slave to execute the service, helping to optimize the execution efficiency of the write task, fully utilizing the performance of the slave, improving the overall writing capability of the system, and by dynamically adjusting the slice ratio of the master-slave according to the service type, the system can better adapt to the requirements of different types of the task, improving the performance of the overall system, and the flexible slice adjustment helps to optimize the task in different systems running environments.
And, an apparatus for performing timed tasks in a ex-situ dual activity system, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement a method of performing timed tasks in a ex situ dual activity system.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention. Structures, devices and methods of operation not specifically described and illustrated herein, unless otherwise indicated and limited, are implemented according to conventional means in the art.

Claims (10)

1. A method for performing timed tasks in a ex-situ dual activity system, comprising:
acquiring master-slave information of a database, and distinguishing a host from a slave;
the host and the slave establish a channel through Netty Server Channel Initializer to realize the exchange of periodical heartbeat data and the time task fragmentation information, and the host returns an ACK message after receiving the information;
if the slave does not sense the heartbeat of the host in the set time or detects the primary-backup switching of the database in the process of regularly accessing the database, the slave indicates that the service of the timing task of the host is abnormal or the primary-backup switching of the database occurs, and triggers the primary-backup switching of the timing task management module;
acquiring a service for creating an elastic job timing task, and marking the read-write type of the task;
the host acquires a JobInstance list of the service fragment instance from the ZooKeeper;
the slave sends heartbeat data;
and acquiring the service type, and adjusting the fragment duty ratio according to the service type.
2. The method for performing timing tasks in a ex-situ dual-activity system as in claim 1, wherein said step of obtaining master-slave information of a database, distinguishing between a master and a slave, comprises:
designating connection information of a master database and a slave database in a database configuration file;
initializing the system when starting, and obtaining configuration information of a database by a management module;
according to the obtained configuration information of the database, the management module judges the role of the current node, wherein the role comprises a host and a slave, the host is a master node responsible for executing timing task management, and the slave is a standby node capable of being switched to the host when the master is switched;
if the master-slave switching of the database occurs, the management module synchronously updates master-slave information in time.
3. The method for performing timing tasks in a two-in-one system according to claim 1, wherein the step of enabling the master and the slave to establish a channel through Netty Server Channel Initializer to exchange periodic heartbeat data and timing task fragmentation information, and returning an ACK message after the master receives the information, comprises:
acquiring Netty Server Channel Initializer;
causing the master and the slave to establish a channel through Netty Server Channel Initializer;
the slave machine periodically sends heartbeat data and timing task segmentation information to the host machine;
after receiving the heartbeat data and the timed task fragmentation information, the host performs task management and scheduling, and returns an ACK message to confirm the reception.
4. The method for performing timing tasks in a two-in-one system according to claim 1, wherein the step of enabling the master and the slave to establish a channel through Netty Server Channel Initializer to exchange periodic heartbeat data and timing task fragmentation information, and returning an ACK message after the master receives the information, further comprises:
if the slave does not sense the heartbeat of the host in the set time or detects the primary-backup switching of the database in the process of regularly accessing the database, the slave indicates that the service of the timing task of the host is abnormal or the primary-backup switching of the database occurs, and triggers the primary-backup switching of the timing task management module;
if the slave is switched to the state of the host, namely, the slave is converted to a master node capable of executing timing task management, the slave sends a switching message to the host to inform the host that the master and the slave are switched, and the host checks the state of the database after receiving the switching message;
if the database is found to be changed into the slave, the host sends an ACK message for agreeing to switch;
if the database is not found to be changed into the slave database, sending an ACK message for refusing the switching;
if the slave machine can not receive the reply of the host machine for a long time, namely the service of the timing task of the host machine is abnormal and can not be communicated, the slave machine is actively switched to the host machine state;
if the timing task service corresponding to the main database is recovered to be normal, the slave is switched back under a certain condition.
5. The method for executing timed tasks in a ex-situ dual-activity system as claimed in claim 1 wherein said creating an elastijob timed task service, marking the read-write type of task comprises:
acquiring an elastic Job timing task service created by a system;
acquiring an interface of task attribute setting in an elastic Job framework;
marking the elasticJob timing task for acquiring the service according to the interface set by the task attribute in the elasticJob framework, wherein the mark comprises a writing task and a writing task.
6. The method for performing timed tasks in a two-in-place system according to claim 1, wherein the step of the host obtaining a list of service fragment instances jobsistance from a ZooKeeper comprises:
acquiring a ZooKeeper path;
the host is connected to the ZooKeeper;
the host acquires information of the service fragment instance from the ZooKeeper path, and the acquired information of the service fragment instance comprises a list of the service fragment instance.
7. The method for performing timed tasks in a ex-situ dual activity system of claim 1 wherein said slave sends heartbeat data comprising:
the slave acquires the heartbeat data format and generates a heartbeat data packet
Adding information of a slave on-board service fragment instance JobInstance into a heartbeat data packet;
and sending the generated heartbeat data packet to a host computer by using the established Netty communication channel.
8. The method for performing timing tasks in a ex-situ dual-activity system as in claim 1 wherein said step of obtaining a traffic type and adjusting a slice ratio according to the traffic type comprises:
acquiring a fragmentation manager;
the segmentation manager receives the information of the task and judges the service type of the task, wherein the service type comprises a reading task and a writing task;
and setting the fragmentation duty ratio according to the service type.
9. The method for performing timing tasks in a ex-situ dual activity system as in claim 1 wherein said step of setting a slice duty cycle based on traffic type comprises:
acquiring a service type;
if the service type is a reading task, the number of fragments and the occupation proportion of the service executed by the host are larger than those of the slave, and the number of fragments and the occupation proportion of the service executed by the host are improved;
if the service type is a writing task, the number of fragments and the occupation ratio of the slave machine executing the service are larger than those of the master machine, and the number of fragments and the occupation ratio of the slave machine executing the service are improved.
10. An apparatus for performing timed tasks in a ex-situ dual activity system, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement a method of performing timed tasks in the ex-situ dual activity system.
CN202311708831.XA 2023-12-13 2023-12-13 Method and device for executing timing task in remote double-activity system Pending CN117827402A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311708831.XA CN117827402A (en) 2023-12-13 2023-12-13 Method and device for executing timing task in remote double-activity system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311708831.XA CN117827402A (en) 2023-12-13 2023-12-13 Method and device for executing timing task in remote double-activity system

Publications (1)

Publication Number Publication Date
CN117827402A true CN117827402A (en) 2024-04-05

Family

ID=90512556

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311708831.XA Pending CN117827402A (en) 2023-12-13 2023-12-13 Method and device for executing timing task in remote double-activity system

Country Status (1)

Country Link
CN (1) CN117827402A (en)

Similar Documents

Publication Publication Date Title
CN113037552B (en) Network method, network device, and computer-readable storage medium
EP1810447B1 (en) Method, system and program product for automated topology formation in dynamic distributed environments
US9999030B2 (en) Resource provisioning method
Pashkov et al. Controller failover for SDN enterprise networks
US20050013255A1 (en) Automatic configuration of network for monitoring
CN110795503A (en) Multi-cluster data synchronization method and related device of distributed storage system
CN102521044A (en) Distributed task scheduling method and system based on messaging middleware
GB2407887A (en) Automatically modifying fail-over configuration of back-up devices
CN110719311B (en) Distributed coordination service method, system and computer readable storage medium
US20060120384A1 (en) Method and system for information gathering and aggregation in dynamic distributed environments
CN113067850A (en) Cluster arrangement system under multi-cloud scene
EP3457668B1 (en) Clustering in unified communication and collaboration services
CN111641521A (en) Ambari-based method for deploying big data clusters on cloud
CN114553867A (en) Cloud-native cross-cloud network monitoring method and device and storage medium
CN112087506B (en) Cluster node management method and device and computer storage medium
CN111064643B (en) Node server and data interaction method and related device thereof
CN109257435B (en) Communication system of multi-instance multi-subsystem application based on message middleware
CN117827402A (en) Method and device for executing timing task in remote double-activity system
CN102694677A (en) New method for disaster recovery data center construction based on remote IP analysis
CN105468446A (en) Linux-based HPC job scheduling high availability method
CN115391058A (en) SDN-based resource event processing method, resource creating method and system
CN112256447A (en) Message notification method and system
CN107644035B (en) Database system and deployment method thereof
CN104158835A (en) Method for intelligent desktop system server to control clients
WO2024093328A1 (en) Virtual desktop connection method and apparatus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination