CN111078418B - Operation synchronization method, device, electronic equipment and computer readable storage medium - Google Patents
Operation synchronization method, device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111078418B CN111078418B CN201911341196.XA CN201911341196A CN111078418B CN 111078418 B CN111078418 B CN 111078418B CN 201911341196 A CN201911341196 A CN 201911341196A CN 111078418 B CN111078418 B CN 111078418B
- Authority
- CN
- China
- Prior art keywords
- progress
- executed
- task
- server
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000008569 process Effects 0.000 claims abstract description 20
- 238000004891 communication Methods 0.000 claims description 13
- 238000001514 detection method Methods 0.000 claims description 13
- 230000004048 modification Effects 0.000 claims description 11
- 238000012986 modification Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 7
- 230000005540 biological transmission Effects 0.000 claims description 2
- 238000012545 processing Methods 0.000 description 7
- 238000012217 deletion Methods 0.000 description 5
- 230000037430 deletion Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000000638 solvent extraction Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
Abstract
The embodiment of the invention provides an operation synchronization method, an operation synchronization device, electronic equipment and a computer readable storage medium, and belongs to the technical field of computers. According to the method, under the condition that the operation quantity of the task to be executed is larger than the preset threshold, the executed task to be executed is executed, and in the process of executing the task to be executed, the generated operation log file is sent to the slave server, after each preset number of operations are executed, the master server judges the progress difference between the first progress of the master server and the second progress of the slave server, and when the progress difference is larger than the preset progress difference threshold, the execution speed of the master server is reduced, so that the fact that the slave server executes the same operation in a longer period after the master server executes all operations can be avoided to a certain extent, further operation synchronization delay is reduced, and operation synchronization efficiency is improved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to an operation synchronization method, an operation synchronization device, an electronic device, and a computer readable storage medium.
Background
In order to improve the quality of the network usage experience for the user, the provider often sets up a server cluster to provide services for the user.
When the clients used by the user interact, the task to be executed submitted by the client is submitted to the master server, the master server directly executes the task to be executed, and after execution is completed, an operation log file generated by executing the task to be executed is sent to the slave server, accordingly, the slave server executes the same operation as the master server according to the operation log file, and accordingly, the data of the master server and the slave server are kept consistent.
However, in this synchronization manner, when the amount of operation of the task to be executed is large, the slave server can complete the task after the master server completes the long time of executing the task, and the synchronization delay of the slave server and the master server is large, so that the synchronization effect is poor.
Disclosure of Invention
The invention provides an operation synchronization method, an operation synchronization device, electronic equipment and a computer readable storage medium, so as to solve the problems of larger synchronization delay and poorer synchronization effect.
In a first aspect of the present invention, there is provided an operation synchronization method, applied to a primary server, the method including:
Executing the task to be executed under the condition that the operation amount of the task to be executed is larger than a preset threshold value, and transmitting the generated operation log file to the slave server in the process of executing the task to be executed; the task to be executed comprises a plurality of operations, and the operation log file is used for indicating the slave server to execute the same operations as the master server;
after each preset number of operations are executed, determining a progress difference between the first progress and the second progress; the first progress is the progress of the master server executing the preset number of operations, and the second progress is the progress of the slave server executing the preset number of operations;
and when the progress difference is larger than a preset progress difference threshold value, reducing the execution speed of the main server.
In a second aspect of the present invention, there is also provided an operation synchronization device applied to a main server, the device including:
The transmission module is used for executing the task to be executed under the condition that the operation quantity of the task to be executed is larger than a preset threshold value, and transmitting the generated operation log file to the slave server in the process of executing the task to be executed; the task to be executed comprises a plurality of operations, and the operation log file is used for indicating the slave server to execute the same operations as the master server;
The first determining module is used for determining a progress difference between the first progress and the second progress after each preset number of operations are executed; the first progress is the progress of the master server executing the preset number of operations, and the second progress is the progress of the slave server executing the preset number of operations;
And the reducing module is used for reducing the execution speed of the main server when the progress difference is larger than a preset progress difference threshold value.
In yet another aspect of the present invention, there is also provided a computer readable storage medium having instructions stored therein, which when run on a computer, cause the computer to perform any of the above-described operation synchronization methods.
In yet another aspect of the invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the above described operation synchronization methods.
According to the operation synchronization method provided by the embodiment of the invention, the operation log file is used for indicating the slave server to execute the same operation as the master server by transmitting the generated operation log file to the slave server under the condition that the operation amount of the task to be executed is larger than the preset threshold value, so that the slave server does not need to wait until the master server completes execution of the task to be executed to begin operation synchronization, the delay degree of operation synchronization can be reduced to a certain extent, the master server further judges the progress difference between the first progress of the master server and the second progress of the slave server after executing the preset number of operations, and reduces the execution speed of the master server when the progress difference is larger than the preset progress difference threshold value, so that the slave server can avoid the execution of the same operation after the master server completes all operations to a certain extent by reducing the execution speed of the master server, the delay of operation synchronization can be reduced, and the operation synchronization efficiency can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a flow chart of steps of a method for synchronizing operations provided by an embodiment of the present invention;
FIG. 2-1 is a flow chart of steps of another method for synchronizing operations provided by an embodiment of the present invention;
FIG. 2-2 is a schematic diagram of a system according to an embodiment of the present invention;
FIG. 3 is a block diagram of an operation synchronization device according to an embodiment of the present invention;
fig. 4 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Fig. 1 is a flowchart of steps of a method for operating synchronization according to an embodiment of the present invention, which is applied to a main server, as shown in fig. 1, and the method may include:
step 101, executing the task to be executed under the condition that the operation amount of the task to be executed is larger than a preset threshold value, and transmitting the generated operation log file to the slave server in the process of executing the task to be executed. The task to be performed includes a plurality of operations.
In the embodiment of the present invention, the plurality of operations included in the task to be executed may be operations of adding, deleting, and modifying the data stored in the main server. The plurality of operations may be represented in SQL statements defined based on a structured query language (Structured Query Language, SQL), where SQL is a language that operates on a database in a server. The sql statement may be of the DDL/DML language type.
Further, the operation amount of the task to be executed may be the number of operations included in the task to be executed, that is, the number of sql statements, or the data size of the sql statement corresponding to the operation included in the task to be executed, which is not limited in the embodiment of the present invention. The preset threshold may be set according to actual requirements, and specifically, the preset threshold may be set according to a data amount that may cause a large time difference between the time points of completing tasks of the master server and the slave server. For example, the preset threshold may be 1000, which is not limited in the embodiment of the present invention.
Further, if the operation amount of the task to be executed is greater than the preset threshold, it may be considered that a larger delay occurs when the master-slave server synchronizes the operations in the task to be executed. Therefore, in this step, the generated operation log file may be transmitted to the slave server during the execution of the task to be executed, and accordingly, after the slave server receives the operation log file, the same operation as that executed by the master server may be executed according to the operation log file. Therefore, the slave server does not need to wait until the master server finishes executing the task to be executed, and operation synchronization is started, so that the delay degree of the operation synchronization can be reduced to a certain extent.
Step 102, after each execution of a preset number of operations, determining a progress difference between the first progress and the second progress; the first progress is the progress of the master server executing the preset number of operations, and the second progress is the progress of the slave server executing the preset number of operations.
In the embodiment of the present invention, the preset number may be set according to actual requirements, for example, the preset number may be 100, which is not limited in the embodiment of the present invention. Specifically, the main server may set a counter, after each operation is performed, control the value recorded by the counter to be increased by one, and after the value recorded by the counter reaches the preset number, perform an operation of determining the progress difference, and set the value recorded by the counter to be 0, so that the counter is continuously used for counting in a subsequent process.
Further, the master server may determine a difference in progress between the first progress of the master server and the second progress of the slave server to determine a current operation delay of the two, if it is detected that a preset number of operations are performed, because execution speeds between the master server and the slave server may be different, due to external influences. Specifically, the first progress may be an operand that has been executed by the master server, and the second progress may be an operand that has been executed by the slave server, and accordingly, an absolute value of a difference between the operand that has been executed by the master server and the operand that has been executed by the slave server may be calculated, and the absolute value may be determined as a progress difference.
And step 103, when the progress difference is larger than a preset progress difference threshold value, reducing the execution speed of the main server.
In the embodiment of the present invention, the preset progress difference threshold may be set according to a progress difference that may cause a larger time difference between the time points of completing tasks of the master server and the slave server, which is not limited in the embodiment of the present invention. If the progress difference is greater than the preset progress difference threshold, it can be considered that if the operation is continuously executed according to the current progress, the slave server can execute the task to be executed only after the master server completes the task to be executed for a long time. Therefore, the main server can reduce the execution speed of the main server under the condition, further reduce the progress difference of the main server and the main server, reduce the time difference between the time point when the main server completes the task and the time point when the main server completes the task, and reduce the delay of operation synchronization.
In summary, in the operation synchronization method provided by the embodiment of the present invention, by performing the task to be performed when the amount of operation of the task to be performed is greater than the preset threshold, and transmitting the generated operation log file to the slave server in the process of performing the task to be performed, where the operation log file is used to instruct the slave server to perform the same operation as the master server, the slave server does not need to wait until the master server performs the task to be performed, so that the delay degree of operation synchronization can be reduced to a certain extent, further, the master server determines the progress difference between the first progress of the master server and the second progress of the slave server after each execution of the preset number of operations, and reduces the execution speed of the master server when the progress difference is greater than the preset progress difference threshold, so that by reducing the execution speed of the master server, the slave server can avoid the execution of the same operation after the master server completes all operations, thereby reducing the delay of operation synchronization after the master server completes all operations, and improving the efficiency of operation synchronization.
Fig. 2 is a flowchart of steps of another method for synchronizing operations provided in an embodiment of the present invention, which is applied to a main server, as shown in fig. 2, and the method may include:
Step 201, dividing the task to be executed into a plurality of subtasks according to the preset number; wherein each subtask includes an operand that is the preset number.
In this step, a plurality of operations included in the task to be executed may be sequentially divided according to a preset number, so as to obtain a plurality of subtasks. For example, assuming that the task to be performed includes 1000 operations, and the preset number is 100, each 100 operations may be divided into one subtask, and thus 10 subtasks are obtained. For example, when partitioning may be performed, a particular partitioning tool may be employed, which may be denoted as "Oak-chunk-update", where a particular partitioning tool may be a particular component (openark-kit) from the MySQL toolset of open sources, which may implement a subtask by partitioning a field, where the field may be a primary key index field, a unique index field, or a specified field.
Step 202, executing the subtask, and in the process of executing the subtask, sending an operation log file corresponding to the subtask to the slave server.
In this step, the operation log file may be a file specified by a configuration file log-bin, and the log in the operation log file may be a binary log (binary log). When the primary server performs an operation, changes to data in the internal database may be recorded into the binary log, the binary log recorded with the data changes belonging to a binary log event (binary log events). Accordingly, the master server performs the same operation as that performed by the master server by sending the operation log file to the slave server, reading the binary log events in the operation log file through the I/O thread after receiving the operation log file from the slave server, and writing the binary log events into the relay log (relay log) of the slave server, and performing the same operation as that performed by the master server, namely, "data replay", one piece by one piece of event information in the relay log, thereby realizing that the recorded changes are reflected in the data of the slave server itself. In the embodiment of the invention, the master server generates the operation log file according to the execution process by executing the operation first and sends the operation log file to the slave server for operation synchronization, so that the slave server can realize the operation synchronization by playing back the data based on the operation log file, and the efficiency of the operation synchronization is improved to a certain extent.
Step 203, determining a progress difference between the first progress and the second progress after each sub-task is executed.
Specifically, the specific implementation of determining the progress difference may refer to the description in step 102, and the embodiment of the present invention is not described herein. In the embodiment of the invention, the task to be executed is divided into the subtasks according to the preset number, and after one subtask is executed, the fact that the preset number of operations are executed can be directly confirmed, so that the progress difference judging operation can be executed. Therefore, the number of operations which are executed currently does not need to be counted, and the needed processing resources can be saved to a certain extent. Meanwhile, the to-be-executed task with large data volume is divided into a plurality of subtasks, and the main server realizes the mode of executing the to-be-executed task by executing the plurality of subtasks, so that the operation execution pressure of the main server can be reduced, and the influence on a database deployed in the main server when the to-be-executed task is executed is further reduced.
And 204, when the progress difference is larger than a preset progress difference threshold value, reducing the execution speed of the main server.
Specifically, the main server may reduce the execution speed to 0, i.e., stop executing the task to be executed. Thus, by reducing the execution speed to 0, the progress difference can be reduced with maximum efficiency, and the overall progress of operation execution can be further accelerated.
Step 205, determining a progress difference between the first progress and the second progress once every preset time.
In this step, the preset duration may be preset, for example, the preset duration may be 5 seconds, which is not limited in this embodiment of the present invention, and since the execution speed of the master server is reduced, the progress difference between the master server and the slave server may be reduced, so that the progress difference between the master server and the slave server may be determined once every preset duration, so that the progress of the master server may be recovered in time in the subsequent step when the progress difference between the master server and the slave server is reduced to be not greater than the preset progress difference threshold. Meanwhile, compared with the real-time detection of the progress difference between the two, the method and the device can reduce the detection times and further save the processing resources of the main server when the progress of the main server is recovered in time due to the fact that the progress difference between the two is detected in real time and a large amount of processing resources are consumed.
And 206, when the progress difference is not greater than the preset progress difference threshold, restoring the execution speed to the target speed and executing the next subtask.
In this step, the target speed may be a default execution speed set in advance, or may be a speed before the execution speed is reduced. If the progress difference is not greater than the preset progress difference threshold, it can be considered that at this time, if the operation is continued, no large delay occurs between the master server and the slave server, and accordingly, the execution speed can be restored to the target speed and the next sub-task can be executed. Fig. 2-2 are schematic diagrams of a system provided in an embodiment of the present invention, where, as shown in fig. 2-2, the tasks of the task-1, the task-2, and the task-3 respectively represent three subtasks, after each subtask is executed, an operation of "checking a progress difference" may be performed, and if the progress difference is not greater than a preset progress difference threshold, the next subtask is continuously executed.
It should be noted that, before executing the task, the main server may receive a modification instruction, where the modification instruction may be sent by the user to the main server through the used terminal. The primary server may then add the progress difference detection function carried in the modification instruction to the primary server. The progress difference detection function is used for detecting the first progress and the second progress and calculating the progress difference between the first progress and the second progress. The main server can be deployed with a default preset execution tool, and the operation synchronization method can take the default execution tool as an implementation basis. Specifically, the modification instruction may be an instruction for modifying a tool, and the main server may add a progress difference detection function carried in the modification instruction to the tool, so as to implement adding detection logic in the tool. In the embodiment of the invention, the progress difference detection function is added in the tool, so that the progress difference detection function can be realized after each part of tasks are executed, and the scheme has lower realization cost. The implementation difficulty is low.
Further, the main server may extract an update value in the parameter update instruction and a parameter to be updated corresponding to the update value under the condition that the parameter update instruction is received. The parameters to be updated are one or more of a preset progress threshold, a preset number and a preset duration, and the parameters can be predefined in a preset task execution tool. The parameter update instruction may be sent by the user to the main server via the terminal used. For example, a parameter update interface may be displayed, and the user may input a corresponding update value for one or more of a preset progress difference threshold, a preset number, and/or a preset duration according to the requirement, and then click a send button to implement sending the parameter modification instruction. Then, the main server may set the value of the parameter to be updated corresponding to the updated value as the updated value. For example, assume that the parameter update instruction includes an update value: 110, parameters to be updated corresponding to the update value: the preset number, the host server may modify the preset number to a value of 110. In the embodiment of the invention, the parameters can be updated according to the parameter updating instruction, so that a user can conveniently and flexibly adjust the parameters according to the actual execution condition.
Further, a progress difference detection control interface may be defined in the main server, and accordingly, the main server may perform an operation of the task to be performed when the operation amount of the task to be performed is greater than a preset threshold value in a case where a start instruction is received. The starting instruction is sent through the progress difference detection control interface. In this step, by reserving a control interface for the user, the user can flexibly control whether to use the operation synchronization method in the embodiment of the invention to perform operation synchronization according to the requirement.
Further, in the case that the master server has performed the task to be performed and the slave server has not performed the task to be performed, that is, the master server has performed the last sub-task, but the slave server has not performed the last sub-task, the master server may first close the access right of the slave server. Specifically, an access right closing instruction may be sent to the slave server, and after the slave server receives the closing instruction, the access right may be closed to suspend interaction with other devices. In an actual application scenario, a master server and a slave server may be separated from each other in reading and writing, for example, the master server is used for providing a writing service, the slave server is used for providing a reading service, that is, a storage device writes new data into the master server, and when the new data needs to be read, an online device reads the new data from the server. If there is a delay between the master server and the slave server, the on-line device cannot read the data from the slave server normally, which results in an on-line failure. Therefore, in the embodiment of the invention, the master server closes the access authority of the slave server under the condition that the task to be executed is executed and the slave server does not execute the task to be executed, so that the problem of online fault caused by delay of the master server and the slave server is avoided.
For example, taking a task to be performed as an example for deleting 3 million data, when an operation of deleting 3 million data is performed by the master server, and then the generated operation log file is transmitted to the slave server, the slave server needs to consume a day for deletion in the case that the slave server performs the same operation through the operation log files, and the slave server can complete the entire deletion operation after 8 hours of completion of deletion by the master server. By adopting the operation synchronization method provided by the embodiment of the invention, the main server needs 8 days to delete, but the slave server can complete the whole deletion operation within 1 second after the main server completes the deletion.
In summary, according to the operation synchronization method provided by the embodiment of the present invention, the task to be executed is divided into a plurality of subtasks according to the preset number, so that the task to be executed is divided into the subtasks according to the preset number, and after one subtask is executed, it is directly confirmed that the preset number of operations have been executed, and further, the required processing resources can be saved to a certain extent. Then, the subtasks can be executed, and in the process of executing the subtasks, the operation log files corresponding to the subtasks are sent to the slave servers. The operation log file is used for indicating the slave server to execute the same operation as the master server, so that the slave server does not need to wait until the master server executes the task to be executed, the delay degree of operation synchronization can be reduced to a certain extent, and further, the master server can judge the progress difference between the first progress and the second progress after executing one subtask. When the progress difference is larger than a preset progress difference threshold, the execution speed of the main server is reduced, so that the same operation can be prevented from being executed by the auxiliary server only in a longer period after all operations are executed by the main server to a certain extent by reducing the execution speed of the main server, further delay of operation synchronization is reduced, and efficiency of operation synchronization is improved.
Fig. 3 is a block diagram of an operation synchronization device provided in an embodiment of the present invention, applied to a main server, as shown in fig. 3, the device 30 may include:
a sending module 301, configured to execute a task to be executed if an operation amount of the task to be executed is greater than a preset threshold, and send the generated operation log file to a slave server in a process of executing the task to be executed; the task to be executed comprises a plurality of operations, and the operation log file is used for indicating the slave server to execute the same operations as the master server.
A first determining module 302, configured to determine a progress difference between the first progress and the second progress after each execution of a preset number of operations; the first progress is the progress of the master server executing the preset number of operations, and the second progress is the progress of the slave server executing the preset number of operations.
And the reducing module 303 is configured to reduce the execution speed of the main server when the progress difference is greater than a preset progress difference threshold.
Optionally, the sending module 301 is specifically configured to:
Dividing the task to be executed into a plurality of subtasks according to the preset number; wherein each subtask includes an operand that is the preset number.
And executing the subtasks, and transmitting the operation log files corresponding to the subtasks to the slave server in the process of executing the subtasks.
The first determining module is specifically configured to:
After each sub-task is executed, a progress difference between the first progress and the second progress is determined.
Optionally, the reducing module 303 is specifically configured to:
the execution speed is set to 0.
Accordingly, the apparatus 30 further comprises:
And the second determining module is used for determining the progress difference between the first progress and the second progress once every interval for a preset duration.
And the recovery module is used for recovering the execution speed to the target speed and executing the next subtask when the progress difference is not greater than the preset progress difference threshold value.
Optionally, the apparatus 30 further includes:
And the receiving module is used for receiving the modification instruction.
The adding module is used for adding the progress difference detection function carried in the modification instruction to the main server; the progress difference detection function is used for detecting the first progress and the second progress and calculating the progress difference between the first progress and the second progress.
Optionally, the apparatus 30 further includes:
The extraction module is used for extracting an update value in the parameter update instruction and a parameter to be updated corresponding to the update value under the condition that the parameter update instruction is received; the parameter to be updated is one or more of the preset progress difference threshold value, the preset quantity and the preset duration.
And the setting module is used for setting the value of the parameter to be updated corresponding to the updated value as the updated value.
Optionally, the apparatus 30 further includes:
And the closing module is used for closing the access right of the slave server under the condition that the task to be executed is executed and the slave server does not execute the task to be executed.
In summary, in the operation synchronization device provided in the embodiment of the present invention, by performing the task to be performed when the amount of operation of the task to be performed is greater than the preset threshold, and sending the generated operation log file to the slave server in the process of performing the task to be performed, where the operation log file is used to instruct the slave server to perform the same operation as the master server, the slave server does not need to wait until the master server performs the task to be performed, so that the delay degree of operation synchronization can be reduced to a certain extent, further, the master server determines, after each time a preset number of operations are performed, a difference in progress between a first progress of the master server and a second progress of the slave server, and reduces the execution speed of the master server when the difference in progress is greater than the preset difference in progress threshold, so that by reducing the execution speed of the master server, the slave server can avoid to a certain extent that the master server performs the same operation after all operations are performed by a longer time, and further delay of operation synchronization can be reduced, and efficiency of operation synchronization can be improved.
For the above-described device embodiments, since they are substantially similar to the method embodiments described above, the description is relatively simple, and reference is made to the description of the method embodiments in part.
The embodiment of the invention also provides an electronic device, as shown in fig. 4, which comprises a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402 and the memory 403 complete communication with each other through the communication bus 404,
A memory 403 for storing a computer program;
The processor 401, when executing the program stored in the memory 403, implements the following steps:
Executing the task to be executed under the condition that the operation amount of the task to be executed is larger than a preset threshold value, and transmitting the generated operation log file to the slave server in the process of executing the task to be executed; the task to be executed comprises a plurality of operations, and the operation log file is used for indicating the slave server to execute the same operations as the master server;
after each preset number of operations are executed, determining a progress difference between the first progress and the second progress; the first progress is the progress of the master server executing the preset number of operations, and the second progress is the progress of the slave server executing the preset number of operations;
And when the progress difference is larger than a preset progress difference threshold value, reducing the execution speed of the main server. The communication bus mentioned by the above electronic device may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus. The communication interface is used for communication between the electronic device and other devices.
Further, the memory may include random access memory (Random Access Memory, abbreviated as RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
Further, the above-mentioned processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), etc.; but may also be a digital signal processor (DIGITAL SIGNAL Processing, DSP), application Specific Integrated Circuit (ASIC), field-Programmable gate array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components.
Further, in yet another embodiment of the present invention, there is also provided a computer-readable storage medium having instructions stored therein, which when run on a computer, cause the computer to perform the steps of the operation synchronization method described in any one of the above embodiments.
Further, in yet another embodiment provided by the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the steps of the operation synchronization method described in any one of the above embodiments.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. Further, the computer instructions may be stored in or transmitted from one computer readable storage medium to another, for example, by wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means from one website, computer, server, or data center. Alternatively, the computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device including one or more servers, data centers, etc. that can be integrated with the available medium. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (8)
1. A method of operation synchronization, applied to a primary server, the method comprising:
executing the task to be executed under the condition that the operation amount of the task to be executed is larger than a preset threshold value, and transmitting the generated operation log file to the slave server in the process of executing the task to be executed, wherein the method comprises the following steps: dividing the task to be executed into a plurality of subtasks by dividing a primary key index field, a unique index field or a designated field according to a preset number; wherein each subtask includes an operand of the preset number; executing the subtasks, and transmitting the operation log files corresponding to the subtasks to the slave server in the process of executing the subtasks; the task to be executed comprises a plurality of operations, and the operation log file is used for indicating the slave server to execute the same operations as the master server;
After each sub-task is executed, judging the progress difference between the first progress and the second progress; the first progress is an operand of the preset number of operations executed by the master server, and the second progress is an operand of the preset number of operations executed by the slave server;
and when the progress difference is larger than a preset progress difference threshold value, reducing the execution speed of the main server.
2. The method of claim 1, wherein the reducing the execution speed of the primary server comprises:
setting the execution speed to 0;
accordingly, after the execution speed of the main server is reduced, the method further includes:
determining a progress difference between the first progress and the second progress once every preset time length;
And when the progress difference is not larger than the preset progress difference threshold value, the execution speed is restored to the target speed, and the next subtask is executed.
3. The method according to claim 1, wherein, in the case where the operation amount of the task to be executed is greater than a preset threshold, the method further comprises, before executing the task to be executed:
receiving a modification instruction;
Adding a progress difference detection function carried in the modification instruction to the main server; the progress difference detection function is used for detecting the first progress and the second progress and calculating the progress difference between the first progress and the second progress.
4. The method according to claim 1, wherein the method further comprises:
under the condition that a parameter updating instruction is received, extracting an updating value in the parameter updating instruction and a parameter to be updated corresponding to the updating value; the parameter to be updated is one or more of the preset progress difference threshold value, the preset quantity and the preset duration;
and setting the value of the parameter to be updated corresponding to the updated value as the updated value.
5. The method according to claim 1, wherein the method further comprises:
And closing the access right of the slave server under the condition that the task to be executed is executed and the slave server does not execute the task to be executed.
6. An operation synchronization device, characterized by being applied to a main server, the device comprising:
The transmission module is used for executing the task to be executed under the condition that the operation quantity of the task to be executed is larger than a preset threshold value, and transmitting the generated operation log file to the slave server in the process of executing the task to be executed; the task to be executed comprises a plurality of operations, and the operation log file is used for indicating the slave server to execute the same operations as the master server;
the first determining module is used for determining a progress difference between the first progress and the second progress after each preset number of operations are executed; the first progress is an operand of the preset number of operations executed by the master server, and the second progress is an operand of the preset number of operations executed by the slave server;
The reducing module is used for reducing the execution speed of the main server when the progress difference is larger than a preset progress difference threshold value;
the sending module is specifically configured to:
dividing the task to be executed into a plurality of subtasks by dividing a primary key index field, a unique index field or a designated field according to the preset number; wherein each subtask includes an operand of the preset number;
Executing the subtasks, and transmitting the operation log files corresponding to the subtasks to the slave server in the process of executing the subtasks;
The first determining module is specifically configured to:
After each sub-task is executed, a progress difference between the first progress and the second progress is determined.
7. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
a processor for carrying out the method steps of any one of claims 1-5 when executing a program stored on a memory.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911341196.XA CN111078418B (en) | 2019-12-23 | 2019-12-23 | Operation synchronization method, device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911341196.XA CN111078418B (en) | 2019-12-23 | 2019-12-23 | Operation synchronization method, device, electronic equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111078418A CN111078418A (en) | 2020-04-28 |
| CN111078418B true CN111078418B (en) | 2024-04-16 |
Family
ID=70316975
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911341196.XA Active CN111078418B (en) | 2019-12-23 | 2019-12-23 | Operation synchronization method, device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111078418B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112328529B (en) * | 2020-11-03 | 2023-09-05 | 北京奇艺世纪科技有限公司 | Method, device, terminal and storage medium for determining reading progress of electronic book |
| CN114553895B (en) * | 2022-04-24 | 2022-07-22 | 浙江中控技术股份有限公司 | Data synchronization method, system, storage medium and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108958922A (en) * | 2017-05-17 | 2018-12-07 | 北京京东尚科信息技术有限公司 | Method and apparatus for executing task |
| CN109905457A (en) * | 2019-01-09 | 2019-06-18 | 广州视源电子科技股份有限公司 | Principal and subordinate's server data synchronous method, device, computer equipment and storage medium |
| CN110019502A (en) * | 2017-08-29 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Synchronous method, Database Systems and equipment between primary database and standby database |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8266107B2 (en) * | 2009-03-11 | 2012-09-11 | International Business Machines Corporation | Method for mirroring a log file by threshold driven synchronization |
| CN106302806B (en) * | 2016-09-13 | 2019-10-25 | 腾讯科技（深圳）有限公司 | A kind of method of data synchronization, system, synchronous obtaining method and relevant apparatus |
-
2019
- 2019-12-23 CN CN201911341196.XA patent/CN111078418B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108958922A (en) * | 2017-05-17 | 2018-12-07 | 北京京东尚科信息技术有限公司 | Method and apparatus for executing task |
| CN110019502A (en) * | 2017-08-29 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Synchronous method, Database Systems and equipment between primary database and standby database |
| CN109905457A (en) * | 2019-01-09 | 2019-06-18 | 广州视源电子科技股份有限公司 | Principal and subordinate's server data synchronous method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111078418A (en) | 2020-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11023355B2 (en) | Dynamic tracing using ranking and rating | |
| US9684534B2 (en) | Monitoring and modifying allocated computing resources | |
| CN111177165B (en) | Method, device and equipment for detecting data consistency | |
| WO2022063284A1 (en) | Data synchronization method and apparatus, device, and computer-readable medium | |
| US8516499B2 (en) | Assistance in performing action responsive to detected event | |
| KR102311032B1 (en) | Database Synchronization | |
| US9355002B2 (en) | Capturing trace information using annotated trace output | |
| CN108255620B (en) | Service logic processing method, device, service server and system | |
| US10031948B1 (en) | Idempotence service | |
| CN107656705B (en) | Computer storage medium and data migration method, device and system | |
| CN107092628B (en) | Time series data processing method and device | |
| CN112434043B (en) | Data synchronization method, device, electronic equipment and medium | |
| CN111078418B (en) | Operation synchronization method, device, electronic equipment and computer readable storage medium | |
| CN110990365A (en) | Data synchronization method, device, server and storage medium | |
| CN113157426B (en) | Task scheduling method, system, equipment and storage medium | |
| US9817730B1 (en) | Storing request properties to block future requests | |
| US20160050135A1 (en) | Real-time measurement of user interface performance in a remote desktop environment | |
| JP2012089049A (en) | Computer system and server | |
| EP4160421A1 (en) | Method and apparatus for obtaining browser running data, and storage medium | |
| CN115640280A (en) | Data migration method and device | |
| CN112699129A (en) | Data processing system, method and device | |
| CN113238815A (en) | Interface access control method, device, equipment and storage medium | |
| CN112988497A (en) | Method, electronic device and computer program product for managing backup system | |
| US20240143473A1 (en) | Systems and methods for dynamically configuring a client application | |
| CN115629918B (en) | Data processing method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |