CN114416885A - Data synchronization method and device based on DRBD, computer equipment and storage medium - Google Patents
Data synchronization method and device based on DRBD, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114416885A CN114416885A CN202210338457.8A CN202210338457A CN114416885A CN 114416885 A CN114416885 A CN 114416885A CN 202210338457 A CN202210338457 A CN 202210338457A CN 114416885 A CN114416885 A CN 114416885A
- Authority
- CN
- China
- Prior art keywords
- synchronization
- data
- mode
- module
- drbd
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application belongs to the field of data processing, and relates to a data synchronization method, a data synchronization device, computer equipment and a storage medium based on a DRBD, wherein the data synchronization method comprises the steps of dividing a data synchronization mode into a normal synchronization mode and an offline synchronization mode; acquiring a network response speed, and starting an offline synchronization mode when the synchronization management module detects that the network response speed is lower than a network response speed threshold, or starting a normal synchronization mode; acquiring a current data synchronization mode; when the current data synchronization mode is the offline synchronization mode, the synchronization management module marks the data blocks related to the write request through a bitmap, the bitmap comprises the data blocks, the modification times of the data blocks are counted, when the data synchronization mode is switched from the offline mode to the normal synchronization mode, the synchronization module sequences according to the modification times of the data blocks, and the synchronization module performs data synchronization according to the modification times of the data blocks from small to large so as to avoid invalid data synchronization as far as possible, so that network resources are utilized more efficiently, and the data synchronization efficiency is accelerated.
Description
Technical Field
The present application relates to the field of data synchronization technologies, and in particular, to a data synchronization method and apparatus based on a DRBD, a computer device, and a storage medium.
Background
Drbd (distributed Replicated Block device) is a distributed Block device replication scheme, and data at one end of a network is Replicated to the other end of the network in a synchronous, semi-synchronous or asynchronous manner, so as to ensure reliability.
As shown in fig. 1 and fig. 2, the DRBD relates to two block devices, which are divided into a master copy and a slave copy, where only the master copy can accept a read/write request of a file system, and the slave copy passively accepts a data synchronization request of the master copy, so as to finally achieve data consistency of the two copies. The DRBD block device is divided into a metadata area and a load data area, wherein the metadata area contains an active journal area which is used for storing data blocks which have recently undergone write operation and can be used for accelerating data synchronization after fault recovery of the block device. After receiving a write request of the file system, the DRBD writes data into a load data area and an active log area, then initiates data synchronization to the secondary copy, and finally completes the write request after the secondary copy data is synchronized, and returns to the file system.
The DRBD currently has a problem that when the primary copy encounters a frequent large data volume write request, the network transmission time is inevitably increased due to data synchronization involving network transmission, so that the return time of the write request is increased, and a file system is jammed or even crashed, and the write performance of the frequent large data volume cannot be optimized.
Disclosure of Invention
An object of the embodiments of the present application is to provide a data synchronization method, an apparatus, a computer device, and a storage medium based on a DRBD, so as to solve the problem that when a master copy encounters a frequent write request with a large data volume, since data synchronization involves network transmission, network transmission time is inevitably increased, thereby increasing the return time of the write request, causing a jam or even a crash to a file system, and failing to optimize write performance with the large data volume.
In order to solve the foregoing technical problem, an embodiment of the present application provides a data synchronization method based on a DRBD, which adopts the following technical solution, including:
a synchronization management module is added in the DRBD, and a data synchronization mode is divided into a normal synchronization mode and an offline synchronization mode;
obtaining a network response speed, and starting the offline synchronization mode when a synchronization management module detects that the network response speed is lower than a network response speed threshold, otherwise, starting the normal synchronization mode;
acquiring a current data synchronization mode;
when the current data synchronization mode is an offline synchronization mode, the synchronization management module marks the data blocks related to the writing request through a bitmap, the bitmap comprises the data blocks, the modification times are accumulated once the data blocks are modified, the modification times of the data blocks are counted, when the data synchronization mode is switched from the offline mode to the normal synchronization mode, the synchronization module sequences according to the modification times of the data blocks, and the synchronization module performs data synchronization according to the modification times of the data blocks from small to large.
Further, when the current data synchronization mode is the normal synchronization mode, the data synchronization of the slave copies is gradually completed through the bitmap, and finally the data consistency of the master copies and the slave copies is achieved.
Further, the bitmap represents a 4KB or 8KB or 16KB block of data with 1 bit.
Further, the bitmap represents a 4KB or 8KB or 16KB block of data with 2 bits.
Further, the synchronization management module includes:
the system comprises a mode judgment module, a normal synchronization module and an offline synchronization module, wherein the mode judgment module is used for selecting a data synchronization mode according to the current network condition; the normal synchronization module is used for directly synchronizing the data modified by the master copy to the slave copy, and the write request is completed after the slave copy responds; the offline synchronization module is used for marking the data blocks related to the modified data by using the bitmap for the master copy and then synchronizing the modified data to the slave copy according to the network condition.
In order to solve the foregoing technical problem, an embodiment of the present application further provides a data synchronization apparatus based on a DRBD, which adopts the following technical solution, including:
the adding module is used for adding a synchronization management module in the DRBD and dividing a data synchronization mode into a normal synchronization mode and an offline synchronization mode;
the judging module is used for acquiring the network response speed, and when the synchronous management module detects that the network response speed is lower than a network response speed threshold value, the off-line synchronous mode is started, otherwise, the normal synchronous mode is started;
the acquisition module is used for acquiring a current data synchronization mode;
and the synchronization management module is used for marking the data blocks related to the writing request through a bitmap when the current data synchronization mode is the offline synchronization mode, the bitmap comprises the data blocks, the modification times are accumulated once the data blocks are modified, the modification times of the data blocks are counted, when the data synchronization mode is switched from the offline mode to the normal synchronization mode, the synchronization module sequences according to the modification times of the data blocks, and the synchronization module performs data synchronization according to the modification times of the data blocks from small to large.
Further, the synchronization management module includes:
the system comprises a mode judgment module, a normal synchronization module and an offline synchronization module, wherein the mode judgment module is used for selecting a data synchronization mode according to the current network condition; the normal synchronization module is used for directly synchronizing the data modified by the master copy to the slave copy, and the write request is completed after the slave copy responds; the offline synchronization module is used for marking the data blocks related to the modified data by using the bitmap for the master copy and then synchronizing the modified data to the slave copy according to the network condition.
Further, the bitmap represents a 4KB or 8KB or 16KB block of data with 1 bit. In order to solve the above technical problem, an embodiment of the present application further provides a computer device, which adopts the following technical solutions:
comprising a memory having computer readable instructions stored therein and a processor that implements the steps of the DRBD-based data synchronization method described above when the processor executes the computer readable instructions.
In order to solve the above technical problem, an embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solutions:
the computer readable storage medium has stored thereon computer readable instructions, which when executed by a processor, implement the steps of the DRBD-based data synchronization method described above.
Compared with the prior art, the embodiment of the application mainly has the following beneficial effects: the data synchronization mode is divided into a normal synchronization mode and an offline synchronization mode by adding the synchronization management module in the DRBD, the problem that the response time of a write request is increased due to the fact that the network delay is increased or the network is busy caused by frequent large data volume write requests can be solved by setting the offline synchronization mode, the response time of the write request is rapid, the file system can be prevented from being blocked or crashed, and good experience is brought to a user.
Drawings
In order to more clearly illustrate the solution of the present application, the drawings needed for describing the embodiments of the present application will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
FIG. 1 is a diagram illustrating a DRBD-based data synchronization architecture in the prior art;
fig. 2 is a schematic diagram illustrating the working principle of data synchronization based on DRBD in the prior art;
FIG. 3 is an exemplary system architecture diagram to which the present application may be applied;
FIG. 4 is a flowchart of one embodiment of a DRBD-based data synchronization method of the present application;
fig. 5 is a schematic diagram illustrating an operation principle of the DRBD-based data synchronization method according to the present application;
fig. 6 is a schematic structural diagram of an embodiment of a DRBD-based data synchronization apparatus according to the present application;
FIG. 7 is a block diagram of one embodiment of a computer device of the present application.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof, in the description and claims of this application and the description of the above figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or in the above-described drawings are used for distinguishing between different objects and not for describing a particular order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
As shown in fig. 3, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have various communication client applications installed thereon, such as a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, and the like.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, E-book readers, MP3 players (Moving Picture E DRBD-based data synchronization property Group Audio Layer III, mpeg compression standard Audio Layer 3), MP4 (Moving Picture E DRBD-based data synchronization property Group Audio Layer IV, mpeg compression standard Audio Layer 4) players, laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that the DRBD-based data synchronization method provided in the embodiments of the present application is generally executed by a server/terminal device, and accordingly, the DRBD-based data synchronization apparatus is generally disposed in the server/terminal device.
It should be understood that the number of terminal devices, networks, and servers in fig. 3 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continuing reference to fig. 4, a flow diagram of one embodiment of a method of DRBD-based data synchronization in accordance with the present application is shown. The data synchronization method based on the DRBD comprises the following steps:
step S201, add a synchronization management module in the DRBD, and divide the data synchronization mode into a normal synchronization mode and an offline synchronization mode.
In the present embodiment, an electronic device (e.g., the server/terminal device shown in fig. 3) on which the DRBD-based data synchronization method operates may receive the DRBD-based data synchronization request through a wired connection manner or a wireless connection manner. It should be noted that the wireless connection means may include, but is not limited to, a 3G/4G/5G connection, a WiFi connection, a bluetooth connection, a wimax DRBD-based data synchronization connection, a Zigbee connection, a uwb (ultra wideband) connection, and other wireless connection means now known or developed in the future.
In this embodiment, the synchronization management module includes: the system comprises a mode judgment module, a normal synchronization module and an offline synchronization module, wherein the mode judgment module is used for selecting a data synchronization mode according to the current network condition; the normal synchronization module is used for directly synchronizing the data modified by the master copy to the slave copy, and the write request is completed after the slave copy responds; the offline synchronization module is used for marking the data blocks related to the modified data by using the bitmap for the master copy and then synchronizing the modified data to the slave copy according to the network condition.
Step S202, obtaining the network response speed, when the synchronization management module detects that the network response speed is lower than the network response speed threshold value, starting the offline synchronization mode, otherwise, starting the normal synchronization mode.
In this embodiment, the network response time may be obtained from the duplicate through ping, and in general, the network response time of the same lan is less than 5 milliseconds, and the specific network response speed threshold may be set according to the actual network topology. The network response speed threshold of the present embodiment is set to 5 milliseconds.
Step S203, acquires the current data synchronization pattern.
In this embodiment, the current data synchronization mode may be divided into an offline synchronization mode and a normal synchronization mode. The current data synchronization mode may be obtained based on current network conditions.
Step S204, when the current data synchronization mode is an offline synchronization mode, the synchronization management module marks the data blocks related to the writing request through a bitmap, the bitmap comprises the data blocks, the modification times are accumulated once the data blocks are modified, the modification times of the data blocks are counted, when the data synchronization mode is switched from the offline mode to the normal synchronization mode, the synchronization module sequences according to the modification times of the data blocks, and the synchronization module performs data synchronization according to the modification times of the data blocks from small to large.
The step of completing data synchronization according to the current data synchronization mode specifically further comprises:
when the current data synchronization mode is an offline synchronization mode, the synchronization management module marks the data block related to the write request through a bitmap, the write request is completed after the data block related to the write request is marked by the bitmap, and the bitmap uses 1 bit to represent the data block of 4 KB. The 1-bit representation of a 4KB data block is a common method at present, and may also represent 8KB, 16KB, 32KB, 64KB, etc. Of course, the bitmap may represent a data block with a size of 4KB, 8KB, 16KB, 32KB, or 64KB with 2 bits, as required. It should be noted that if the data block value represented by 1 bit is set too large, the data amount to be synchronized is also increased, for example, set to 64KB, and even if only 1KB of data is modified, the data amount of 64KB is synchronized. The present embodiment selects a block of bits whose bitmap represents 4KB of data with 1 bit. And counting the modification times of the data block, wherein the modification times are accumulated by one every time the data block is modified. And when the data synchronization mode is switched from the off-line mode to the normal synchronization mode, the synchronization module sequences according to the modification times of the data blocks to form a sequence of the modification times of the data blocks from small to large. If the data synchronization without the difference sequence is executed, it may happen that the data block is modified by the client after being synchronized, and the data synchronization is executed again, which causes a waste of network resources. Because the probability that the data blocks with the large number of modification times are modified again is higher, in order to reduce unnecessary data synchronization, the synchronization module preferentially synchronizes the data blocks with the smaller number of modification times. The data blocks with less modification times are synchronized firstly, and then the data blocks with more modification times are synchronized, so that invalid data synchronization is avoided as much as possible, network resources are utilized more efficiently, and the data synchronization efficiency is accelerated.
When the current data synchronization mode is a normal synchronization mode, the data synchronization of the slave copies is gradually completed through the bitmap, and finally the data consistency of the master copy and the slave copy is achieved, wherein the bitmap uses 1 bit to represent a 4KB data block.
According to the data synchronization method and device, the synchronization management module is added in the DRBD, the data synchronization mode is divided into the normal synchronization mode and the offline synchronization mode, and the synchronization module preferentially synchronizes the data blocks with less data block modification times for reducing unnecessary data synchronization by setting the offline synchronization mode. The data blocks with less modification times are synchronized first, and then the data blocks with more modification times are synchronized, so that invalid data synchronization is avoided as much as possible, network resources are utilized more efficiently, the data synchronization efficiency is improved, the problem that the response time of a write request is increased due to the fact that network delay is increased or the network is busy caused by frequent write requests with large data volumes can be solved, the response time of the write request is rapid, the file system is prevented from being blocked or broken down, and good experience is brought to a user.
In some optional implementation manners of this embodiment, after completing data synchronization according to the current data synchronization mode in step 204, the electronic device may further perform the following steps:
adding md5 value check to the synchronous data, calculating md5 value of the main and auxiliary copy data before synchronization, receiving md5 value of the data calculated from the copy, adding the md5 value calculated from the copy to the response data, extracting md5 value from the main copy after receiving the response from the copy, comparing the md5 value with the md5 value calculated by the main copy, and judging the correctness of the synchronous data after the data synchronization is finished.
According to the method and the device, after the data synchronization is completed according to the current data synchronization mode, the accuracy of the data synchronization is verified in an md5 value verification mode.
Fig. 5 is a schematic diagram illustrating an operation principle of a DRBD-based data synchronization method according to the present application. As shown in fig. 5, when the network is good, the synchronization management module adopts a normal synchronization mode, and the synchronization mode is the same as the conventional mode. Under the condition that the network is busy, the synchronization management module adopts an off-line synchronization mode, a bitmap represents a data block related to a write request, the bitmap represents a data block of 4KB by using 1 bit, the bitmap marks the data block related to the write request, the write request is completed, when the management module detects that the network condition is recovered to a normal state, the data synchronization of the slave copies is gradually completed through the bitmap, and finally the data consistency of the master copy and the slave copy is achieved. And after the network condition is continuously good, the synchronous management module switches the synchronous module into a normal mode.
The application is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware associated with computer readable instructions, which can be stored in a computer readable storage medium, and when executed, the processes of the embodiments of the methods described above can be included. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM).
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
With further reference to fig. 6, as an implementation of the method shown in fig. 4, the present application provides an embodiment of a DRBD-based data synchronization apparatus, which corresponds to the method embodiment shown in fig. 4, and which can be applied in various electronic devices.
As shown in fig. 6, the DRBD-based data sync device 400 according to the present embodiment includes: an adding module 401, a judging module 402, an obtaining module 403 and a synchronizing module 404. Wherein:
an adding module 401, configured to add a synchronization management module to the DRBD, and divide a data synchronization mode into a normal synchronization mode and an offline synchronization mode;
a determining module 402, configured to obtain a network response speed, and when the synchronization management module detects that the network response speed is lower than a network response speed threshold, start the offline synchronization mode, otherwise start the normal synchronization mode;
an obtaining module 403, configured to obtain a current data synchronization mode;
a synchronization module 404, configured to mark, by a bitmap when the current data synchronization mode is an offline synchronization mode, the data block related to the write request, where the bitmap includes the data block, and the number of times of modification of the data block is accumulated by one every time the data block is modified, and count the number of times of modification of the data block, and when the data synchronization mode is switched from the offline mode to the normal synchronization mode, the synchronization module sorts the data blocks according to the number of times of modification of the data blocks, and the synchronization module performs data synchronization according to the number of times of modification of the data blocks.
In some optional implementations of this embodiment, the synchronization management module 411 includes: a mode determining module 4111, a normal synchronizing module 4112, and an offline synchronizing module 4113, wherein the mode determining module is configured to select a data synchronizing mode according to a current network condition; the normal synchronization module is used for directly synchronizing the data modified by the master copy to the slave copy, and the write request is completed after the slave copy responds; the offline synchronization module is used for marking the data blocks related to the modified data by using the bitmap for the master copy and then synchronizing the modified data to the slave copy according to the network condition.
In this embodiment, a synchronization management module is added to the DRBD to divide the data synchronization mode into a normal synchronization mode and an offline synchronization mode, and the synchronization module preferentially synchronizes data blocks with fewer data block modification times by setting the offline synchronization mode to reduce unnecessary data synchronization. The data blocks with less modification times are synchronized first, and then the data blocks with more modification times are synchronized, so that invalid data synchronization is avoided as much as possible, network resources are utilized more efficiently, the data synchronization efficiency is improved, the problem that the response time of a write request is increased due to the fact that network delay is increased or the network is busy caused by frequent write requests with large data volumes can be solved, the response time of the write request is rapid, the file system is prevented from being blocked or broken down, and good experience is brought to a user.
In order to solve the technical problem, an embodiment of the present application further provides a computer device. Referring to fig. 7, fig. 7 is a block diagram of a basic structure of a computer device according to the present embodiment.
The computer device 6 comprises a memory 61, a processor 62, a network interface 63 communicatively connected to each other via a system bus. It is noted that only a computer device 6 having components 61-63 is shown, but it is understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead. As will be understood by those skilled in the art, the computer device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like.
The computer device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The computer equipment can carry out man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch panel or voice control equipment and the like.
The memory 61 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or D DRBD-based data synchronous memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 61 may be an internal storage unit of the computer device 6, such as a hard disk or a memory of the computer device 6. In other embodiments, the memory 61 may also be an external storage device of the computer device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the computer device 6. Of course, the memory 61 may also comprise both an internal storage unit of the computer device 6 and an external storage device thereof. In this embodiment, the memory 61 is generally used for storing an operating system installed in the computer device 6 and various application software, such as computer readable instructions of the DRBD-based data synchronization method. Further, the memory 61 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 62 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 62 is typically used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is configured to execute the computer readable instructions stored in the memory 61 or process data, for example, execute the computer readable instructions of the DRBD-based data synchronization method.
The network interface 63 may comprise a wireless network interface or a wired network interface, and the network interface 63 is typically used for establishing a communication connection between the computer device 6 and other electronic devices.
In the embodiment, the data synchronization mode is divided into the normal synchronization mode and the offline synchronization mode by adding the synchronization management module in the DRBD, and by setting the offline synchronization mode, the problem that the response time of the write request is increased due to the fact that the network delay is increased or the network is busy caused by frequent write requests with large data volume can be solved, the response time of the write request is fast, the file system is prevented from being stuck or crashed, and good experience is brought to a user.
The present application further provides another embodiment, which is to provide a computer-readable storage medium storing computer-readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the DRBD-based data synchronization method as described above.
In this embodiment, a synchronization management module is added to the DRBD to divide the data synchronization mode into a normal synchronization mode and an offline synchronization mode, and the synchronization module preferentially synchronizes data blocks with fewer data block modification times by setting the offline synchronization mode to reduce unnecessary data synchronization. The data blocks with less modification times are synchronized first, and then the data blocks with more modification times are synchronized, so that invalid data synchronization is avoided as much as possible, network resources are utilized more efficiently, the data synchronization efficiency is improved, the problem that the response time of a write request is increased due to the fact that network delay is increased or the network is busy caused by frequent write requests with large data volumes can be solved, the response time of the write request is rapid, the file system is prevented from being blocked or broken down, and good experience is brought to a user.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.
It is to be understood that the above-described embodiments are merely illustrative of some, but not restrictive, of the broad invention, and that the appended drawings illustrate preferred embodiments of the invention and do not limit the scope of the invention. This application is capable of embodiments in many different forms and is provided for the purpose of enabling a thorough understanding of the disclosure of the application. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to one skilled in the art that the present application may be practiced without modification or with equivalents of some of the features described in the foregoing embodiments. All equivalent structures made by using the contents of the specification and the drawings of the present application are directly or indirectly applied to other related technical fields and are within the protection scope of the present application.
Claims (10)
1. A data synchronization method based on DRBD, comprising the steps of:
a synchronization management module is added in the DRBD, and a data synchronization mode is divided into a normal synchronization mode and an offline synchronization mode;
obtaining a network response speed, and starting the offline synchronization mode when a synchronization management module detects that the network response speed is lower than a network response speed threshold, otherwise, starting the normal synchronization mode;
acquiring a current data synchronization mode;
when the current data synchronization mode is an offline synchronization mode, the synchronization management module marks the data blocks related to the writing request through a bitmap, the bitmap comprises the data blocks, the modification times are accumulated once the data blocks are modified, the modification times of the data blocks are counted, when the data synchronization mode is switched from the offline mode to the normal synchronization mode, the synchronization module sequences according to the modification times of the data blocks, and the synchronization module performs data synchronization according to the modification times of the data blocks from small to large.
2. The DRBD-based data synchronization method of claim 1, wherein when the current data synchronization mode is a normal synchronization mode, the data synchronization of the slave copies is gradually completed through a bitmap, and finally the data consistency of the master and slave copies is achieved.
3. The DRBD based data synchronization method of claim 1, wherein:
the bitmap represents a 4KB or 8KB or 16KB block of data with 1 bit.
4. The DRBD based data synchronization method of claim 1, wherein:
the bitmap represents a 4KB or 8KB or 16KB block of data with 2 bits.
5. The DRBD-based data synchronization method according to any one of claims 1 to 4, wherein the synchronization management module comprises:
the system comprises a mode judgment module, a normal synchronization module and an offline synchronization module, wherein the mode judgment module is used for selecting a data synchronization mode according to the current network condition; the normal synchronization module is used for directly synchronizing the data modified by the master copy to the slave copy, and the write request is completed after the slave copy responds; the offline synchronization module is used for marking the data blocks related to the modified data by using the bitmap for the master copy and then synchronizing the modified data to the slave copy according to the network condition.
6. A DRBD-based data synchronization apparatus, comprising:
the adding module is used for adding a synchronization management module in the DRBD and dividing a data synchronization mode into a normal synchronization mode and an offline synchronization mode;
the judging module is used for acquiring the network response speed, and when the synchronous management module detects that the network response speed is lower than a network response speed threshold value, the off-line synchronous mode is started, otherwise, the normal synchronous mode is started;
the acquisition module is used for acquiring a current data synchronization mode;
and the synchronization management module is used for marking the data blocks related to the writing request through a bitmap when the current data synchronization mode is the offline synchronization mode, the bitmap comprises the data blocks, the modification times are accumulated once the data blocks are modified, the modification times of the data blocks are counted, when the data synchronization mode is switched from the offline mode to the normal synchronization mode, the synchronization module sequences according to the modification times of the data blocks, and the synchronization module performs data synchronization according to the modification times of the data blocks from small to large.
7. The DRBD-based data sync device of claim 6, wherein the sync management module comprises:
the system comprises a mode judgment module, a normal synchronization module and an offline synchronization module, wherein the mode judgment module is used for selecting a data synchronization mode according to the current network condition; the normal synchronization module is used for directly synchronizing the data modified by the master copy to the slave copy, and the write request is completed after the slave copy responds; the offline synchronization module is used for marking the data blocks related to the modified data by using the bitmap for the master copy and then synchronizing the modified data to the slave copy according to the network condition.
8. The DRBD-based data synchronization device of claim 6, wherein the bitmap represents a 4KB or 8KB or 16KB data block with 1 bit.
9. A computer device comprising a memory having computer readable instructions stored therein and a processor which when executed implements the steps of the DRBD based data synchronization method of any of claims 1 to 5.
10. A computer readable storage medium having computer readable instructions stored thereon, which when executed by a processor, implement the steps of the DRBD-based data synchronization method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210338457.8A CN114416885A (en) | 2022-04-01 | 2022-04-01 | Data synchronization method and device based on DRBD, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210338457.8A CN114416885A (en) | 2022-04-01 | 2022-04-01 | Data synchronization method and device based on DRBD, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114416885A true CN114416885A (en) | 2022-04-29 |
Family
ID=81263164
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210338457.8A Pending CN114416885A (en) | 2022-04-01 | 2022-04-01 | Data synchronization method and device based on DRBD, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114416885A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115277606A (en) * | 2022-08-01 | 2022-11-01 | 成都安恒信息技术有限公司 | Method for optimizing DRBD data synchronization |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101968755A (en) * | 2010-11-04 | 2011-02-09 | 清华大学 | Application load change adaptive snapshot generating method |
| CN102761615A (en) * | 2012-06-29 | 2012-10-31 | 浪潮(北京)电子信息产业有限公司 | Method and device for realizing data synchronism of long-distance duplication system |
| CN107547306A (en) * | 2017-07-21 | 2018-01-05 | 新华三技术有限公司 | A kind of data syn-chronization restoration methods, device and server |
| CN111831472A (en) * | 2019-04-18 | 2020-10-27 | 阿里巴巴集团控股有限公司 | Snapshot creating method and device and electronic equipment |
| CN112786188A (en) * | 2021-02-05 | 2021-05-11 | 北京致医健康信息技术有限公司 | Offline working method and device of auxiliary diagnosis system, terminal equipment and medium |
| CN113360324A (en) * | 2021-08-10 | 2021-09-07 | 北京华科海讯科技有限公司 | Data backup device based on distributed file data |
| US20210374155A1 (en) * | 2020-05-29 | 2021-12-02 | Vmware, Inc. | Synchronizing changes to stale components of a distributed object using synchronization bitmaps |
-
2022
- 2022-04-01 CN CN202210338457.8A patent/CN114416885A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101968755A (en) * | 2010-11-04 | 2011-02-09 | 清华大学 | Application load change adaptive snapshot generating method |
| CN102761615A (en) * | 2012-06-29 | 2012-10-31 | 浪潮(北京)电子信息产业有限公司 | Method and device for realizing data synchronism of long-distance duplication system |
| CN107547306A (en) * | 2017-07-21 | 2018-01-05 | 新华三技术有限公司 | A kind of data syn-chronization restoration methods, device and server |
| CN111831472A (en) * | 2019-04-18 | 2020-10-27 | 阿里巴巴集团控股有限公司 | Snapshot creating method and device and electronic equipment |
| US20210374155A1 (en) * | 2020-05-29 | 2021-12-02 | Vmware, Inc. | Synchronizing changes to stale components of a distributed object using synchronization bitmaps |
| CN112786188A (en) * | 2021-02-05 | 2021-05-11 | 北京致医健康信息技术有限公司 | Offline working method and device of auxiliary diagnosis system, terminal equipment and medium |
| CN113360324A (en) * | 2021-08-10 | 2021-09-07 | 北京华科海讯科技有限公司 | Data backup device based on distributed file data |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115277606A (en) * | 2022-08-01 | 2022-11-01 | 成都安恒信息技术有限公司 | Method for optimizing DRBD data synchronization |
| CN115277606B (en) * | 2022-08-01 | 2023-11-24 | 成都安恒信息技术有限公司 | Method for optimizing DRBD data synchronization |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111414334A (en) | File fragment uploading method, device, equipment and storage medium based on cloud technology | |
| CN112162965B (en) | Log data processing method, device, computer equipment and storage medium | |
| CN105786405A (en) | Online upgrading method, device and system | |
| CN112631910A (en) | Front-end testing method and device, computer equipment and storage medium | |
| CN112380227A (en) | Data synchronization method, device and equipment based on message queue and storage medium | |
| CN113254445A (en) | Real-time data storage method and device, computer equipment and storage medium | |
| CN112199442A (en) | Distributed batch file downloading method and device, computer equipment and storage medium | |
| WO2015035897A1 (en) | Search methods, servers, and systems | |
| CN103177042A (en) | Method and device for browsing webpage | |
| CN109324874A (en) | A kind of virutal machine memory snapshot imports the method, system and device of block device | |
| EP4095686A2 (en) | Method for switching skin of mini-program page, and electronic device | |
| CN113886332B (en) | Large file difference comparison method and device, computer equipment and storage medium | |
| CN114416885A (en) | Data synchronization method and device based on DRBD, computer equipment and storage medium | |
| CN107422980B (en) | Internet of things data file storage system and data file storage method thereof | |
| CN101388039A (en) | Response message executing method and device | |
| CN113836157A (en) | Method and device for acquiring incremental data of database | |
| CN111475334A (en) | TiDB database maintenance method and device, computer equipment and storage medium | |
| EP4092544A1 (en) | Method, apparatus and storage medium for deduplicating entity nodes in graph database | |
| CN102760212B (en) | Virtual desktop malicious code detecting method based on storage mirroring cloning mechanism | |
| CN114615325A (en) | Message pushing method and device, computer equipment and storage medium | |
| CN114722003A (en) | Centralized data acquisition method, device, equipment and storage medium | |
| CN113342836A (en) | Synchronous data processing method and device, computer equipment and storage medium | |
| CN109547290B (en) | Cloud platform garbage data detection processing method, device, equipment and storage medium | |
| CN111090633A (en) | Small file aggregation method, device and equipment of distributed file system | |
| CN118503298B (en) | Data processing method, device, equipment and storage medium based on double caches |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220429 |
|
| RJ01 | Rejection of invention patent application after publication |