CN111628895B - Configuration data synchronization method, device, equipment and readable storage medium - Google Patents

Abstract

The invention discloses a configuration data synchronization method, comprising: detecting whether the configuration data of a target service module is changed; wherein, the configuration data of each control node is stored in the memory pool of each control node; The changed configuration data is sent to other control nodes, so that the other control nodes perform configuration data synchronization on the configuration data stored in the memory pools of the other control nodes according to the changed configuration data. It can be seen that in this application, when synchronizing the configuration data, the original event-driven method is changed to the data synchronization method through the memory pool, so that the processing logic of the business module is separated from the configuration data. This synchronization method is more Universal, easy to expand business modules. The invention also discloses a configuration data synchronization device, equipment and a computer-readable storage medium, which can also achieve the above technical effects.

Description

A configuration data synchronization method, apparatus, device and readable storage medium

technical field

The present invention relates to the technical field of data synchronization, and more particularly, to a configuration data synchronization method, apparatus, device, and computer-readable storage medium.

Background technique

The cluster system in the storage system is a small distributed cluster system of control nodes. The configuration data between the control nodes needs to be synchronized at any time with the business process, and this synchronization is based on events. For example, one of the control nodes needs to be synchronized at any time. When the node receives the configuration command from the user, it needs to send events to other nodes in the cluster to keep the configuration data synchronized between the control nodes in the cluster. The configuration data includes: the information of the configuration data volume, the LBA of each data volume (Logical Block Address, logical block address), the capacity of each data volume, status, and so on. Event-based configuration data synchronization and business logic are usually tightly coupled, and new events need to be defined when adding new business modules, which is not conducive to the expansion of business modules.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a configuration data synchronization method, apparatus, device and computer-readable storage medium, so as to realize the configuration data synchronization between different control nodes.

To achieve the above object, the present invention provides a configuration data synchronization method, comprising:

Detecting whether the configuration data of the target business module has been changed; wherein, the configuration data of each control node is stored in the memory pool of each control node;

If so, the changed configuration data is sent to other control nodes, so that the other control nodes perform configuration data synchronization on the configuration data stored in the memory pools of the other control nodes according to the changed configuration data.

Wherein, before the detecting whether the configuration data of the target business module is changed, the method further includes:

Each control node divides the memory pool applied for by the control node into memory blocks, each memory block has a unique corresponding number of the memory block, and records the correspondence between the corresponding number of each memory block and the memory linear address; the memory pool Created by the kernel's huge page mechanism and exists in RAM;

The service module applies for a memory block from the memory pool of the control node, and receives the memory block corresponding number and the corresponding memory linear address returned by the memory pool; the memory blocks allocated by the service module constitute the service data blocks of the service module, and are different from each other. The service data block has a unique corresponding service data block corresponding number.

Wherein, the sending of the changed configuration data to other control nodes includes:

Send the changed memory block data, the corresponding number of the memory block of the changed memory block, the corresponding number of the business data block, the change time stamp and the change sequence number to other control nodes.

Wherein, the other control nodes perform configuration data synchronization on the configuration data stored in the memory pools of the other control nodes according to the changed configuration data, including:

the other control nodes receive the configuration data;

Cache the configuration data in the memory pools of the other control nodes, and send a change notification to the target service module according to the corresponding number of the service data block, the change time stamp and the change sequence number in the configuration data;

If the return result of the target service module is that it needs to be changed, use the changed memory block data in the configuration data and the corresponding number of the memory block of the changed memory block to compare the corresponding memory blocks in the memory pools of the other control nodes. Perform data synchronization.

Wherein, after caching the configuration data in the memory pools of the other control nodes, the method further includes:

Determine whether the number of cached configuration data reaches a predetermined threshold;

If yes, execute the step of sending a change notification to the target service module according to the corresponding number of the service data block, the change time stamp and the change sequence number in the configuration data.

Wherein, when the change notification is sent to the target service module according to the corresponding number of the service data block, the change time stamp and the change sequence number in the configuration data, the method further includes:

If the quantity of configuration data corresponding to the same service module is greater than one, the change notifications of the configuration data corresponding to the same service module are merged, and only one change notification is sent to the corresponding service module.

The solution further includes: backing up the data in the memory pool to the ROM or the hard disk.

In order to achieve the above object, the present invention further provides a configuration data synchronization device, comprising:

a detection module for detecting whether the configuration data of the target business module is changed; wherein, the configuration data of each control node is stored in the memory pool of each control node;

The sending module is used to send the changed configuration data to other control nodes when the configuration data of the target service module is changed, so that the memory of the other control nodes can be sent to the memory of the other control nodes by the other control nodes according to the changed configuration data. The configuration data stored in the pool is synchronized with the configuration data.

To achieve the above object, the present invention further provides an electronic device, comprising:

The memory is used to store the computer program; the processor is used to implement the steps of the above-mentioned configuration data synchronization method when the computer program is executed.

To achieve the above object, the present invention further provides a computer-readable storage medium, where a computer program is stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned configuration data synchronization method are implemented.

It can be seen from the above solutions that a configuration data synchronization method provided by an embodiment of the present invention includes: detecting whether the configuration data of the target service module is changed; wherein, the configuration data of each control node is stored in the memory of each control node. If yes, send the changed configuration data to other control nodes, so that the other control nodes can synchronize the configuration data stored in the memory pools of the other control nodes according to the changed configuration data. .

It can be seen that in this application, when synchronizing the configuration data, the original event-driven method is changed to the data synchronization method through the memory pool, so that the processing logic of the business module is separated from the configuration data, and this synchronization method is more Universal, easy to expand business modules. The invention also discloses a configuration data synchronization device, equipment and a computer-readable storage medium, which can also achieve the above technical effects.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a schematic flowchart of a configuration data synchronization method disclosed in an embodiment of the present invention;

2 is a schematic structural diagram of a configuration data synchronization apparatus disclosed in an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of an electronic device disclosed in an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Embodiments of the present invention disclose a configuration data synchronization method, apparatus, device, and computer-readable storage medium, so as to realize configuration data synchronization between different control nodes.

Referring to FIG. 1, a configuration data synchronization method provided by an embodiment of the present invention includes:

S101, detecting whether the configuration data of the target service module is changed; wherein, the configuration data of each control node is stored in the memory pool of each control node;

If yes, execute S102; if not, continue to execute S101;

Wherein, before the detecting whether the configuration data of the target business module is changed, the method further includes:

Each control node divides the memory pool applied for by the control node into memory blocks, each memory block has a unique corresponding number of the memory block, and records the correspondence between the corresponding number of each memory block and the memory linear address; the memory pool Created by the kernel's huge page mechanism and exists in RAM;

The service module applies for a memory block from the memory pool of the control node, and receives the memory block corresponding number and the corresponding memory linear address returned by the memory pool; the memory blocks allocated by the service module constitute the service data blocks of the service module, and are different from each other. The service data block has a unique corresponding service data block corresponding number.

In this application, the configuration data is the data that needs to be synchronized by each control node in the cluster, including: data volume information, LBA of each data volume, capacity size, status of each data volume, etc.; and, in this application The business modules are specifically the cache module, protocol processing module, raid module and so on in the storage software system.

It should be noted that, in this application, each control node is provided with a special memory pool DATA POOL, and the configuration data of each control node is stored in the memory pool of each control node. Specifically, when DATA POOL is created, it uses the kernel's huge page mechanism to resident in memory to reduce the performance overhead caused by frequent swaps. DATA POOL applies for a large memory block from the OS (Operating System), and divides it into appropriate memory blocks in the page. Each memory block has a corresponding memory block corresponding number, and records the correspondence between the memory block corresponding number and the memory linear address relation. Further, before the business module performs the operation, it needs to apply for a business data block (WORK DATA CHUNK) to the DATA POOL according to the specified size, and the DATAPOOL will return the corresponding number and linear logical address of the business data block. The business module registers the data block change notification processing logic at the same time when applying for the business data block, which is used to finely control the business process.

Among them, the memory space allocated to the business module by the DATA POOL is composed of multiple memory blocks, and the small blocks that form the business data block are continuous in linear space. Since the data content and data size required by the business module need to be set in advance, and the logic of each control node in the cluster is basically the same, the corresponding relationship between the business module and the corresponding number of the business data block can be set in advance.

S102. Send the changed configuration data to other control nodes, so that the other control nodes perform configuration data synchronization on the configuration data stored in the memory pools of the other control nodes according to the changed configuration data.

The sending of the changed configuration data to other control nodes includes: sending the changed memory block data, the corresponding number of the memory block of the changed memory block, the corresponding number of the service data block, the change time stamp and the change sequence number to other control nodes node.

It should be noted that in this application, when the business module changes the configuration data, it will trigger the DATA POOL to perform a synchronization action, and the DATA POOL will synchronize the changed configuration data (memory blocks and other necessary data) to other control nodes in the cluster, so that , the business module does not need to send specific events to notify other control nodes, which reduces the complexity of business module expansion.

Specifically, the DATA POOL in this application itself does not maintain the cluster relationship. When sending configuration data to other control nodes, it is necessary to obtain the cluster relationship, and then use Ethernet or FC to transfer the changed memory block data and the corresponding number of the memory block. , the corresponding number, timestamp, and change sequence of the business data block are spread to other nodes in the cluster, and other nodes synchronize the configuration data.

It can be seen that in this application, when synchronizing the configuration data, the original event-driven method is changed to the data synchronization method through the memory pool, so that the processing logic of the business module is separated from the configuration data. This synchronization The method is more general, which is convenient for the expansion of business modules.

Based on the foregoing embodiment, in this embodiment, when the other control node performs configuration data synchronization on the configuration data stored in the memory pool of the other control node according to the changed configuration data, it specifically includes:

the other control nodes receive the configuration data;

Cache the configuration data in the memory pools of the other control nodes, and send a change notification to the target service module according to the corresponding number of the service data block, the change time stamp and the change sequence number in the configuration data;

If the return result of the target service module is that it needs to be changed, use the changed memory block data in the configuration data and the corresponding number of the memory block of the changed memory block to compare the corresponding memory blocks in the memory pools of the other control nodes. Perform data synchronization.

It should be noted that, in this application, after other nodes receive a notification of changing configuration information, they will not directly overwrite the corresponding local memory block according to the configuration information, but will cache it first, and then send a change notification to the corresponding business module. When the business logic processes the change notification, if it is determined that the changed data is repeated, it does not need to be overwritten again. Otherwise, the memory block data needs to be overwritten. The memory pool determines whether to overwrite the corresponding memory block through the return value. In this way, repeated coverage of memory blocks can be avoided and data synchronization time can be saved.

Further, in this application, after the configuration data is cached in the memory pools of the other control nodes, the method further includes:

Determine whether the number of cached configuration data reaches a predetermined threshold; if so, execute the step of sending a change notification to the target service module according to the service data block corresponding number, change timestamp and change sequence number in the configuration data. Moreover, if the quantity of configuration data corresponding to the same service module is greater than one, the change notifications of the configuration data corresponding to the same service module are merged, and only one change notification is sent to the corresponding service module.

Specifically, the present application caches all configuration data first, and if the number of cached configuration data reaches a certain number of times or reaches a predetermined processing period, a change notification can be sent to the corresponding service module. In addition, if the application continuously receives changes to the memory blocks in the same service data block, the changes will be combined and processed, and only one change notification will be triggered, which is to avoid malicious attacks. It should be noted that if a new node joins the cluster, a data block request will be sent to the existing node to quickly synchronize the latest configuration data.

It should be noted that the application can also back up the data in the memory pool to a ROM or a hard disk. That is, since the DATA POOL in this application exists in the RAM, there is a risk of power failure and loss, and there is also the problem of data loss caused by the complete extinction of cluster nodes. Therefore, the present application can implement the configuration data synchronization mechanism with the ROM or the hard disk in a timing or policy-based manner. The synchronization mechanism will synchronize the changes to the hard disk according to the change records during this period, and will also synchronize the data to be updated after receiving the change notification to avoid data loss caused by cache problems.

From the above, it can be seen that this solution is mainly to change the synchronization method of configuration data, from the original event-driven synchronization method to a more general synchronization based on memory data blocks, only need to define the business data blocks of different business modules. The corresponding number, and the occupied space of the configuration data of the business module. The business logic processing is separated from the actual configuration data, and the data synchronization participated by the cluster business logic is reduced to the synchronization between the node configuration data, which reduces the risk of the cluster being out of synchronization. Developers are solely responsible for the logic of business data block changes, reducing the need to consider clusters.

The synchronization apparatus provided by the embodiment of the present invention is introduced below, and the synchronization apparatus described below and the synchronization method described above can be referred to each other.

Referring to FIG. 2, a configuration data synchronization apparatus provided by an embodiment of the present invention includes:

The detection module 100 is used to detect whether the configuration data of the target service module is changed; wherein, the configuration data of each control node is stored in the memory pool of each control node;

The sending module 200 is configured to send the changed configuration data to other control nodes when the configuration data of the target service module is changed, so that the other control nodes can send the changed configuration data to the other control nodes according to the changed configuration data. The configuration data stored in the memory pool is synchronized with the configuration data.

Wherein, the device also includes:

The memory pool division module is used for each control node to divide the memory pool applied for by the control node into memory blocks, and each memory block has a unique corresponding memory block number;

a recording module for recording the correspondence between the corresponding number of each memory block and the memory linear address; the memory pool is created by the huge page mechanism of the kernel, and exists in the RAM;

The memory block application module is used for receiving the memory block application from the business module to the memory pool of the control node, and receiving the memory block corresponding number and the corresponding memory linear address returned by the memory pool; the memory blocks allocated by the business module constitute the business The service data block of the module, and different service data blocks have unique corresponding service data block corresponding numbers.

The sending module is specifically used for: sending the changed memory block data, the corresponding number of the memory block of the changed memory block, the corresponding number of the service data block, the change time stamp and the change sequence number to other control nodes.

Among them, other control nodes include synchronization modules, including:

a receiving unit, configured to receive the configuration data;

a cache unit, configured to cache the configuration data in the memory pools of the other control nodes;

a notification sending unit, configured to send a change notification to the target service module according to the corresponding number of the service data block, the change time stamp and the change sequence number in the configuration data;

A synchronization unit, configured to use the changed memory block data in the configuration data and the memory block corresponding number of the changed memory block to update the memory pools of the other control nodes when the return result of the target service module is that it needs to be changed. The corresponding memory block in the data synchronization is performed.

Among them, other control nodes also include:

a judgment module for judging whether the number of cached configuration data reaches a predetermined threshold;

If yes, trigger the notification sending unit to send a change notification to the target service module according to the corresponding number of the service data block, the change time stamp and the change sequence number in the configuration data.

Wherein, if the quantity of configuration data corresponding to the same service module is greater than one, the notification sending unit merges the change notifications of the configuration data corresponding to the same service module, and only sends one change notification to the corresponding service module.

Wherein, the device further includes a backup module for backing up the data in the memory pool to the ROM or the hard disk.

Referring to FIG. 3, an embodiment of the present invention also discloses an electronic device, including:

memory for storing computer programs;

The processor is configured to implement the steps of the configuration data synchronization method described in the above method embodiments when executing the computer program.

In this embodiment, the device may be a PC (Personal Computer, personal computer), or may be a terminal device such as a smart phone, a tablet computer, a palmtop computer, and a portable computer.

The device may include a memory 11 , a processor 12 and a bus 13 .

The memory 11 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (eg, SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, and the like. The memory 11 may in some embodiments be an internal storage unit of the device, such as a hard disk of the device. In other embodiments, the memory 11 may also be an external storage device of the device, such as a plug-in hard disk equipped on the device, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card ( Flash Card), etc. Further, the memory 11 may also include both an internal storage unit of the device and an external storage device. The memory 11 can be used not only to store application software installed in the device and various types of data, such as program codes for executing the synchronization method, etc., but also to temporarily store data that has been output or will be output.

The processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chips in some embodiments, for running program codes or processing stored in the memory 11 Data, such as program code that executes a synchronized method, etc.

The bus 13 may be a peripheral component interconnect (PCI for short) bus or an extended industry standard architecture (Extended industry standard architecture, EISA for short) bus or the like. The bus can be divided into address bus, data bus, control bus and so on. For ease of presentation, only one thick line is used in FIG. 3, but it does not mean that there is only one bus or one type of bus.

Further, the device may also include a network interface 14, and the network interface 14 may optionally include a wired interface and/or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used between the device and other electronic devices Establish a communication connection.

Optionally, the device may further include a user interface, and the user interface may include a display (Display), an input unit such as a keyboard (Keyboard), and an optional user interface may further include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, and the like. Among them, the display may also be appropriately referred to as a display screen or a display unit, for displaying information processed in the device and for displaying a visual user interface.

FIG. 3 only shows the device with components 11-14. Those skilled in the art can understand that the structure shown in FIG. 3 does not constitute a limitation on the device, and may include fewer or more components than those shown. Either some components are combined, or different component arrangements.

Embodiments of the present invention further disclose a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the configuration data synchronization method described in the above method embodiment is implemented. step.

Wherein, the storage medium may include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other various storage media that can store program codes medium.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (7)

1. a configuration data synchronization method, is characterized in that, comprises: Each control node divides the memory pool applied for by the control node into memory blocks, each memory block has a unique corresponding number of the memory block, and records the correspondence between the corresponding number of each memory block and the memory linear address; the memory pool Created by the kernel's huge page mechanism and exists in RAM; The service module applies for a memory block from the memory pool of the control node, and receives the memory block corresponding number and the corresponding memory linear address returned by the memory pool; the memory blocks allocated by the service module constitute the service data blocks of the service module, and are different from each other. The service data block has a unique corresponding service data block corresponding number; Detecting whether the configuration data of the target business module has been changed; wherein, the configuration data of each control node is stored in the memory pool of each control node; If so, send the changed configuration data to other control nodes, so that the other control nodes perform configuration data synchronization on the configuration data stored in the memory pools of the other control nodes according to the changed configuration data; The sending of the changed configuration data to other control nodes includes: sending the changed memory block data, the corresponding number of the memory block of the changed memory block, the corresponding number of the service data block, the change time stamp and the change sequence number to other control nodes node; Wherein, the other control nodes perform configuration data synchronization on the configuration data stored in the memory pools of the other control nodes according to the changed configuration data, including: the other control nodes receive the configuration data; Cache the configuration data in the memory pools of the other control nodes, and send a change notification to the target service module according to the corresponding number of the service data block, the change time stamp and the change sequence number in the configuration data; If the return result of the target service module is that it needs to be changed, use the changed memory block data in the configuration data and the corresponding number of the memory block of the changed memory block to compare the corresponding memory blocks in the memory pools of the other control nodes. Perform data synchronization. 2 . The configuration data synchronization method according to claim 1 , wherein after the configuration data is cached in the memory pools of the other control nodes, the method further comprises: 2 . Determine whether the number of cached configuration data reaches a predetermined threshold; If yes, execute the step of sending a change notification to the target service module according to the corresponding number of the service data block, the change time stamp and the change sequence number in the configuration data. 3. configuration data synchronization method according to claim 2, is characterized in that, described according to the service data block correspondence number in described configuration data, change time stamp and change sequence number when sending change notification to described target service module ,Also includes: If the quantity of configuration data corresponding to the same service module is greater than one, the change notifications of the configuration data corresponding to the same service module are merged, and only one change notification is sent to the corresponding service module. 4 . The configuration data synchronization method according to claim 1 , further comprising: backing up data in the memory pool to a ROM or a hard disk. 5 . 5. A configuration data synchronization device, comprising: a detection module for detecting whether the configuration data of the target business module is changed; wherein, the configuration data of each control node is stored in the memory pool of each control node; The sending module is used to send the changed configuration data to other control nodes when the configuration data of the target service module is changed, so that the memory of the other control nodes can be sent to the memory of the other control nodes by the other control nodes according to the changed configuration data. The configuration data stored in the pool is synchronized with the configuration data; The memory pool division module is used for each control node to divide the memory pool applied for by the control node into memory blocks, and each memory block has a unique corresponding memory block number; a recording module for recording the correspondence between the corresponding number of each memory block and the memory linear address; the memory pool is created by the huge page mechanism of the kernel, and exists in the RAM; The memory block application module is used for receiving the memory block application from the business module to the memory pool of the control node, and receiving the memory block corresponding number and the corresponding memory linear address returned by the memory pool; the memory blocks allocated by the business module constitute the business The business data block of the module, and different business data blocks have unique corresponding business data block corresponding numbers; The sending module is specifically used for: sending the changed memory block data, the corresponding number of the memory block of the changed memory block, the corresponding number of the business data block, the change time stamp and the change sequence number to other control nodes; Among them, other control nodes include synchronization modules, including: a receiving unit, configured to receive the configuration data; a cache unit, configured to cache the configuration data in the memory pools of the other control nodes; a notification sending unit, configured to send a change notification to the target service module according to the corresponding number of the service data block, the change time stamp and the change sequence number in the configuration data; A synchronization unit, configured to use the changed memory block data in the configuration data and the memory block corresponding number of the changed memory block to update the memory pools of the other control nodes when the return result of the target service module is that it needs to be changed. The corresponding memory block in the data synchronization is performed. 6. An electronic device, characterized in that, comprising: memory for storing computer programs; The processor is configured to implement the steps of the configuration data synchronization method according to any one of claims 1 to 4 when executing the computer program. 7. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the configuration according to any one of claims 1 to 4 is realized The steps of the data synchronization method.

Images