CN117349381A - Data synchronization method, device, management equipment, controller, system and media - Google Patents

Abstract

The invention provides a data synchronization method, a device, a management device, a controller, a system and a medium, wherein the method is applied to the management device and can comprise the following steps: setting a database and bitmaps corresponding to the controllers one by one in the storage equipment; setting a main controller to enable the main controller to independently obtain the read-write permission of the database, and updating the data writing position to the bitmap of the main controller when executing the data writing task to the database; setting a takeover controller when determining that the master controller fails, so that the takeover controller takes over the read-write permission of the database, and synchronizing data in a bitmap of the master controller to the bitmap of the takeover controller, so as to resume execution of the data writing service which is not completed by the master controller according to the bitmap of the takeover controller; the method can set the database shared by all controllers, and can ensure that the taking-over controller can restore the service only by synchronizing the bitmap information which is generated by the main controller and records the writing condition of the database, thereby improving the stability of the multi-control storage array.

Description

Data synchronization method, device, management equipment, controller, system and media

Technical field

The present invention relates to the field of storage, and in particular to data synchronization methods, devices, management equipment, controllers, systems and media.

Background technique

With the explosive growth of data and the development of storage technology, multi-control storage arrays have gradually become a storage method to improve system performance and data reliability. Multi-control storage arrays usually include multiple controllers. These multiple controllers can store storage resource information such as users, hard disks, storage pools, and logical volumes in the storage system in the database to achieve management and control of the multi-control storage array.

In the related art, the database files of each controller are usually stored in the system disks corresponding to each controller. Since the system disks of multiple controllers are isolated from each other, data synchronization means need to be used to synchronize the database files of each controller. This causes the data synchronization cycle and data synchronization algorithm to become more complex when the number of controllers increases, greatly increasing the risk of data loss; more seriously, if database synchronization is in progress when the main controller fails, As a result, the database contents obtained by the primary controller and the backup controller are inconsistent.

Contents of the invention

The purpose of the present invention is to provide a data synchronization method, device, management equipment, controller, system and medium that can set up a database shared by all controllers and ensure that the takeover controller only synchronizes the records generated by the main controller with database writing The business can be restored by inputting the status bitmap information, thereby improving the stability of the multi-control storage array.

In order to solve the above technical problems, the present invention provides a data synchronization method, which is applied to management equipment, including:

Set the database and the bitmap corresponding to each controller in the storage device;

Set the main controller so that the main controller independently obtains the read and write permissions of the database, and when performing a data writing task to the database, updates the data writing position to the bitmap of the main controller middle;

When it is determined that the main controller fails, a takeover controller is set so that the takeover controller takes over the read and write permissions of the database and synchronizes the data in the bitmap of the main controller to the takeover controller. bitmap, so that the unfinished data writing service of the main controller can be resumed according to the bitmap of the takeover controller.

Optionally, before setting the database and the bitmap corresponding to each controller in the storage device, it also includes:

Select multiple hard disk devices in the storage system, and use the multiple hard disk devices to form a disk array, so that the disk array allows at least one hard disk device to be damaged without interrupting storage services;

The disk array is configured as the storage device.

Optionally, using the plurality of hard disk devices to form a disk array includes:

Divide a partition space of a preset size in each of the hard disk devices;

The disk array is constructed using the partition space corresponding to each hard disk device.

Optionally, after setting up the takeover controller, also include:

Send the unfinished data writing service of the main controller to the takeover controller, so that the takeover controller synchronizes the data in the bitmap of the main controller to the takeover controller. After the bitmap, determine the completion progress of the data writing service by the main controller according to the bitmap of the data writing service and the takeover controller, and resume execution of the unfinished work of the main controller according to the completion progress. data writing business.

Optionally, also includes:

Divide the storage space in the storage device into multiple data blocks, and set a one-to-one mapping relationship between each bit in the bitmap and the data blocks;

The mapping relationship information recorded with the mapping relationship is delivered to each of the controllers, so that the main controller writes the data in the bitmap of the main controller according to the data block corresponding to the data writing position. The corresponding bit is updated.

The present invention also provides a data synchronization method, which is applied to the controller. The method includes:

When the managed device is set as the master controller, it independently obtains the read and write permissions of the database in the storage device, and when performing data writing tasks to the database, updates the data writing location to the storage device and itself. Corresponding bitmap; the storage device is provided with a bitmap that corresponds to each controller one-to-one;

When the management device is set to take over the controller, it takes over the read and write permissions of the database and synchronizes the data in the bitmap of the main controller to its own corresponding bitmap so that it can be used according to its own corresponding bitmap. Resume execution of unfinished data writing services of the main controller.

Optionally, after being set by the management device to take over the controller, it also includes:

After receiving the unfinished data writing service of the main controller issued by the management device and completing the synchronization of the data in the bitmap of the main controller to its corresponding bitmap, according to the data The writing service and its corresponding bitmap determine the completion progress of the data writing service by the main controller, and resume execution of the unfinished data writing service by the main controller according to the completion progress.

Optionally, after synchronizing the data in the bitmap of the main controller to its corresponding bitmap, it also includes:

Clears the data in the host controller's bitmap.

Optionally, updating the data writing location to a bitmap corresponding to itself in the storage device includes:

Obtain mapping relationship information; the mapping relationship information records a one-to-one mapping relationship between each bit in the bitmap and each data block in the storage device;

According to the data block corresponding to the data writing position, the corresponding bit in its own bitmap is updated.

The invention also provides a data synchronization device applied to management equipment, including:

The storage device setting module is used to set the database in the storage device and the bitmap corresponding to each controller;

The main controller setting module is used to set the main controller so that the main controller independently obtains the read and write permissions of the database, and when performing a data writing task to the database, updates the data writing position to In the bitmap of the main controller;

A takeover controller setting module, configured to set a takeover controller when it is determined that the main controller is faulty, so that the takeover controller takes over the read and write permissions of the database and changes the bitmap in the main controller to The data is synchronized to the bitmap of the takeover controller, so that the unfinished data writing service of the main controller can be resumed according to the bitmap of the takeover controller.

The invention also provides a data synchronization device applied to a controller, including:

The main controller module is used to independently obtain read and write permissions for the database in the storage device when the managed device is set as the main controller, and when performing data writing tasks to the database, update the data writing location to A bitmap corresponding to itself in the storage device; a bitmap corresponding to each controller in a one-to-one manner is provided in the storage device;

A takeover module, configured to take over the read and write permissions of the database when the management device is set to take over the controller, and synchronize the data in the bitmap of the main controller to its corresponding bitmap so that the data can be processed according to the The corresponding bitmap resumes execution of the unfinished data writing service of the main controller.

The invention also provides a management device, including:

Memory, used to store computer programs;

A processor, configured to implement the above-mentioned data synchronization method applied to management equipment when executing the computer program.

The invention also provides a controller, including:

Memory, used to store computer programs;

A processor, configured to implement the data synchronization method applied to the controller as described above when executing the computer program.

The invention also provides a data synchronization system, including:

Storage device, used to set the database and the bitmap corresponding to each controller;

A management device configured to perform the data synchronization method applied to the management device as described above;

A controller that performs the data synchronization methods applied to the controller as described above.

The present invention also provides a computer-readable storage medium. Computer-executable instructions are stored in the computer-readable storage medium. When the computer-executable instructions are loaded and executed by a processor, they can be applied to management equipment as described above. The data synchronization method and/or the data synchronization method applied to the controller as described above.

The invention provides a data synchronization method, which is applied to management equipment, including: setting a database and a bitmap corresponding to each controller in a storage device; setting a main controller so that the main controller can obtain the The read and write permissions of the database, and when performing data writing tasks to the database, the data writing position is updated to the bitmap of the main controller; when it is determined that the main controller fails, a takeover controller is set, So that the takeover controller takes over the read and write permissions of the database, and synchronizes the data in the bitmap of the main controller to the bitmap of the takeover controller, so that according to the bitmap of the takeover controller Resume execution of unfinished data writing services of the main controller.

It can be seen that the management device in the present invention can first set a database and a bitmap corresponding to each controller in the storage device. There is only one set of databases and is shared by each controller, and the bitmap is used to record the controller. Updates to the database; subsequently, the management device can set the main controller so that the main controller obtains read and write permissions to the database alone, and updates the data writing location to the main controller when performing data writing tasks to the database. in the bitmap; when it is determined that the main controller is faulty, the management device can set the takeover controller so that the takeover controller takes over the read and write permissions of the database and synchronizes the data in the main controller's bitmap to the takeover controller bitmap, so that the unfinished data writing business of the main controller can be resumed based on the bitmap of the takeover controller, that is, the takeover controller only needs to synchronize the bitmap data recorded by the main controller, and can write data based on the bitmap data. The update status of the database by the main controller can be grasped, so that the unfinished data writing business of the main controller can be resumed without complete synchronization of the database; and the bitmap data generated by the main controller is recorded independently of the main controller. On the storage device of the controller, when the main controller fails, the takeover controller can still complete data synchronization, thus greatly improving the stability of the multi-control storage array and reducing the risk of data loss. The invention also provides a data synchronization device, management equipment, controller, system and computer-readable storage medium, which have the above beneficial effects.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only These are embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without exerting creative efforts.

Figure 1 is a structural block diagram of a data synchronization system provided by an embodiment of the present invention;

Figure 2 is a flow chart of a data synchronization method provided by an embodiment of the present invention;

Figure 3 is a flow chart of another data synchronization method provided by an embodiment of the present invention;

Figure 4 is a schematic diagram of data distribution of a RAID1 device with redundancy protection function provided by an embodiment of the present invention;

Figure 5 is a structural block diagram of a data synchronization device provided by an embodiment of the present invention;

Figure 6 is a structural block diagram of another data synchronization device provided by an embodiment of the present invention;

Figure 7 is a structural block diagram of a management device provided by an embodiment of the present invention;

Figure 8 is a structural block diagram of a controller provided by an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

In the related art, the database files of each controller are usually stored in the system disks corresponding to each controller. Since the system disks of multiple controllers are isolated from each other, data synchronization means need to be used to synchronize the database files of each controller. This causes the data synchronization cycle and data synchronization algorithm to become more complex when the number of controllers increases, greatly increasing the risk of data loss; more seriously, if database synchronization is in progress when the main controller fails, As a result, the database contents obtained by the primary controller and the backup controller are inconsistent. In view of this, the present invention can provide a data synchronization method, which can set up a database shared by all controllers, and ensure that the takeover controller can resume business by only synchronizing the bitmap information generated by the main controller with records of database writing. , thereby improving the stability of multi-control storage arrays.

For ease of understanding, please refer to Figure 1 , which is a structural block diagram of a data synchronization system provided by an embodiment of the present invention. This system includes a management device 10, multiple controllers 20 and a storage device 30. Multiple controllers 20 can form a cluster, and the management device 10 can centrally manage the cluster. For example, a main controller and a storage device can be set in the controller 20. Standby controller; the management device 10 is also used to set a database and a bitmap corresponding to each controller in the storage device, where the database is shared by all controllers 20, and the bitmap is used to record the controller 20's response to the database. updates. Specifically, the standby controller can perform read operations on the database, while the primary controller can perform read operations and write operations on the database. When writing data to the database, the main controller can further record the location of the written data in its corresponding bitmap. Furthermore, when the main controller fails, since the writing status of the main controller to the database can be determined based on the bitmap of the main controller, the takeover controller that takes over the main controller can synchronize the bitmap data generated by the main controller to in its own bitmap to recover the unfinished write business of the main controller based on the bitmap data in its own bitmap. Obviously, in the embodiment of the present invention, since each controller 20 does not save the database file independently, but sets a database shared by all controllers 20, and the database is set in a storage device independent of the controller 20, therefore It not only avoids the synchronization of database files between controllers 20, but also avoids the situation where the takeover controller cannot obtain the database files of the main controller when the main controller fails; and, during business recovery, the takeover controller only needs to synchronize the main controller's database files. The bitmap data generated by the controller is sufficient, without the need for complete synchronization of the database, which can greatly reduce the amount of data synchronization. Therefore, embodiments of the present invention can significantly improve the stability of the multi-control storage array and reduce the risk of data loss.

It should be noted that the embodiment of the present invention does not limit whether the management device 10 and the controller 20 exist in the form of physical devices or in the form of virtual machines. They can be set according to actual application requirements. When existing in the form of a physical device, the embodiment of the present invention does not limit the specific physical devices corresponding to the management device 10 and the controller 20, for example, they can be personal computers, servers, etc.; when existing in the form of a virtual machine, the embodiment of the present invention does not limit the specific physical devices corresponding to the management device 10 and the controller 20. The specific type of the physical device on which the management device 10 and the controller 20 are deployed or on which the management device 10 and the controller 20 are deployed is not limited. For example, it may be a personal computer, a server, etc. The embodiment of the present invention does not limit the number of controllers 20, which can be set according to actual application requirements.

Based on the above introduction to the data synchronization system, the data synchronization method applied to the management device provided by the embodiment of the present invention will be introduced in detail below. Please refer to Figure 2, which is a flow chart of a data synchronization method provided by an embodiment of the present invention. This method should be used to manage devices and can include:

S201. Set a database and a bitmap corresponding to each controller in the storage device.

In this embodiment of the present invention, the management device may first set up a database and a bitmap corresponding to each controller in the storage device. Among them, the database is shared by all controllers. For example, all controllers have read permissions to the database, and the main controller further has read and write permissions to the database; and the bitmap is used to save the updates to the database by the corresponding controller. . For the database, the controller in the embodiment of the present invention will not maintain the database file separately, but can use the same database to save important data in the multi-control storage array (such as users, hard disks, storage pools, logical volumes, etc. Storage resource information), so it can avoid the problems of poor stability and easy data loss caused by data synchronization between controllers. For bitmaps, first of all, each controller can independently maintain its corresponding bitmap, which can avoid data confusion; and, because the main controller uses its corresponding bitmap to record updates to the database, it takes over the control of the main controller. The controller can determine the writing status of the main controller to the database based on the bitmap of the main controller. Therefore, after synchronizing the bitmap data generated by the main controller to its own bitmap, the takeover controller can quickly restore the data that the main controller has not written. Completed database writing business. More importantly, since the storage device is independent of each controller, when a controller failure occurs, other controllers can still access bitmaps and databases from independent storage devices, thus avoiding data loss due to controller failure. Acquisition,inconsistency issues.

It should be noted that the embodiments of the present invention are not limited to the specific manner of setting up the database in the storage device. Reference may be made to related technologies of the storage system. For example, the storage device may be formatted into an XFS file system (X File System, a new generation file system). ), and save the database file in the file system to form a database. To ensure that each controller can access the database, the management device can also mount the data set path to each controller. The embodiment of the present invention is not limited to the specific way of setting the bitmap in the storage device. For example, each controller can be allocated a corresponding bitmap space to use each bit in the bitmap space as a bitmap in the bitmap. of everyone.

Furthermore, the storage device in the embodiment of the present invention may be composed of a single hard disk device or multiple hard disk devices. In order to further improve the storage reliability of databases and bitmaps, embodiments of the present invention not only use multiple hard disk devices to form the above storage device, but also use hard disk device components to form a disk array, so that the disk array allows at least one hard disk device to be damaged. The storage service is not interrupted, thus ensuring that the database and bitmaps are not lost when some hard disk devices are damaged. It should be noted that the embodiment of the present invention does not limit the specific form of the array. For example, multiple hard disk device components can be used in RAID1, RAID5, and RAID10 arrays. These three arrays allow at least one hard disk device to be damaged without interrupting storage services. Choose according to actual application needs. The embodiment of the present invention does not limit the specific number of hard disk devices, which can also be set according to actual application requirements.

Based on this, before setting the database and the bitmap corresponding to each controller in the storage device, you can also include:

Step 11: Select multiple hard disk devices in the storage system and use multiple hard disk devices to form a disk array so that the disk array can allow at least one hard disk device to be damaged without interrupting storage services;

Step 12: Set up the disk array as a storage device.

Furthermore, since the above-mentioned database and bitmap do not occupy much space on the hard disk device, in order to avoid resource waste, embodiments of the present invention can also only divide the partition space of a preset size in each hard disk device, and use the corresponding partition space of each hard disk device. Partition space to form a disk array. It should be noted that the embodiment of the present invention does not limit the specific preset size, which can be set according to actual application requirements, for example, it can be 8G.

Based on this, multiple hard disk devices are used to form a disk array, which can include:

Step 21: Divide the partition space of the preset size in each hard disk device;

Step 22: Use the partition space corresponding to each hard disk device to build a disk array.

S202. Set the main controller so that the main controller independently obtains the read and write permissions of the database, and when performing a data writing task to the database, updates the data writing position to the bitmap of the main controller.

After completing the database and bitmap settings, the management device in the embodiment of the present invention selects the main controller among all controllers, so that the main controller independently obtains the read and write permissions of the database. When the main controller performs a data writing task to the database, it can update the data writing position to the bitmap of the main controller.

In order to ensure that the controller can use the bitmap to record the data writing location, when setting up the storage device, the management device can divide the storage space in the storage device into multiple data blocks and set each bit in the bitmap to the data block. The one-to-one mapping relationship between them; then, the management device can send the mapping relationship information recorded with the mapping relationship to each controller, so that the main controller can update the bits of the main controller according to the data block corresponding to the data writing position. The corresponding bits in the figure are updated. It is worth pointing out that when the storage device appears as a disk array, the above-mentioned data block can be a chunk block in a disk array, so that each bit in the bitmap corresponds to the basic storage unit in the disk array for data recovery. This facilitates data recovery.

Based on this, the present invention may also include:

Step 31: Divide the storage space in the storage device into multiple data blocks, and set a one-to-one mapping relationship between each bit in the bitmap and the data blocks;

Step 32: Send the mapping relationship information recorded with the mapping relationship to each controller, so that the main controller updates the corresponding bits in the main controller's bitmap according to the data block corresponding to the data writing position.

S203. When it is determined that the main controller is faulty, set the takeover controller so that the takeover controller takes over the read and write permissions of the database, and synchronizes the data in the bitmap of the main controller to the bitmap of the takeover controller so that the takeover controller can be used according to the takeover control. The bitmap of the controller resumes and executes the unfinished data writing business of the main controller.

When it is determined that the main controller is faulty, the management platform can select a takeover controller from the remaining normally working backup controller to use the takeover controller to take over the main controller. It should be noted that the takeover controller is a special master controller. Before the takeover controller starts writing data to the database, it needs to synchronize the bitmap data generated by the failed main controller to grasp the write and update status of the main controller to the database. After the data synchronization is completed, the takeover controller is officially changed to the new main controller, and then the unfinished data writing business of the main controller can be resumed according to its own bitmap, or it can be determined that the previous main controller has no unfinished data writing. When entering the business, it starts to respond to the new data writing business.

Specifically, after setting up the takeover controller, the management platform can deliver the unfinished data writing business of the main controller to the takeover controller, so that the takeover controller can synchronize the data in the bitmap of the main controller to After taking over the controller's bitmap, the master controller determines the completion progress of the data write service based on the data writing service and the takeover controller's bitmap, and resumes execution of the main controller's unfinished data write service based on the completion progress. Of course, the takeover controller can also obtain the unfinished data writing business of the main controller from the management platform, and can set it according to actual application requirements.

Further, it should be noted that the embodiment of the present invention does not limit how the management platform detects whether the main controller is faulty. For example, a heartbeat mechanism can be set up between the management platform and each controller, and the detection can be based on how each controller responds to heartbeat messages. Determine the operation and fault conditions of each controller.

Based on the above embodiments, the management device in the present invention can first set a database and a bitmap corresponding to each controller in the storage device, where only one set of databases is set and shared by each controller, and the bitmap is used for Record the update of the database by the controller; then, the management device can set the main controller so that the main controller obtains read and write permissions to the database alone, and when performing data writing tasks to the database, the data writing location is updated to in the bitmap of the main controller; when it is determined that the main controller is faulty, the management device can set a takeover controller so that the takeover controller takes over the read and write permissions of the database and synchronizes the data in the bitmap of the main controller to The bitmap of the takeover controller is used to resume execution of the unfinished data writing business of the main controller based on the bitmap of the takeover controller. That is, the takeover controller only needs to synchronize the bitmap data recorded by the main controller and can use the bitmap according to the bitmap of the takeover controller. The graph data can grasp the update status of the database by the main controller, so that the unfinished data writing business of the main controller can be resumed without the need for complete synchronization of the database; and, the bitmap data generated by the main controller is recorded in On a storage device that is independent of the main controller, when the main controller fails, the takeover controller can still complete data synchronization, thus greatly improving the stability of the multi-control storage array and reducing the risk of data loss.

Based on the above embodiments, the data synchronization method applied to the controller provided by the embodiment of the present invention will be introduced in detail below. Please refer to Figure 3. Figure 3 is a flow chart of another data synchronization method provided by an embodiment of the present invention. This method is applied to a controller and may include:

S301. When the managed device is set as the master controller, it independently obtains the read and write permissions of the database in the storage device, and when performing the data writing task to the database, updates the data writing location to the corresponding location in the storage device. Bitmap; a bitmap corresponding to each controller is set in the storage device.

It can be understood that this controller is adjusted from the backup controller to the main controller. As mentioned above, the storage device is provided with bitmaps corresponding to each controller, and these bitmaps are independently maintained by each controller. Therefore, when the managed device is set as the master controller, when the controller performs data writing tasks to the database, it needs to update the data writing location to the bitmap corresponding to itself in the storage device.

In order to ensure that the controller can use the bitmap to record the data writing location, when setting up the storage device, the management device can divide the storage space in the storage device into multiple data blocks and set each bit in the bitmap to the data block. The one-to-one mapping relationship between them; then, the management device can send the mapping relationship information recorded with the mapping relationship to each controller, so that the main controller can update the bits of the main controller according to the data block corresponding to the data writing position. The corresponding bits in the figure are updated.

Based on this, updating the data writing location to the bitmap corresponding to itself in the storage device may include:

Step 41: Obtain the mapping relationship information; the mapping relationship information records the one-to-one mapping relationship between each bit in the bitmap and each data block in the storage device;

Step 42: According to the data block corresponding to the data writing position, update the corresponding bit in its own bitmap.

S302. When the managed device is set to take over the controller, it takes over the read and write permissions of the database, and synchronizes the data in the bitmap of the main controller to its corresponding bitmap, so as to resume execution of the main control according to its corresponding bitmap. The unfinished data writing business of the server.

It is also understandable that this controller is adjusted from a standby controller to a takeover controller. As mentioned above, before starting to write data to the database, the takeover controller needs to synchronize the bitmap data generated by the failed main controller to grasp the write and update status of the main controller to the database. After the data synchronization is completed, the takeover controller is officially changed to the new main controller, and then the unfinished data writing business of the main controller can be resumed according to its own bitmap, or it can be determined that the previous main controller has no unfinished data writing. When entering the business, it starts to respond to the new data writing business.

For the scenario of resuming the data writing service, the controller receives the unfinished data writing service of the main controller from the management device and completes the synchronization of the data in the main controller's bitmap to its own corresponding bitmap. At that time, the main controller's completion progress of the data writing service can be determined based on the data writing service and its corresponding bitmap, and the unfinished data writing service of the main controller can be resumed based on the completion progress. Of course, the takeover controller can also obtain the unfinished data writing business of the main controller from the management platform, and can set it according to actual application requirements.

Furthermore, after completing the synchronization of the bitmap data, in order to prevent the previously failed main controller from coming online again and causing errors in reading the old bitmap data, the takeover controller synchronizes the data in the main controller's bitmap to itself. After corresponding bitmap, you can also clear the data in the bitmap of the main controller.

Based on the above embodiments, the management device in the present invention can first set a database and a bitmap corresponding to each controller in the storage device, where only one set of databases is set and shared by each controller, and the bitmap is used for Record the update of the database by the controller; then, the management device can set the main controller so that the main controller obtains read and write permissions to the database alone, and when performing data writing tasks to the database, the data writing location is updated to in the bitmap of the main controller; when it is determined that the main controller is faulty, the management device can set a takeover controller so that the takeover controller takes over the read and write permissions of the database and synchronizes the data in the bitmap of the main controller to The bitmap of the takeover controller is used to resume execution of the unfinished data writing business of the main controller based on the bitmap of the takeover controller. That is, the takeover controller only needs to synchronize the bitmap data recorded by the main controller and can use the bitmap according to the bitmap of the takeover controller. The graph data can grasp the update status of the database by the main controller, so that the unfinished data writing business of the main controller can be resumed without the need for complete synchronization of the database; and, the bitmap data generated by the main controller is recorded in On a storage device that is independent of the main controller, when the main controller fails, the takeover controller can still complete data synchronization, thus greatly improving the stability of the multi-control storage array and reducing the risk of data loss.

The following is a detailed introduction to the above data synchronization method based on specific examples.

1. Use the first two hard disks in the multi-control storage system to divide 8G partitions to form a RAID1 device, and set the first 512K space of the member disk of the RAID1 device as bitmap space. It mainly includes the following steps:

(1) Select the first two hard disks in the multi-control storage system as the member disks sdb and sdc of the RAID1 device;

(2) Partition sdb and sdc, and divide 8G partition space sdb1 and sdc1;

(3) Combine sdb1 and sdc1 to form a RAID1 device, divide the RAID1 device into specified chunk sizes, and map 1 bit of the bitmap bitmap to the chunk, that is, 1 bit in the bitmap corresponds to the size of 1 chunk in the RAID1 device;

(4) Use bitmap bitmap data to record data changes in RAID1, reserve and set the first 512K space of RAID1 device member disks sdb1 and sdc1 as bitmap space, with data redundancy function.

2. Assign a unique slot number to each controller in the multi-control storage system. Each controller establishes a 4K bitmap bitmap space. The bitmap number corresponds to the controller slot number one-to-one. It mainly includes the following steps:

(1) Each controller in the multi-control storage system has a unique slot number. For example, the slot number of controller 1 is 1, and the slot number of controller N is N;

(2) Each controller maintains a 4K-sized bitmap data, that is, the controller with slot number N corresponds to the bitmap data of bitmapnode00N, recording all write operations and data changes on the controller.

Please refer to FIG. 4 , which is a schematic diagram of data distribution of a RAID1 device with redundancy protection function provided by an embodiment of the present invention. It can be seen that in the sdb and sdc partitions of the two hard disks, the first 512K space is the bitmap space, and each controller corresponds to a 4K-sized bitmap bitmap space. The bitmap number is the same as the controller slot number. One correspondence. At the same time, each bit in the bitmap corresponds to each data block in the partition.

3. Set the read and write priority for the controller to access the RAID1 device, format the RAID1 device as an XFS file system, and start the database service. It mainly includes the following steps:

(1) The main controller has read and write permissions for RAID1 devices, and other alternative controllers have read and write permissions for RAID devices;

(2) Format the RAID1 device into the XFS file system and mount the XFS file system to the system directory of the controller;

(3) Save the database file in the XFS file system directory, start the database service, the main controller can read and write normally, and other alternative controllers can read the database data normally.

4. In response to receiving the database update request, the main controller database service writes the data into the database file and updates its bitmap bitmap data changes at the same time. It mainly includes the following steps:

(1) When the main controller receives a database update request, that is, a write request, the main controller database service writes the data into the database file;

(2) Update the bitmap data changes of bitmap00X corresponding to the main controller and record the write request.

5. In response to the failure of any member disk of the RAID1 device, the other member disk can still provide services to the database normally. It mainly includes the following steps:

(1) When any member disk in the RAID1 device fails, the member disk is set to the fail state (fault state);

(2) Another member disk in the RAID1 device has complete backup data of the failed hard disk and can still continue to provide services to the database.

In response to the failure of the main controller, the takeover controller obtains the read and write permissions of the RAID1 device, copies the bitmap data of the main controller to the bitmap space of this controller, and continues to provide database services.

It mainly includes the following steps:

(1) Take over the controller to obtain the read and write permissions of the RAID1 device, copy the bitmap data of the main controller to the corresponding bitmap space of this controller, and clear the bitmap data of the main controller;

(2) The takeover controller starts RAID1 data synchronization based on the bitmap bitmap data, recovers the data that was not written when the main controller failed, and performs data synchronization to ensure data consistency.

(3) Take over the controller to obtain the read and write permissions of the RAID1 device and continue to provide database services.

The following is an introduction to the data synchronization device, management equipment, controller, data synchronization system and computer-readable storage medium provided by the embodiment of the present invention. The data synchronization device, management equipment, controller, data synchronization system and computer-readable storage medium described below are The media and the data synchronization method described above can be mutually referenced.

Please refer to Figure 5. Figure 5 is a structural block diagram of a data synchronization device provided by an embodiment of the present invention. The device is applied to management equipment and may include:

The storage device setting module 501 is used to set the database and the bitmap corresponding to each controller in the storage device;

The main controller setting module 502 is used to set the main controller so that the main controller independently obtains the read and write permissions of the database, and updates the data writing position to the position of the main controller when performing a data writing task to the database. in the picture;

The takeover controller setting module 503 is used to set the takeover controller when it is determined that the main controller fails, so that the takeover controller takes over the read and write permissions of the database, and synchronizes the data in the bitmap of the main controller to the takeover controller. bitmap, so that the unfinished data writing business of the main controller can be resumed based on the bitmap of the takeover controller.

Optionally, the device may also include:

The disk array component module is used to select multiple hard disk devices in the storage system and use multiple hard disk devices to form a disk array, so that the disk array can allow at least one hard disk device to be damaged without interrupting storage services; set the disk array as a storage device .

Optionally, the disk array component module is specifically used for:

Divide partition space of preset size in each hard disk device;

Use the partition space corresponding to each hard disk device to build a disk array.

Optionally, taking over the controller setting module 503 may also include:

The data writing service delivery module is used to deliver the unfinished data writing service of the main controller to the takeover controller, so that the takeover controller can synchronize the data in the bitmap of the main controller to the takeover controller. After the bitmap, the completion progress of the data writing business by the main controller is determined based on the data writing business and the bitmap of the takeover controller, and the unfinished data writing business of the main controller is resumed based on the completion progress.

Optionally, the device may also include:

The mapping relationship setting module is used to divide the storage space in the storage device into multiple data blocks and set the one-to-one mapping relationship between each bit in the bitmap and the data blocks;

The mapping relationship delivery module is used to deliver the mapping relationship information recorded with the mapping relationship to each controller, so that the main controller can update the corresponding bits in the bitmap of the main controller according to the data block corresponding to the data writing position. Make an update.

Please refer to Figure 6. Figure 6 is a structural block diagram of another data synchronization device provided by an embodiment of the present invention. This device is applied to a controller and may include:

The main controller module 601 is used to independently obtain read and write permissions for the database in the storage device when the managed device is set as the main controller, and when performing data writing tasks to the database, update the data writing location to the storage device. The bitmap corresponding to itself in the device; the bitmap corresponding to each controller is set in the storage device;

The takeover module 602 is used to take over the read and write permissions of the database when the managed device is set to take over the controller, and synchronize the data in the bitmap of the main controller to its corresponding bitmap, so that the data in the bitmap of the main controller can be synchronized according to the corresponding bitmap of the managed device. Resume the execution of the unfinished data writing service of the main controller.

Optionally, the takeover module 602 may also include:

The business recovery submodule is used to receive the unfinished data writing service of the main controller issued by the management device and complete the synchronization of the data in the main controller's bitmap to its corresponding bitmap. According to the data writing The incoming business and its corresponding bitmap determine the completion progress of the data writing business by the main controller, and resume execution of the unfinished data writing business of the main controller based on the completion progress.

Optionally, the device may also include:

Clear module, used to clear the data in the bitmap of the main controller.

Optionally, the main controller module 601 can be used for:

Obtain the mapping relationship information; the mapping relationship information records the one-to-one mapping relationship between each bit in the bitmap and each data block in the storage device; according to the data block corresponding to the data writing position, map the bitmap in its own The corresponding bit is updated.

Please refer to Figure 7, which is a structural block diagram of a management device provided by an embodiment of the present invention. An embodiment of the present invention provides a management device 10, including a processor 11 and a memory 12; wherein, the memory 12, It is used to save the computer program; the processor 11 is used to execute the data synchronization method provided by the previous embodiment when executing the computer program.

Regarding the specific process of the above data synchronization method, reference may be made to the corresponding content provided in the foregoing embodiments, which will not be described again here.

Moreover, the memory 12, as a carrier for resource storage, can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the storage method can be short-term storage or permanent storage.

In addition, the management device 10 also includes a power supply 13, a communication interface 14, an input and output interface 15 and a communication bus 16; wherein the power supply 13 is used to provide operating voltage for each hardware device on the management device 10; The communication interface 14 can create a data transmission channel between the management device 10 and external devices, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present invention, which is not specifically limited here; The input and output interface 15 is used to obtain external input data or output data to the external world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

Please refer to Figure 8. Figure 8 is a structural block diagram of a controller provided by an embodiment of the present invention. An embodiment of the present invention provides a controller 20, including a processor 21 and a memory 22; wherein, the memory 22, It is used to save the computer program; the processor 21 is used to execute the data synchronization method provided by the previous embodiment when executing the computer program.

Regarding the specific process of the above data synchronization method, reference may be made to the corresponding content provided in the foregoing embodiments, which will not be described again here.

Moreover, the memory 22, as a carrier for resource storage, may be a read-only memory, a random access memory, a magnetic disk or an optical disk, and the storage method may be short-term storage or permanent storage.

In addition, the controller 20 also includes a power supply 23, a communication interface 24, an input and output interface 25 and a communication bus 26; wherein the power supply 23 is used to provide working voltage for each hardware device on the controller 20; The communication interface 24 can create a data transmission channel between the controller 20 and external devices, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present invention, which is not specifically limited here; The input and output interface 25 is used to obtain external input data or output data to the external world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

Please refer to Figure 1, which is a structural block diagram of a data synchronization system provided by an embodiment of the present invention. The device may include:

Storage device 30, used to set the database and bitmaps corresponding to each controller;

Management device 10, configured to perform the data synchronization method applied to the management device as described above;

The controller 20 is configured to execute the data synchronization method applied to the controller as described above.

Since the embodiments of the system part and the embodiments of the data synchronization method part correspond to each other, for the embodiments of the storage medium part, please refer to the description of the embodiments of the data synchronization method part, which will not be described again here.

Embodiments of the present invention also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the data synchronization method of any of the above embodiments are implemented.

Since the embodiments of the computer-readable storage medium part correspond to the embodiments of the data synchronization method part, for the embodiments of the storage medium part, please refer to the description of the embodiments of the data synchronization method part, which will not be described again here.

Each embodiment in the specification is described in a progressive manner. Each embodiment focuses on its differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For relevant details, please refer to the description in the method section.

Those skilled in the art may further realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. In order to clearly illustrate the possible functions of hardware and software, Interchangeability, in the above description, the composition and steps of each example have been generally described according to functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered to be beyond the scope of the present invention.

The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination of both. Software modules may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or anywhere in the field of technology. any other known form of storage media.

The data synchronization method, device, management equipment, controller, system and medium provided by the present invention have been introduced in detail above. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and the core idea of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the scope of the claims of the present invention.

Claims (15)

1. A data synchronization method, characterized in that it is applied to management equipment, including: Set the database and the bitmap corresponding to each controller in the storage device; Set the main controller so that the main controller independently obtains the read and write permissions of the database, and when performing a data writing task to the database, updates the data writing position to the bitmap of the main controller middle; When it is determined that the main controller fails, a takeover controller is set so that the takeover controller takes over the read and write permissions of the database and synchronizes the data in the bitmap of the main controller to the takeover controller. bitmap, so that the unfinished data writing service of the main controller can be resumed according to the bitmap of the takeover controller. 2. The data synchronization method according to claim 1, characterized in that, before setting the database and the bitmap corresponding to each controller in the storage device, it also includes: Select multiple hard disk devices in the storage system, and use the multiple hard disk devices to form a disk array, so that the disk array allows at least one hard disk device to be damaged without interrupting storage services; The disk array is configured as the storage device. 3. The data synchronization method according to claim 2, characterized in that said utilizing the plurality of hard disk devices to form a disk array includes: Divide a partition space of a preset size in each of the hard disk devices; The disk array is constructed using the partition space corresponding to each hard disk device. 4. The data synchronization method according to claim 1, characterized in that, after setting the takeover controller, it further includes: Send the unfinished data writing service of the main controller to the takeover controller, so that the takeover controller synchronizes the data in the bitmap of the main controller to the takeover controller. After the bitmap, determine the completion progress of the data writing service by the main controller according to the bitmap of the data writing service and the takeover controller, and resume execution of the unfinished work of the main controller according to the completion progress. data writing business. 5. The data synchronization method according to any one of claims 1 to 4, further comprising: Divide the storage space in the storage device into multiple data blocks, and set a one-to-one mapping relationship between each bit in the bitmap and the data blocks; The mapping relationship information recorded with the mapping relationship is delivered to each of the controllers, so that the main controller writes the data in the bitmap of the main controller according to the data block corresponding to the data writing position. The corresponding bit is updated. 6. A data synchronization method, characterized in that it is applied to a controller, and the method includes: When the managed device is set as the master controller, it independently obtains the read and write permissions of the database in the storage device, and when performing data writing tasks to the database, updates the data writing location to the storage device and itself. Corresponding bitmap; the storage device is provided with a bitmap that corresponds to each controller one-to-one; When the management device is set to take over the controller, it takes over the read and write permissions of the database and synchronizes the data in the bitmap of the main controller to its own corresponding bitmap so that it can be used according to its own corresponding bitmap. Resume execution of unfinished data writing services of the main controller. 7. The data synchronization method according to claim 6, characterized in that, after being set by the management device to take over the controller, it further includes: After receiving the unfinished data writing service of the main controller issued by the management device and completing the synchronization of the data in the bitmap of the main controller to its corresponding bitmap, according to the data The writing service and its corresponding bitmap determine the completion progress of the data writing service by the main controller, and resume execution of the unfinished data writing service by the main controller according to the completion progress. 8. The data synchronization method according to claim 6, characterized in that, after synchronizing the data in the bitmap of the main controller to its corresponding bitmap, it further includes: Clears the data in the host controller's bitmap. 9. The data synchronization method according to any one of claims 6 to 8, wherein the updating of the data writing position to the bitmap corresponding to itself in the storage device includes: Obtain mapping relationship information; the mapping relationship information records a one-to-one mapping relationship between each bit in the bitmap and each data block in the storage device; According to the data block corresponding to the data writing position, the corresponding bit in its own bitmap is updated. 10. A data synchronization device, characterized in that it is applied to management equipment, including: The storage device setting module is used to set the database in the storage device and the bitmap corresponding to each controller; The main controller setting module is used to set the main controller so that the main controller independently obtains the read and write permissions of the database, and when performing a data writing task to the database, updates the data writing position to In the bitmap of the main controller; A takeover controller setting module, configured to set a takeover controller when it is determined that the main controller is faulty, so that the takeover controller takes over the read and write permissions of the database and changes the bitmap in the main controller to The data is synchronized to the bitmap of the takeover controller, so that the unfinished data writing service of the main controller can be resumed according to the bitmap of the takeover controller. 11. A data synchronization device, characterized in that it is applied to a controller and includes: The main controller module is used to independently obtain read and write permissions for the database in the storage device when the managed device is set as the main controller, and when performing data writing tasks to the database, update the data writing location to A bitmap corresponding to itself in the storage device; a bitmap corresponding to each controller in a one-to-one manner is provided in the storage device; A takeover module, configured to take over the read and write permissions of the database when the management device is set to take over the controller, and synchronize the data in the bitmap of the main controller to its corresponding bitmap so that the data can be processed according to the The corresponding bitmap resumes execution of the unfinished data writing service of the main controller. 12. A management device, characterized in that it includes: Memory, used to store computer programs; A processor, configured to implement the data synchronization method according to any one of claims 1 to 5 when executing the computer program. 13. A controller, characterized by comprising: Memory, used to store computer programs; A processor, configured to implement the data synchronization method according to any one of claims 6 to 9 when executing the computer program. 14. A data synchronization system, characterized by including: Storage device, used to set the database and the bitmap corresponding to each controller; Management device, used to perform the data synchronization method according to any one of claims 1 to 5; A controller configured to execute the data synchronization method according to any one of claims 6 to 9. 15. A computer-readable storage medium, characterized in that computer-executable instructions are stored in the computer-readable storage medium. When the computer-executable instructions are loaded and executed by a processor, claims 1 to 5 are implemented. The data synchronization method according to any one of claims 6 to 9.