RU2552151C2 - Method of checking correctness of data entry in dual-controller data storage system based on array of nonvolatile media and apparatus therefor - Google Patents

Abstract

FIELD: physics, computer engineering.

SUBSTANCE: invention relates to computer engineering. A method of checking correctness of data entry in a dual-controller data storage system based on an array nonvolatile media, including first and second controllers, each having cache in random-access memory divided into segments, the method comprising: writing data received from an initiator into the cache of the first controller, wherein the data written in the cache are marked as "dirty"; calculating in the cache of the first controller, a check sum of the received data and other "dirty" data stored in the cache; transmitting the data received from the initiator from the first controller to the cache of the second controller, wherein the data written in the cache are marked as "dirty"; calculating in the cache of the second controller a check sum of the received "dirty" data and other "dirty" data stored in the cache; transmitting the check sum calculated in the first controller to the second controller; comparing, in the second controller, the check sums of the first and second controllers; transmitting, from the second controller to the first controller, the check sum comparison result; if the check sums match, the completion of the data synchronisation process is considered correct.

EFFECT: reduced probability of losing "dirty" data upon failure of one of the controllers.

14 cl, 3 dwg

Description

The present invention relates to methods for inputting digital data into non-volatile memory, and more specifically, to methods for checking the correctness of data recording in a dual-controller data storage system based on an array of non-volatile media by synchronizing the operation of two controllers containing a cache.

RAID arrays of non-volatile storage media, such as disks (RAID stands for Redundant Array of Independent Disks, i.e., Redundant Array of Independent Disks), have become the dominant type of storage for large amounts of data. A RAID array is an array of non-volatile media (disks) in which the number of one or more disks is used to store redundant information - a copy of the data or a checksum of data that allows you to restore data if one or more disks fail.

A typical RAID-based storage system contains one or two controllers that provide a consistent, coherent image of the data stored in the repository to all processors connected via connection interfaces.

The two-controller data storage system (Figure 1) is two identical devices, called disk array controllers, each of which is connected to a common disk array. The controllers are interconnected by a hot-plug interface and are also connected to external processors via hot-plug interfaces.

A hot-plug interface is a type of connection interface through which communication can be established at any time during operation of the components it connects.

An external processor is a computer that uses a data storage system to store data and initiates data read and write operations. When using a dual-controller storage system, an external processor must be connected to each controller with at least one hot-plug interface.

The controller may contain a cache-temporary buffer for data in high-speed volatile or non-volatile memory, used to speed up data write and read operations.

Often, a delayed write method is used to speed up data recording, when first the data is received from external processors in the controller cache, and later in the asynchronous process they are written to an array of non-volatile media. In this case, there is a need for a mechanism for synchronizing data between controller caches, such that data that has already entered the cache of one of the controllers also falls into the cache of another controller, since in the absence of such a mechanism a failure of one of the controllers would lead to the loss of data already recorded to the cache, but not yet on non-volatile media. In this synchronization process, it is necessary to be able to detect failures, including those related to the correctness of the synchronized data.

Several methods are known for checking the correctness of data recording used in a data storage system based on arrays of non-volatile media.

In particular, a known method for checking the correctness of data recording, implemented in the device according to the patent US 5911779, publication 15.06.1999, IPC G06F 11/10. In this device, the correctness of data recording is ensured by the calculation of checksums when writing to non-volatile memory - RAID disks. However, the main objective of this invention is to save time when calculating the checksum when writing to RAID5. So, the controller writes the data received from the user to the cache and then sends a confirmation that the data has been written. Further, if the user needs this data, then he gets it from the cache, that is, much faster than reading from disks. The checksum is calculated or recalculated upon direct recording from the cache to disks. As an example of the implementation of this device, a diagram of a device with several controllers is also given. The controllers work with disjoint areas of non-volatile memory addresses and they do not synchronize caches. Therefore, if one of the controllers fails, the other controller cannot provide work with data recorded in the memory area serviced by the failed controller, and there is a possibility of loss of this data.

Known European application EP 0632379, publication 04.01.1994, IPC G06F 11/10, which describes the device fault-tolerant controller system.

The application describes a system that implements the method of operation of two controllers that provide the operation of two arrays of hard drives. Moreover, each controller ensures the operation of its own disk array, therefore, caches are not synchronized between them, since they do not use shared resources. In case of failure of one of the controllers, the remaining one will ensure the operation of both disk arrays, since there is a system of connections between them. However, each controller works without checking the correct storage of data in the controller itself, which can lead to errors in transferring data to non-volatile memory.

The closest analogue is the decision on the application EP 0967552, publication 29.12.1999, IPC G06F 11/20. The application describes a device containing two active controllers, each of which is connected to non-volatile media, each of the controllers containing a cache in RAM consisting of segments, and the controllers are capable of exchanging data with each other. The application describes a method of checking correctness in a dual-controller system for recording data in RAID arrays. The method includes data synchronization operations in controller caches both when writing data to the cache and when reading data while transferring to volatile memory — a RAID array. However, checking the data in the caches is performed only with respect to the current data, therefore, the system will notice a failure only in this current data, concerning one operation of reading or writing data during transfer to the RAID array.

When using the method of deferred recording, data received from external processors in the controller cache, but not yet written to an array of non-volatile media, is called "dirty." The technical problem solved in the claimed invention is to reduce the likelihood of losing "dirty" data in the event of a failure of one of the controllers due to the fact that not only the "dirty" data synchronized between the caches of the two controllers during the current write operation is checked, but also the "dirty" »Data accumulated in caches at the time of the operation.

The method for checking the correctness of data recording works in a data storage system based on an array of non-volatile media, which includes the first and second controllers, each of which is connected to non-volatile media, each of the controllers containing a cache in RAM, consisting of segments, and the controllers are configured data exchange among themselves. The method includes the following operations:

- write the data received from the initiator, for example, an external processor, to the cache of the first controller, while the data written to the cache are marked as "dirty";

- calculate in the cache of the first controller the checksum of the received data and other "dirty" data stored in the cache;

- transmit the data received from the initiator from the first controller to the cache of the second controller, while the data recorded in the cache is marked as "dirty";

- calculate in the cache of the second controller the checksum of the received data and other "dirty" data stored in the cache;

- transmit the checksum calculated in the first controller to the second controller;

- compare the checksums of the first and second controllers in the second controller;

- transmit from the second controller to the first controller the result of comparing the checksums;

- when the checksums match, they decide on the correctness of the completion of the data synchronization operation.

Ensuring the identity of all, and not only related to the same write operation stored in the caches of both dirty data controllers, in case of failure of one of the controllers, the method ensures data correctness both when writing dirty data from the controller cache to non-volatile memory, and when reading "dirty" data from the controller cache by external processors.

New in this method, in relation to the closest analogue, is the lack of a hardware device for managing transactional memory (Memory Transaction Manager). Cache synchronization control is performed by controllers in a software way, which eliminates the need for custom equipment and reduces the cost of the solution.

A significant difference of this method, providing the claimed technical result, is that when this method is implemented, there is no accumulation of data errors stored in the controllers. In the closest analogue to the claimed invention, only the current data related to a separate write cycle to non-volatile memory is checked, therefore, the system will notice a failure only in this data. The claimed system also allows you to detect distortions in all the "dirty" data of the controller cache, for example, a failure in the memory chip. Therefore, the correct storage of all "dirty" data stored in the caches of both controllers is ensured, which provides a greater probability of detecting errors in the data.

Thus, the claimed method can significantly reduce the likelihood of losing data stored in the controller caches.

In the particular case of the implementation of this method, the dirty data stored in the cache segments of the first controller and the dirty data stored in the corresponding cache segments of the second controller are periodically “cleaned”, while:

- write "dirty" data from several segments of the cache of the first controller to an array of non-volatile media;

- remove the “dirty” mark from the data written from the cache of the first controller to an array of non-volatile media;

- calculate the checksum of all "dirty" data in the cache of the first controller;

- transmit from the first controller to the second controller a command to “clear” the cache segments of the second controller corresponding to those cache segments of the first controller on which the “dirty” mark is removed from the data;

- remove the "dirty" mark from the data contained in the "cleared" cache segments of the second controller;

- calculate the checksum of all (remaining) "dirty" data in the cache of the second controller;

- transmit the checksum of all "dirty" cache data of the first controller to the second controller;

- in the second controller, the calculated checksum is compared with the sum received from the first controller;

- transmit from the second controller to the first controller the result of comparing the checksums;

- when the checksums match, they decide on the compliance of the "dirty" data in the first and second controllers.

In particular, if the second controller reports to the first controller about the checksum mismatch, the second controller is restarted.

A RAID array of disks can be used as an array of non-volatile media.

In particular, receive data from at least two external processors, while the recording of the obtained data is performed sequentially.

The data checksum in the first and second controllers can be calculated as a hash function of the data.

In addition, if the first controller fails, the second controller is switched to the first controller mode

In particular, each of the controllers acts as the first or second controller within each separate write operation from the side of the external processor, so if the external processor writes data to one of the controllers, it acts as the first controller in this operation, while the other controller plays the role of the second controller.

Another object of the invention is a device for recording in a data storage system, including non-volatile media, first and second controllers, each of which is connected to non-volatile media, each of the controllers containing a cache in RAM, divided into segments. The controllers are configured to exchange data with each other. In this case, the first controller is configured to record received data received from at least one external initiator, for example, a processor, in the cache, calculate the checksum of the received data and store other dirty data in the cache, and send the received data and the checksum to second controller. The second controller is configured to calculate the checksum of the "dirty" data received from the first controller, and compare the calculated checksum with the amount received from the first controller.

A recording device in a data storage system, as well as a method, reduces the likelihood of data loss due to the fact that it checks not only the "dirty" data stored in the caches of the first and second controllers and related to this write operation, but also the "dirty" »Data accumulated in the cache of each of the controllers at the time of the operation.

In the particular case of the device, the controllers are configured to:

- calculation of the checksum of "dirty" data in the entire cache of the first controller;

- execution in the second controller of the "clear" cache segments received from the first controller;

- removal of “dirty” marks from the data stored in the “cleared” segments in the entire cache of the second controller;

- calculation of the checksum of "dirty" data in the entire cache of the second controller;

- comparison in the second controller of the calculated checksum with the amount received from the first controller.

In particular, the first and second controllers are configured to control a RAID array of disks.

In case of a possible failure of the first controller, the first controller is configured to operate in the mode of the second controller.

In the particular case, both controllers are configured to fulfill the role of the first and second controller within each separate data write operation of the side of the external processor, for example, if the external processor writes data to one of the controllers, it acts as the first controller in this operation, while how the other controller acts as the second controller; in the next operation, the controllers can switch roles.

In particular, the first and second controllers communicate via SCSI.

The method and device are illustrated by diagram and diagrams.

Figure 1 shows a block diagram of a device for a data storage system.

2 is a flowchart of writing data to a data storage system.

Figure 3 shows a sequence diagram for “clearing cache segments.”

The dual-controller data storage system (SHD) 2 provides an external processor 1 or another initiator with access to data via data transfer channels 10 and 11. The data storage system contains two disk array controllers (non-volatile media) 3, 4, each of which includes a random access memory that stores data caches 5, 6. Caches consist of separately addressed segments of equal size 7, 8. Storage controllers 3, 4 are connected to non-volatile memory devices - disks 9 via data transfer channels 13, 14 in such a way for each controller is in communication with each disk.

Controllers 3, 4 are interconnected by a data channel 12, used to synchronize data between controllers 3, 4. Data channels 10, 11 and 12 are hot-plug interfaces.

When recording, the data received by the controller 3 from the external processor 1, goes to cache 5 on channel 10, then transferred to cache 6 of controller 4 on channel 12 and then written to disks 9 on channel 13.

When reading, data falls from disks 9 through channel 13 to cache 5 of controller 3, from where they are transmitted via channel 10 to external processor 1.

It should be noted the symmetry of the data storage system 2, allowing the controllers 3, 4 to change roles if necessary.

The inventive method is implemented as follows.

The inventive method implements the recording of data from an external processor into the controller caches of the dual-controller data storage system and "cleansing" the caches of "dirty" data while storing them on non-volatile storage media.

Recording data from an external processor

When implementing write to storage system 2 from external processor 1 to caches 5, 6 of controllers 3.4. The implementation feature is the unidirectional transmission of data and checksums: the first controller transmits data and checksum to the second controller, the second tells the first the fact of receiving data and the result of comparing the checksums. The flowchart of the algorithm is shown in FIG. 2, then the process is described in detail in blocks.

In block 2-2. The external processor 1 transmits, and the first controller 3 receives the recorded data;

In block 2-3. The first controller 3 places the received data in one or more consecutive segments 7 of the cache 5, while marking this data as “dirty”. Segments are called sequential if they are mapped to sequential data addresses on disks 9. Data labeling can be implemented in various ways, for example, using a bitmap or storing a list of address ranges.

In block 2-4. The first controller 3 calculates Σ1 - the checksum of all the "dirty" data in cache 5, including the data received in block 2. This sum can be calculated in various ways, for example, by a complete resummation of all the "dirty" data, or incrementally using the previously calculated sum.

In block 2-5. The first controller 3 transmits the data received in block 2 to the second controller 4 via the data channel 12.

In block 2-6. The second controller 4 places the received data in one or more consecutive segments 8 of the cache 6, while marking this data as “dirty”.

In block 2-7. The second controller 4 calculates Σ2 - the checksum of all the "dirty" data in the cache 6, including the data received in block 5.

In block 2-8. The second controller 4 informs the first controller 3 about the completion of the operation of receiving data transmitted in block 5.

In block 2-9. The first controller 3 transmits a checksum Σ1 to the second controller 4.

In block 2-10. The second controller 4 compares the checksums Σ1 and Σ2 and stores the result of the comparison.

In block 2-11. The second controller 4 informs the first controller 3 of the result of the checksum comparison: equality or inequality.

In block 2-12. The first controller 3 in the case of equality of checksums transfers control to block 15.

In block 2-13. The first controller 3 stops the synchronization of "dirty" data between the caches 5, 6 of the controllers 3, 4 due to an error.

In block 2-14. The first controller 3 puts the second controller in an error state from which it can be returned into operation, for example, by rebooting;

In block 2-15. The first controller informs the external processor about the completion of the operation of receiving data transmitted in block 2.

"Clearing" the cache when saving data to disks

Part of the method is the implementation of the "cleaning" of "dirty" data in segments 7, 8 of caches 5, 6 of controllers 3, 4 when saving them to disks 9. The necessity of this procedure is explained by the fact that while the data is marked as "dirty", containing their segments it is impossible to use to receive new data, so if you do not “clear” it, over time the cache may become full and the storage system will stop working.

The flowchart of the algorithm is shown in FIG. 3, then the process is described in detail in blocks.

In block 3-2. The first controller 3 stores the "dirty" data from the group of consecutive segments 7 of cache 5 on disks 9.

In block 3-3. The first controller 3 removes the “dirty” mark from the data from the group of segments 7, the data of which was recorded on the disks in block 2.

In block 3-4. The first controller 3 calculates Σ1 - the checksum of all the "dirty" data in the cache 5 after removing the "dirty" mark from part of the data in block 3.

In block 3-5. The first controller 3 sends a “clear” command to the group of segments 8 of the cache 6, corresponding to the “cleared” group of segments 7 of the cache 5, to the second controller 4 via the data channel 12.

In block 3-6. The second controller 4 removes the "dirty" mark from the data from the group of segments 8 of the cache 6.

In block 3-7. The second controller 4 calculates Σ2 - the checksum of all the "dirty" data in the cache 6 after removing the "dirty" mark from part of the data in block 6.

операции «очистки» группы сегментов, инициированной в блоке 5.In block 3-8. The second controller 4 informs the first controller 3 of completion $$$$

operations of "cleaning" a group of segments initiated in block 5.

In block 3-9. The first controller 3 transmits a checksum Σ1 to the second controller 4.

In block 3-10. The second controller 4 compares the checksums Σ1 and Σ2 and stores the result of the comparison.

In block 3-11. The second controller 4 informs the first controller 3 of the result of the checksum comparison: equality or inequality.

In block 12. The first controller 3 in the case of equality of checksums terminates the algorithm.

In block 3-13. The first controller 3 stops the synchronization of "dirty" data between the caches of 5.6 of the controllers 3, 4 due to an error.

In block 3-14. The first controller 3 puts the second controller in an error state from which it can be returned into operation, for example, by rebooting.

The inventive method and device can significantly reduce the likelihood of losing data stored in the controller caches, thereby providing high reliability dual-controller data storage system based on an array of non-volatile media.

Claims (14)

1. A method of checking the correctness of data recording in a two-controller data storage system on an array of non-volatile media, including the first and second controllers, each of which is connected to non-volatile media, each of the controllers containing a cache in RAM, divided into segments, and the controllers are configured exchange of data with each other, the method includes the following operations:
- write the data received from the initiator, for example, an external processor, to the cache of the first controller, while the data written to the cache are marked as "dirty";
- calculate in the cache of the first controller the checksum of the received data and other "dirty" data stored in the cache;
- transmit the data received from the initiator from the first controller to the cache of the second controller, while the data recorded in the cache is marked as "dirty";
- calculate in the cache of the second controller the checksum of the received "dirty" data and stored in the cache of other "dirty"data;
- transmit the checksum calculated in the first controller to the second controller;
- compare the checksums of the first and second controllers in the second controller;
- transmit from the second controller to the first controller the result of comparing the checksums;
- when the checksums match, they decide on the correctness of the completion of the data synchronization operation. 2. The method according to p. 1, in which periodically "clean""dirty" data stored in the cache segments of the first controller, and "dirty" data stored in the corresponding cache segments of the second controller, wherein:
- write "dirty" data from several segments of the cache of the first controller to an array of non-volatile media;
- remove the “dirty” mark from the data written from the cache of the first controller to an array of non-volatile media;
- calculate the checksum of all "dirty" data in the cache of the first controller;
- transmit from the first controller to the second controller a command to "clear" the cache segments of the second controller corresponding to those cache segments of the first controller on which the "dirty" mark was removed from the data;
- remove the "dirty" mark from the data contained in the "cleared" cache segments of the second controller;
- calculate the checksum of all "dirty" data in the cache of the second controller;
- transmit the checksum of all "dirty" cache data of the first controller to the second controller;
- in the second controller, the calculated checksum is compared with the sum received from the first controller;
- transmit from the second controller to the first controller the result of comparing the checksums;
- when the checksums match, they decide on the compliance of the "dirty" data in the first and second controllers. 3. The method according to PP. 1, 2, in which the second controller is restarted if the second controller informs the first controller of the checksum mismatch. 4. The method according to claim 1, wherein a RAID array of disks is used as an array of non-volatile media. 5. The method according to p. 1, in which data is received from at least two external processors, while the recording of the obtained data is performed sequentially. 6. The method of claim 1, wherein the data checksum in the first and second controllers is calculated as a hash function of the data. 7. The method according to p. 1, in which the failure of the first controller switches the second controller to the first controller mode. 8. The method according to claim 1, in which each of the controllers performs the role of the first or second controller within each separate write operation from the side of the external processor, so if the external processor writes data to one of the controllers, it acts as the first controller in this operation while the other controller acts as the second controller. 9. A recording device in a data storage system including non-volatile media, first and second controllers, each of which is connected to non-volatile media, each of the controllers containing a cache in RAM, divided into segments, and the controllers are capable of exchanging data with each other, the first controller is configured to record received data received from at least one external initiator, such as a processor, in the cache, calculate the checksum of the received data and stored in the cache other "dirty" data and direction data received checksum and the second controller; the second controller is configured to calculate the checksum of the "dirty" data received from the first controller and stored in the cache of other "dirty" data, comparing the calculated checksum with the amount received from the first controller. 10. The device according to claim 9, in which the controllers are configured to: calculate a checksum of "dirty" data in the entire cache of the first controller; execution in the second controller of the "clear" cache segments received from the first controller;
removing “dirty” marks from data stored in the “cleared” cache segments of the second controller; calculating the checksum of "dirty" data in the entire cache of the second controller; comparing in the second controller the calculated checksum with the amount received from the first controller. 11. The device according to claim 9, in which the first and second controllers are configured to control a RAID array of disks. 12. The device according to claim 9, in which the first controller is configured to operate in a second controller mode. 13. The device according to claim 9, in which both controllers are configured to act as the first and second controllers for each individual data recording operation from the side of the external processor. 14. The device according to claim 9, in which the first and second controllers communicate via SCSI.

Images