CN110928496B - Data processing method and device on multi-control storage system - Google Patents
Data processing method and device on multi-control storage system Download PDFInfo
- Publication number
- CN110928496B CN110928496B CN201911102263.2A CN201911102263A CN110928496B CN 110928496 B CN110928496 B CN 110928496B CN 201911102263 A CN201911102263 A CN 201911102263A CN 110928496 B CN110928496 B CN 110928496B
- Authority
- CN
- China
- Prior art keywords
- target
- data
- controller
- fingerprint
- write cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
- G06F3/0641—De-duplication techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data processing method and device on a multi-control storage system, which can enable a controller to receive a disk refreshing instruction and determine a target fingerprint corresponding to target cache data based on the disk refreshing instruction; acquiring a target storage controller and a target write cache address corresponding to the target fingerprint based on a locally recorded storage controller, the write cache address of the storage controller and the corresponding relation of the fingerprint; and when the controller is not the target storage controller, sending the control information carrying the target fingerprint and the target write cache address to the target storage controller, so that the target storage controller acquires corresponding target cache data in a local write cache according to the control information and performs flash processing. Therefore, when the disk is flushed, the fingerprint of the target cache data and the target write cache address are only transmitted, and the data cannot be transmitted, so that bandwidth resources are prevented from being preempted with mirror image data, and the processing efficiency of the write-in data is improved.
Description
Technical Field
The present application relates to the field of deduplication technologies, and in particular, to a method and an apparatus for processing data on a multi-control storage system.
Background
The multi-control storage system is a storage system with more than two controllers, the more the controllers are, the stronger the processing capacity of the storage system is, and meanwhile, the controllers are mutually hot-standby, so that the reliability is higher. In the multi-control storage system, each controller is provided with an independent CPU, an internal memory and IO interface equipment, so that each controller has independent computing capacity; and each controller can also mutually transmit information, and each controller is connected with the hard disk and can access all storage data in the hard disk space.
In the related art, when data is written in the multi-control storage system, the controller may mirror the locally written data to the controller that is in a hot backup with the controller. When data is flushed, if the controller receiving the disk flushing instruction is not the controller to which the disk flushing data belongs, the disk flushing data corresponding to the disk flushing instruction needs to be sent to the controller to which the disk flushing data belongs for processing, and data transmission between the controllers and transmission of mirror image data compete for bandwidth resources, so that the efficiency of the mirror image data is affected, and the problem of data writing request delay is caused.
Disclosure of Invention
In view of the above, the present application provides a data processing method and apparatus on a multi-control storage system to solve the problem in the prior art that the transmission of the flash data between the controllers competes for the bandwidth with the mirror data.
Specifically, the method is realized through the following technical scheme:
in a first aspect, the present application provides a data processing method on a multi-control storage system, where the method is applied to a controller in the multi-control storage system, and the method includes:
receiving a disk refreshing instruction, and determining a target fingerprint corresponding to target cache data based on the disk refreshing instruction;
acquiring a target storage controller and a target write cache address corresponding to the target fingerprint based on a locally recorded storage controller, the write cache address of the storage controller and the corresponding relation of the fingerprint;
when the controller is not a target storage controller, sending control information carrying the target fingerprint and the target write cache address to the target storage controller, so that the target storage controller obtains corresponding target cache data in a local write cache according to the control information, and performing flash processing on the target cache data.
Based on the same concept, the present application further provides a data processing method on a multi-control storage system, where the method is applied to a controller in the multi-control storage system, and the method includes:
receiving control information which is sent to the controller and comprises a target fingerprint and a target write cache address corresponding to a disk refreshing instruction when other controllers which are connected with the same hard disk in the multi-control storage system are determined to be target storage controllers where the target fingerprints corresponding to the disk refreshing instruction are located;
and acquiring corresponding target cache data in a local write cache according to the control information, and performing flash processing on the target cache data.
In a second aspect, the present application provides a data processing apparatus on a multi-control storage system, the apparatus being applied to a controller in the multi-control storage system, the apparatus including:
the receiving unit is used for receiving a disk refreshing instruction and determining a target fingerprint corresponding to target cache data based on the disk refreshing instruction;
the acquisition unit is used for acquiring a target storage controller and a target write cache address corresponding to the target fingerprint based on a locally recorded storage controller, the write cache address of the storage controller and the corresponding relation of the fingerprint;
and the sending unit is used for sending the control information carrying the target fingerprint and the target write cache address to the target storage controller when the controller is not the target storage controller, so that the target storage controller obtains corresponding target cache data in a local write cache according to the control information and performs flash processing on the target cache data.
Based on the same concept, the present application also provides a data processing apparatus on a multi-control storage system, where the apparatus is applied to a controller in the multi-control storage system, and the apparatus includes:
the receiving unit is used for receiving control information which is sent to the controller and comprises a target fingerprint corresponding to the disk refreshing instruction and a target write cache address when other controllers of the same hard disk connected with the controller in the multi-control storage system determine that the controller is the target storage controller where the target fingerprint corresponding to the disk refreshing instruction is located;
and the processing unit is used for acquiring corresponding target cache data in a local write cache according to the control information and performing flash processing on the target cache data.
In a third aspect, the present application further provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements any step of the above-mentioned data processing method on a multi-control storage system.
In a fourth aspect, the present application further provides a network device, which includes a memory, a controller, a communication interface, and a communication bus; the memory, the controller and the communication interface are communicated with each other through the communication bus;
the memory is used for storing a computer program;
the controller is used for executing the computer program stored in the memory, and the processor realizes any step of the data processing method on the multi-control storage system when executing the computer program.
Therefore, the application provides a data processing method and device on a multi-control storage system, which can enable a controller to receive a disk-brushing instruction and determine a target fingerprint corresponding to target cache data based on the disk-brushing instruction; acquiring a target storage controller and a target write cache address corresponding to the target fingerprint based on a locally recorded storage controller, the write cache address of the storage controller and the corresponding relation of the fingerprint; and when the controller is not the target storage controller, sending the control information carrying the target fingerprint and the target write cache address to the target storage controller, so that the target storage controller acquires corresponding target cache data in a local write cache according to the control information and performs flash processing. Therefore, when the disk is flushed, the fingerprint of the target cache data and the target write cache address are only transmitted, and the data cannot be transmitted, so that bandwidth resources are prevented from being preempted with mirror image data, and the processing efficiency of the write-in data is improved.
Drawings
FIG. 1 is a process flow diagram of a data processing method on a multi-control storage system in an exemplary embodiment of the present application;
FIG. 2 is a process flow diagram of another method for processing data on a multi-control storage system in an exemplary embodiment of the present application;
FIG. 3 is a system architecture diagram in an exemplary embodiment of the present application;
FIG. 4 is a schematic diagram of data interaction in an exemplary embodiment of the present application;
FIG. 5 is a logical block diagram of a data processing apparatus on a multi-control storage system in an exemplary embodiment of the present application;
FIG. 6 is a logical block diagram of another data processing apparatus on a multi-control storage system in an exemplary embodiment of the present application;
fig. 7 is a hardware block diagram of a network device in an exemplary embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
Referring to fig. 1, a flowchart of a data processing method on a multi-control storage system according to an exemplary embodiment of the present application is shown, where the method is applied to a controller in the multi-control storage system, and the method includes:
in this embodiment, each controller in the multi-control storage system is connected to the same hard disk, and can store data into the hard disk, and each controller is provided with a write cache, which can be used for caching the written data. However, since the space of the write cache is limited, the write cache of each controller needs to be flushed according to the flushing condition, and the data in the write cache is written into the hard disk, so that a part of the storage space in the write cache is released, and when a data write instruction is subsequently received, sufficient cache space in the write cache is used for data writing.
The multi-control storage system can divide write cache spaces of a plurality of controllers into a plurality of logical storage blocks LUN, each logical storage block can contain write cache spaces in a plurality of controllers, and the write cache space of one controller can also divide a plurality of logical storage blocks. When the system background determines that the write cache space of the logical storage block to which the controller belongs meets the condition of disk flushing, for example, the remaining storage space is lower than a threshold value, or the written data amount is greater than the remaining storage space, and the like, it may be determined that the controller flushes the write cache space of the logical storage block, so that the controller may receive a disk flushing instruction, where the disk flushing instruction may include a size of a storage space that indicates that the write cache of the logical storage block needs to be released. After receiving a disk-refreshing instruction sent by a background, the controller can release the size of the storage space based on the requirement in the disk-refreshing instruction, and then determine the address of the write cache to be refreshed from the local write cache according to a preset disk-refreshing strategy.
In an embodiment, after the controller acquires the write cache address of the disk to be flushed in the disk flushing instruction, the controller may acquire a fingerprint corresponding to the write cache address of the disk to be flushed based on a locally recorded correspondence between the storage controller, the write cache address of the storage controller, and the fingerprint, and determine the fingerprint as a target fingerprint corresponding to target cache data.
102, acquiring a target storage controller and a target write cache address corresponding to a target fingerprint based on a locally recorded storage controller, the write cache address of the storage controller and the corresponding relation of the fingerprint;
in one embodiment, the method for establishing the correspondence between the storage controller, the write cache address of the storage controller, and the fingerprint specifically includes:
firstly, when a controller receives a data writing instruction, data to be written in the data writing instruction is obtained, and fingerprints of the data to be written are calculated. The fingerprint of the cached data, i.e., the digest of the cached data, is used to uniquely identify the cached data. The fingerprint of the cached data can be calculated by algorithms such as HASH, MD5, etc., and other algorithms with similar functions can also be used, which is not limited herein.
Secondly, after calculating the fingerprint of the data to be written, the controller can determine a target mirror image controller corresponding to the fingerprint of the data to be written according to a preset data storage rule; and mirroring the data to be written into the target mirroring controller for caching.
The specific method for determining the target storage controller and the target mirror image controller by the controller according to the preset data storage rule comprises the following steps: the controller performs remainder on the fingerprints of the data to be written according to the number of the controllers in the multi-control storage system to obtain a remainder N, and the Nth controller is used as a target storage controller of the fingerprints of the data to be written; and according to the number of the controllers in the multi-control storage system, the fingerprints of the data to be written are subjected to remainder taking to obtain a remainder N, and the (N + 1) th controller is used as a target mirror image controller of the fingerprints of the data to be written.
Then, after the controller determines a target storage controller and a target mirror image controller corresponding to the fingerprint of the data to be written, the data to be written can be further sent to the target storage controller for caching; and then receiving a first write cache address fed back by the target storage controller aiming at the data to be written, and establishing a corresponding relation among the target storage controller, the first write cache address and the fingerprint of the data to be written. And mirroring the data to be written into the target mirroring controller for caching, then receiving a second write cache address fed back by the target mirroring controller for the data to be written, and establishing a corresponding relation among the target mirroring controller, the second write cache address and the fingerprint of the data to be written.
By way of example, assume that the multi-control storage system includes four controllers, controller 1, controller 2, controller 3, and controller 4. When the controller 1 receives a data write command, it calculates that the fingerprint of the data to be written is 2, and then divides the fingerprint by 4 to obtain the remainder 2, and the target storage controller should be the controller 2 and the target mirror controller should be the controller 3. The controller 1 can thus store the data to be written to the controller 2 and mirror the data to be written to the controller 3. The controller 2, the controller 3, and the controller 4 write the data into their own write buffers, and feed back corresponding write buffer addresses, address 2, address 3, and address 4, to the controller 1, respectively, so that the correspondence between the target memory controller, the target mirror controller, and the write buffer addresses and fingerprints generated in the controller 1 is as shown in table 1 below.
| Finger print | Target storage controller | Target mirror image controller |
| 2 | Controller 2 | Controller 3 |
| / | Address 2 | Address 3 |
TABLE 1
And 103, when the controller is not a target storage controller, sending control information carrying the target fingerprint and the target write cache address to the target storage controller, so that the target storage controller obtains corresponding target cache data in a local write cache according to the control information, and performing flash processing on the target cache data.
In this embodiment, after the controller determines the target storage controller through calculation, it may be further determined whether the controller is the target storage controller corresponding to the target fingerprint. When the controller is not a target storage controller, sending control information carrying the target fingerprint and the target write cache address to the target storage controller, so that the target storage controller obtains corresponding target cache data in a local write cache according to the control information, and performing flash processing on the target cache data.
In an embodiment, when the target storage controller receives the control information, it may determine whether the target fingerprint is recorded locally, and if so, perform data deduplication according to a set deduplication policy based on target cache data corresponding to the target write cache address; if not, storing the target cache data in a hard disk, and recording the target fingerprint in a deduplication fingerprint library.
The deduplication strategy may include lossy deduplication and lossless deduplication, and may be set according to actual application requirements. When the controller is a target storage controller, data deduplication can be performed according to a set deduplication strategy based on the target cache data, and a specific deduplication process is not a focus of the application and can be processed by referring to an existing deduplication process; if not, the target cache data is not stored in the hard disk, so that the target cache data can be stored in the hard disk, and the target fingerprint is recorded in the deduplication fingerprint library.
As an embodiment, when the controller is a target storage controller, determining whether the target fingerprint exists in a local deduplication fingerprint library; if so, acquiring target cache data corresponding to the target fingerprint, and performing data deduplication based on a set deduplication strategy; if not, acquiring target cache data corresponding to the target fingerprint, storing the target cache data in a hard disk, and recording the target fingerprint in the re-deleted fingerprint library.
According to the method and the device, the target storage controller and the target mirror image controller corresponding to the data can be determined according to the fingerprint of the data, and the corresponding relation of the writing cache addresses of the data in the fingerprint, the target storage controller, the target mirror image controller and the data in the fingerprint and the target storage controller can be recorded, so that the target fingerprint and the target writing cache address corresponding to the disk refreshing instruction can be determined based on the target fingerprint corresponding to the disk refreshing instruction, the target storage controller corresponding to the target cache data is sent to the target storage controller to which the fingerprint belongs, the target storage controller obtains the target cache data of the local cache according to the target writing cache address so as to execute disk refreshing operation, the target cache data does not need to be transmitted between the controllers, and bandwidth resource competition with the mirror image data is avoided.
Referring to fig. 2, a flowchart of another data processing method applied to a controller in a multi-control storage system according to an exemplary embodiment of the present application is shown, where the method includes:
in this embodiment, the controller may receive control information including the target fingerprint and the target write cache address, where the control information is sent to the controller by another controller in the multi-control storage system that is connected to the same hard disk as the controller when the controller is determined not to be the target storage controller.
In this embodiment, the controller may obtain corresponding target data in a local write cache according to the control information, and perform a disk-flushing process on the target data. Specifically, the controller may obtain target cache data in a write cache of the controller according to the target write cache address; judging whether the target fingerprint is in a local duplicate fingerprint database; if so, performing data deduplication processing according to a set deduplication strategy based on the target cache data; if not, storing the target cache data in a hard disk, and recording the target fingerprint in the re-deleted fingerprint database.
Because the controller of this application only sends control information to the target storage controller to make the target storage controller obtain corresponding target data according to control information, and prior art need transmit the required target data of brush dish between the controller, compare in prior art, need not to transmit the data that are used for brushing the dish between the controller of this application, thereby transmission bandwidth when this application can avoid occupying mirror image data improves the treatment effeciency of write-in data.
In order to make the objects, technical solutions and advantages of the present application more apparent, the solution of the present application is further described in detail below with reference to fig. 3 and 4.
Referring to fig. 3, which is a schematic diagram of a system structure in an exemplary embodiment of the present application, for example, the multi-control storage system 30 includes 4 controllers, i.e., a controller 1, a controller 2, a controller 3, and a controller 4, where the structure of each controller is the same, and the system structure includes: the system comprises a front-end IO receiving module, a write cache (cache) module and a rear-end IO sending module. Each controller is connected with the same hard disk, and can access all resources of the hard disk, and the hard disk can be a single hard disk or a hard disk set consisting of a plurality of hard disks, such as a hard disk cabinet. The interaction flow of the controller in the multi-control storage system 30 is shown in fig. 4, which includes:
step 401, the controller 1 receives a data writing instruction, and acquires data to be written in the data writing instruction;
step 402, the controller 1 calculates the fingerprint of the data to be written, and determines a target storage controller and a target mirror image controller according to the fingerprint;
in this embodiment, assuming that the calculated fingerprint is 2, the calculated fingerprint is divided by the total number of the controllers to obtain a remainder 2, and the calculated fingerprint is divided by 2 and 4 to obtain a remainder 2, the target storage controller corresponding to the data to be written is controller 2, and the target mirror image controller is the next controller adjacent to the target storage controller, and the target mirror image controller is determined to be controller 3.
Step 403, sending the data to be written to the controller 2 and the controller 3 for storage;
step 404, the controller 2 and the controller 3 feed back a write cache address corresponding to the data to be written to the controller 1;
step 405, the controller 1 receives the write cache address fed back by the controller, records the fingerprint of the data to be written and the write cache address fed back by the other controller and the corresponding relation between the controller 2 and the controller 3;
if the write cache address of the data to be written in the controller 3 is address 3 and the write cache address of the data to be written in the controller 2 is address 3, the correspondence between the fingerprint established in the controller 1 and the two write cache addresses and the correspondence between the target storage controller and the target mirror controller are shown in table 1.
Step 406, if the controller 1 receives a background disk-brushing instruction;
step 407, determining a target fingerprint corresponding to target cache data based on the disk refreshing instruction;
step 408, the controller 1 obtains a target storage controller and a target write cache address corresponding to the target fingerprint based on the locally recorded storage controller, the write cache address of the storage controller and the corresponding relationship of the fingerprint;
step 409, the controller 1 sends control information including the target fingerprint and the target write cache address to the controller 3;
step 410, the controller 3 may obtain target cache data in the write cache of the controller according to the target write cache address;
step 411, the controller 3 determines whether the target fingerprint in the control information is locally recorded; if yes, go to step 412; if not, go to step 413;
the controller 3 judges whether the target fingerprint is stored or not, if so, the target cache data of the disk brushing is possibly stored in the hard disk, and if not, the target cache data of the disk brushing is not stored in the hard disk.
Step 412, performing data deduplication processing according to a set deduplication strategy based on the target cache data, and ending;
if the target fingerprint exists, it indicates that the target cache data of the disk-brushing may already be stored in the hard disk, and therefore data deduplication needs to be performed according to a preset deduplication strategy based on the acquired target cache data. For example, if the deduplication policy is lossless deduplication, the controller 3 may obtain the copy data in the hard disk based on the target fingerprint, determine whether the copy data in the hard disk is the same as the target cache data, and if so, then the deduplication is successful; if not, then the conflict is deleted again, and the conflicting data is added to the hard disk.
And 413, storing the target cache data in a hard disk, recording the target fingerprint to a re-deleted fingerprint library, and ending.
If the target fingerprint does not exist, the target cache data of the disk is still stored in the hard disk, so that the target cache data is sent to the hard disk through a self rear-end IO sending module, and the fingerprint is recorded in a local deduplication fingerprint library.
Because the controller 1 of the present application can dynamically determine the target storage controller and the target mirror image controller based on the fingerprint of the data to be written, and then when the controller 1 determines that the target storage controller corresponding to the disk-brushing instruction is not self, the controller can only send the control information to the target storage controller, so that the target storage controller obtains the corresponding target cache data according to the control information, and the target data required for disk-brushing needs to be transmitted between the controllers in the prior art, compared with the prior art, the data for disk-brushing does not need to be transmitted between the controllers of the present application, so that the present application can avoid occupying the transmission bandwidth when the mirror image data is occupied, and the processing efficiency of writing the data is improved.
Corresponding to the embodiment of the data processing method on the multi-control storage system, the application also provides an embodiment of the data processing device on the multi-control storage system.
Referring to fig. 5, a schematic structural diagram of a data processing apparatus on a multi-control storage system according to an exemplary embodiment of the present application, where the apparatus is applied to a controller in the multi-control storage system, and the apparatus 50 includes:
a receiving unit 501, configured to receive a disk-swiping instruction, and determine a target fingerprint corresponding to target cache data based on the disk-swiping instruction;
an obtaining unit 502, configured to obtain a target storage controller and a target write cache address corresponding to a target fingerprint based on a locally recorded storage controller, a write cache address of the storage controller, and a corresponding relationship of the fingerprint;
a sending unit 503, configured to send, when the controller is not a target storage controller, control information carrying the target fingerprint and the target write cache address to the target storage controller, so that the target storage controller obtains corresponding target cache data in a local write cache according to the control information, and performs a flash processing on the target cache data.
As an embodiment, the receiving unit 501 is specifically configured to acquire a write cache address of a to-be-flushed disk in a disk flushing instruction, acquire a fingerprint corresponding to the write cache address of the to-be-flushed disk based on a locally recorded correspondence between a storage controller, the write cache address of the storage controller, and the fingerprint, and determine the fingerprint as a target fingerprint corresponding to target cache data.
As an embodiment, the apparatus further comprises:
the establishing unit 504 is configured to, when a data writing instruction is received, acquire data to be written in the data writing instruction, and calculate a fingerprint of the data to be written; determining a target storage controller corresponding to the fingerprint of the data to be written according to a preset data storage rule; sending the data to be written to the target storage controller for caching; and receiving a first write cache address fed back by the target storage controller aiming at the data to be written, and establishing a corresponding relation among the target storage controller, the first write cache address and the fingerprint of the data to be written.
As an embodiment, the apparatus further comprises:
a writing unit 505, configured to determine, according to a preset data storage rule, a target mirror image controller corresponding to the fingerprint of the data to be written, after calculating the fingerprint of the data to be written; mirroring the data to be written into the target mirroring controller for caching; and receiving a second write cache address fed back by the target mirror image controller aiming at the data to be written, and establishing a corresponding relation among the target mirror image controller, the second write cache address and the fingerprint of the data to be written.
As an embodiment, the writing unit 505 is specifically configured to perform remainder extraction on the fingerprint of the data to be written according to the number of controllers in the multi-control storage system to obtain a remainder N, and use an nth controller as a target storage controller of the fingerprint of the data to be written; and according to the number of the controllers in the multi-control storage system, the fingerprints of the data to be written are subjected to remainder taking to obtain a remainder N, and the (N + 1) th controller is used as a target mirror image controller of the fingerprints of the data to be written.
As an embodiment, the apparatus further comprises:
a processing unit 506, configured to determine whether the target fingerprint exists in a local deduplication fingerprint library when the controller is a target storage controller; if so, acquiring target cache data corresponding to the target fingerprint, and performing data deduplication based on a set deduplication strategy; if not, acquiring target cache data corresponding to the target fingerprint, storing the target cache data in a hard disk, and recording the target fingerprint in the re-deleted fingerprint library.
Referring to fig. 6, a schematic structural diagram of another data processing apparatus on a multi-control storage system according to an exemplary embodiment of the present application, where the apparatus is applied to a controller in the multi-control storage system, and the apparatus 60 includes:
a receiving unit 601, configured to receive control information, which is sent to a controller and includes a target fingerprint and a target write cache address corresponding to a disk-flushing instruction, when it is determined that the controller is a target storage controller where the target fingerprint corresponding to the disk-flushing instruction is located, by other controllers connected to a same hard disk in the multi-control storage system;
the processing unit 602 is configured to obtain corresponding target cache data in a local write cache according to the control information, and perform a disk-flushing process on the target cache data.
As an embodiment, the processing unit 602 is specifically configured to:
acquiring target cache data in a write cache of the controller according to the target write cache address;
judging whether the target fingerprint is in a local duplicate fingerprint database;
if so, performing data deduplication processing according to a set deduplication strategy based on the target cache data;
if not, storing the target cache data in a hard disk, and recording the target fingerprint in the re-deleted fingerprint database.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the application. One of ordinary skill in the art can understand and implement it without inventive effort.
Corresponding to the foregoing embodiments of the data processing method on the multi-control storage system, the present application also provides embodiments of a network device implementing the data processing method on the multi-control storage system.
As shown in fig. 7, the network device includes a memory 71, a controller 72, a communication interface 73, and a communication bus 74; the memory 71, the controller 72 and the communication interface 73 communicate with each other through the communication bus 74;
the memory 71 is used for storing computer programs;
the controller 72 is configured to execute the computer program stored in the memory 71, and when the processor 72 executes the computer program, any step of the data processing method on the multi-control storage system provided in the embodiment of the present application is implemented.
The present application further provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements any step of the data processing method on the multi-control storage system provided in the embodiments of the present application.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for embodiments of the network device and the computer-readable storage medium, since they are substantially similar to the method embodiments, the description is relatively simple, and in relation to the description, reference may be made to some portions of the description of the method embodiments.
In summary, the present application provides a data processing method and apparatus on a multi-control storage system, which enable a controller to receive a disk-flushing instruction, and determine a target fingerprint corresponding to target cache data based on the disk-flushing instruction; acquiring a target storage controller and a target write cache address corresponding to the target fingerprint based on a locally recorded storage controller, the write cache address of the storage controller and the corresponding relation of the fingerprint; and when the controller is not the target storage controller, sending the control information carrying the target fingerprint and the target write cache address to the target storage controller, so that the target storage controller acquires corresponding target cache data in a local write cache according to the control information and performs flash processing. Therefore, when the disk is flushed, the fingerprint of the target cache data and the target write cache address are only transmitted, and the data cannot be transmitted, so that bandwidth resources are prevented from being preempted with mirror image data, and the processing efficiency of the write-in data is improved.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the scope of protection of the present application.
Claims (10)
1. A data processing method on a multi-control storage system, wherein the method is applied to a controller in the multi-control storage system, and the method comprises:
receiving a disk refreshing instruction, and determining a target fingerprint corresponding to target cache data based on the disk refreshing instruction;
acquiring a target storage controller and a target write cache address corresponding to the target fingerprint based on a locally recorded storage controller, the write cache address of the storage controller and the corresponding relation of the fingerprint;
when the controller is not a target storage controller, sending control information carrying the target fingerprint and the target write cache address to the target storage controller, so that the target storage controller obtains corresponding target cache data in a local write cache according to the control information, and performing flash processing on the target cache data.
2. The method of claim 1, wherein determining a target fingerprint corresponding to target cache data based on the flash instruction comprises:
the method comprises the steps of obtaining a write cache address of a to-be-refreshed disk in a disk refreshing instruction, obtaining a fingerprint corresponding to the write cache address of the to-be-refreshed disk based on a locally recorded storage controller, the write cache address of the storage controller and a corresponding relation of the fingerprint, and determining the fingerprint as a target fingerprint corresponding to target cache data.
3. The method according to claim 1, wherein the method for establishing the correspondence relationship between the storage controller, the write cache address of the storage controller, and the fingerprint comprises:
when a data writing instruction is received, acquiring data to be written in the data writing instruction, and calculating a fingerprint of the data to be written;
determining a target storage controller corresponding to the fingerprint of the data to be written according to a preset data storage rule;
sending the data to be written to the target storage controller for caching;
and receiving a first write cache address fed back by the target storage controller aiming at the data to be written, and establishing a corresponding relation among the target storage controller, the first write cache address and the fingerprint of the data to be written.
4. The method of claim 3, wherein after computing the fingerprint of the data to be written, the method further comprises:
determining a target mirror image controller corresponding to the fingerprint of the data to be written according to a preset data storage rule;
mirroring the data to be written into the target mirroring controller for caching;
and receiving a second write cache address fed back by the target mirror image controller aiming at the data to be written, and establishing a corresponding relation among the target mirror image controller, the second write cache address and the fingerprint of the data to be written.
5. The method of claim 4,
the determining a target storage controller corresponding to the fingerprint of the data to be written according to a preset data storage rule includes:
according to the number of the controllers in the multi-control storage system, the fingerprints of the data to be written are subjected to remainder taking to obtain a remainder N, and the Nth controller is used as a target storage controller of the fingerprints of the data to be written;
the determining, according to a preset data storage rule, a target mirror image controller corresponding to the fingerprint of the data to be written includes:
and according to the number of the controllers in the multi-control storage system, the fingerprints of the data to be written are subjected to remainder taking to obtain a remainder N, and the (N + 1) th controller is used as a target mirror image controller of the fingerprints of the data to be written.
6. The method of claim 1, further comprising:
when the controller is a target storage controller, judging whether the target fingerprint exists in a local duplicate fingerprint database; if so, acquiring target cache data corresponding to the target fingerprint, and performing data deduplication based on a set deduplication strategy; if not, acquiring target cache data corresponding to the target fingerprint, storing the target cache data in a hard disk, and recording the target fingerprint in the re-deleted fingerprint library.
7. A data processing method on a multi-control storage system, wherein the method is applied to a controller in the multi-control storage system, and the method comprises:
receiving control information which is sent to the controller and comprises a target fingerprint and a target write cache address corresponding to a disk refreshing instruction when other controllers which are connected with the same hard disk in the multi-control storage system are determined to be target storage controllers where the target fingerprints corresponding to the disk refreshing instruction are located;
and acquiring corresponding target cache data in a local write cache according to the control information, and performing flash processing on the target cache data.
8. The method according to claim 7, wherein the obtaining corresponding target data in a local write cache according to the control information and performing a disk flushing process on the target data includes:
acquiring target cache data in a write cache of the controller according to the target write cache address;
judging whether the target fingerprint is in a local duplicate fingerprint database;
if so, performing data deduplication processing according to a set deduplication strategy based on the target cache data;
if not, storing the target cache data in a hard disk, and recording the target fingerprint in the re-deleted fingerprint database.
9. A data processing apparatus on a multi-control storage system, the apparatus being applied to a controller in the multi-control storage system, the apparatus comprising:
the receiving unit is used for receiving a disk refreshing instruction and determining a target fingerprint corresponding to target cache data based on the disk refreshing instruction;
the acquisition unit is used for acquiring a target storage controller and a target write cache address corresponding to the target fingerprint based on a locally recorded storage controller, the write cache address of the storage controller and the corresponding relation of the fingerprint;
and the sending unit is used for sending the control information carrying the target fingerprint and the target write cache address to the target storage controller when the controller is not the target storage controller, so that the target storage controller obtains corresponding target cache data in a local write cache according to the control information and performs flash processing on the target cache data.
10. A data processing apparatus on a multi-control storage system, the apparatus being applied to a controller in the multi-control storage system, the apparatus comprising:
the receiving unit is used for receiving control information which is sent to the controller and comprises a target fingerprint corresponding to the disk refreshing instruction and a target write cache address when other controllers of the same hard disk connected with the controller in the multi-control storage system determine that the controller is the target storage controller where the target fingerprint corresponding to the disk refreshing instruction is located;
and the processing unit is used for acquiring corresponding target cache data in a local write cache according to the control information and performing flash processing on the target cache data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911102263.2A CN110928496B (en) | 2019-11-12 | 2019-11-12 | Data processing method and device on multi-control storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911102263.2A CN110928496B (en) | 2019-11-12 | 2019-11-12 | Data processing method and device on multi-control storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110928496A CN110928496A (en) | 2020-03-27 |
| CN110928496B true CN110928496B (en) | 2022-04-22 |
Family
ID=69852670
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911102263.2A Active CN110928496B (en) | 2019-11-12 | 2019-11-12 | Data processing method and device on multi-control storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110928496B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11762559B2 (en) | 2020-05-15 | 2023-09-19 | International Business Machines Corporation | Write sort management in a multiple storage controller data storage system |
| US11580022B2 (en) | 2020-05-15 | 2023-02-14 | International Business Machines Corporation | Write sort management in a multiple storage controller data storage system |
| CN113010103B (en) * | 2021-01-15 | 2023-03-21 | 腾讯科技（深圳）有限公司 | Data storage method and device, related equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102023809A (en) * | 2009-09-21 | 2011-04-20 | 成都市华为赛门铁克科技有限公司 | Storage system, method for reading data from storage system and method for writing data to storage system |
| CN104461936A (en) * | 2014-11-28 | 2015-03-25 | 华为技术有限公司 | Cached data disk brushing method and device |
| CN106708444A (en) * | 2017-01-17 | 2017-05-24 | 北京联想核芯科技有限公司 | Data storage method and hard disc controller |
| CN107329704A (en) * | 2017-06-30 | 2017-11-07 | 杭州宏杉科技股份有限公司 | One kind caching mirror method and controller |
| CN108459826A (en) * | 2018-02-01 | 2018-08-28 | 杭州宏杉科技股份有限公司 | A kind of method and device of processing I/O Request |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180196834A1 (en) * | 2015-07-30 | 2018-07-12 | Hewlett Packard Enterprise Development Lp | Storing data in a deduplication store |
-
2019
- 2019-11-12 CN CN201911102263.2A patent/CN110928496B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102023809A (en) * | 2009-09-21 | 2011-04-20 | 成都市华为赛门铁克科技有限公司 | Storage system, method for reading data from storage system and method for writing data to storage system |
| CN104461936A (en) * | 2014-11-28 | 2015-03-25 | 华为技术有限公司 | Cached data disk brushing method and device |
| CN106708444A (en) * | 2017-01-17 | 2017-05-24 | 北京联想核芯科技有限公司 | Data storage method and hard disc controller |
| CN107329704A (en) * | 2017-06-30 | 2017-11-07 | 杭州宏杉科技股份有限公司 | One kind caching mirror method and controller |
| CN108459826A (en) * | 2018-02-01 | 2018-08-28 | 杭州宏杉科技股份有限公司 | A kind of method and device of processing I/O Request |
Non-Patent Citations (1)
| Title |
|---|
| MySQL延迟问题和数据刷盘策略;AIOPS_DBA;《https://blog.51cto.com/wangwei007/2416148》;20190702;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110928496A (en) | 2020-03-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110928496B (en) | Data processing method and device on multi-control storage system | |
| CN108459826B (en) | Method and device for processing IO (input/output) request | |
| US9430161B2 (en) | Storage control device and control method | |
| US9507720B2 (en) | Block storage-based data processing methods, apparatus, and systems | |
| US10860494B2 (en) | Flushing pages from solid-state storage device | |
| CN107329704B (en) | Cache mirroring method and controller | |
| CN108319430B (en) | Method and device for processing IO (input/output) request | |
| US10346039B2 (en) | Memory system | |
| CN107193503B (en) | Data deduplication method and storage device | |
| EP2382544B1 (en) | Determining modified data in cache for use during a recovery operation | |
| US10635356B2 (en) | Data management method and storage controller using the same | |
| US20160170646A1 (en) | Implementing enhanced performance flash memory devices | |
| US10515671B2 (en) | Method and apparatus for reducing memory access latency | |
| CN116107516B (en) | Data writing method and device, solid state disk, electronic equipment and storage medium | |
| KR0175983B1 (en) | Data processing system having demand based write through cache with enforced ordering | |
| CN114827178A (en) | Data processing method, device and equipment and readable storage medium | |
| US20210303213A1 (en) | Semiconductor storage device and control method thereof | |
| CN109783000B (en) | Data processing method and equipment | |
| CN116893789B (en) | Data management method, system, device, equipment and computer storage medium | |
| US20210263668A1 (en) | Information processing device and computer-readable recording medium recording storage control program | |
| US20070083708A1 (en) | Controller of redundant arrays of independent disks and operation method thereof | |
| CN108334457B (en) | IO processing method and device | |
| US20140372672A1 (en) | System and method for providing improved system performance by moving pinned data to open nand flash interface working group modules while the system is in a running state | |
| CN110647476B (en) | Method, device and equipment for writing data in solid state disk and storage medium | |
| CN108519860B (en) | SSD read hit processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |