CN115657943A - Data processing method, device and equipment for thin volume of storage system and storage medium - Google Patents
Data processing method, device and equipment for thin volume of storage system and storage medium Download PDFInfo
- Publication number
- CN115657943A CN115657943A CN202211293115.5A CN202211293115A CN115657943A CN 115657943 A CN115657943 A CN 115657943A CN 202211293115 A CN202211293115 A CN 202211293115A CN 115657943 A CN115657943 A CN 115657943A
- Authority
- CN
- China
- Prior art keywords
- thin volume
- target thin
- data
- controller node
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 238000000034 method Methods 0.000 claims abstract description 60
- 238000004891 communication Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 description 16
- 230000006870 function Effects 0.000 description 14
- 230000009977 dual effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000005056 compaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000010897 surface acoustic wave method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a data processing method, a device, equipment and a storage medium for a storage system thin volume, wherein the method comprises the following steps: creating a double-control cluster and a thin volume; the thin volume is provided with a main controller node and a controller node which correspond to each other in the double control cluster respectively; when a main controller node corresponding to a target thin volume receives a write request, distributing metadata of the target thin volume according to address distribution data and capacity statistical data corresponding to the target thin volume; the metadata is used for storing data corresponding to the write request in the target thin volume; updating the address allocation data and the capacity statistical data corresponding to the target thin volume according to the metadata of the target thin volume; and synchronizing the updated address allocation data and the updated capacity statistical data of the target thin volume to a controller node corresponding to the target thin volume. The embodiment of the invention can effectively distribute the metadata of the target thin volume.
Description
Technical Field
The embodiment of the invention relates to the technical field of storage, in particular to a data processing method of a storage system thin volume, a data processing device of the storage system thin volume, electronic equipment and a computer readable storage medium.
Background
To accommodate current customer needs, many storage systems currently provide thin volumes. Specifically, thin provisioning is a volume capacity virtualization technology, and the core idea is that space is allocated during writing, and data is allocated in a manner of additional writing. Therefore, the concept of metadata is introduced, when data is inserted into the thin volume, the metadata is available, and when data is read, the real data is obtained by reading the disk based on the metadata.
At present, most of storage system compact volumes are implemented based on dual control, one controller node is used as a main controller node, namely an Owner end, and the controller provides data read-write service, and the other controller is used as a Backup controller node, namely a Backup end, and provides a mirror image Backup function of data and metadata, and when the main controller node fails, service takeover is performed. The metadata is distributed according to address distribution and capacity statistics of the thin volume, the address distribution and the capacity statistics of the thin volume are both on a single controller node in a normal IO (Input/Output) writing process, and when a single controller node, for example, a main controller node, fails, there is a problem how to effectively perform the address distribution and the capacity statistics of the thin volume.
Disclosure of Invention
Embodiments of the present invention provide a data processing method and apparatus for a thin volume of a storage system, an electronic device, and a computer-readable storage medium, so as to solve the problem of how to effectively perform address allocation and capacity statistics for the thin volume.
The embodiment of the invention discloses a data processing method for a storage system thin volume, which comprises the following steps:
creating a dual-control cluster and a thin volume for a storage system; the thin volume is provided with a main controller node and a controller node which correspond to each other in the double-control cluster respectively;
when a main controller node corresponding to a target thin volume receives a write request, distributing metadata of the target thin volume according to address distribution data and capacity statistical data corresponding to the target thin volume; the metadata is used for storing data corresponding to the write request in the target thin volume;
updating the address allocation data and the capacity statistical data corresponding to the target thin volume according to the metadata of the target thin volume;
and synchronizing the updated address allocation data and the updated capacity statistical data of the target thin volume to a controller node corresponding to the target thin volume.
In an optional embodiment of the present invention, after the synchronizing the updated address allocation data and the updated capacity statistics data of the target thin volume to the controller node corresponding to the target thin volume, the method further includes:
and when the main controller node corresponding to the target thin volume fails, taking the controller node corresponding to the target thin volume as the main controller node of the target thin volume.
In an optional embodiment of the present invention, the allocating, when a host controller node corresponding to a target thin volume receives a write request, metadata of the target thin volume according to address allocation data and capacity statistics data corresponding to the target thin volume includes:
when a main controller node corresponding to a target thin volume receives a write request, distributing data addresses and metadata addresses of the target thin volume according to address distribution data and capacity statistical data corresponding to the target thin volume; the data address is an address for storing data corresponding to the write request in the target thin volume, and the metadata address is used for indexing to the data address.
In an optional embodiment of the present invention, before the allocating, when a host controller node corresponding to a target thin volume receives a write request, metadata of the target thin volume according to address allocation data and capacity statistics data corresponding to the target thin volume, the method further includes:
when the controller node corresponding to the target thin volume receives a write request, the master controller node corresponding to the target thin volume receives the write request forwarded by the controller node corresponding to the target thin volume.
In an optional embodiment of the present invention, after the allocating, when the master controller node corresponding to the target thin volume receives the write request, the metadata of the target thin volume according to the address allocation data and the capacity statistics data corresponding to the target thin volume, the method further includes:
and storing the metadata of the target thin volume according to a B + Tree structure.
In an optional embodiment of the present invention, after the allocating, when a master controller node corresponding to a target thin volume receives a write request, metadata of the target thin volume according to address allocation data and capacity statistics data corresponding to the target thin volume, the method further includes:
and synchronizing the metadata of the target thin volume to a controller node corresponding to the target thin volume.
In an optional embodiment of the present invention, after the allocating, when a master controller node corresponding to a target thin volume receives a write request, metadata of the target thin volume according to address allocation data and capacity statistics data corresponding to the target thin volume, the method further includes:
when a main controller node corresponding to the target thin volume receives a read request, acquiring a target metadata address from the read request;
determining a target data address corresponding to the target metadata address from the metadata of the target thin volume;
and reading corresponding data from the target thin volume according to the target data address.
The embodiment of the invention also discloses a data processing device of the storage system thin volume, which comprises:
the creation module is used for creating the double-control cluster and the thin volume aiming at the storage system; the thin volume is provided with a main controller node and a controller node which correspond to each other in the double-control cluster respectively;
the allocation module is used for allocating the metadata of the target thin volume according to the address allocation data and the capacity statistical data corresponding to the target thin volume when the main controller node corresponding to the target thin volume receives the write request; the metadata is used for storing data corresponding to the write request in the target thin volume;
an updating module, configured to update the address allocation data and the capacity statistics data corresponding to the target thin volume according to the metadata of the target thin volume;
and the synchronization module is used for synchronizing the updated address allocation data and the updated capacity statistical data of the target thin volume to the controller node corresponding to the target thin volume.
In an optional embodiment of the invention, the apparatus further comprises: a takeover module for:
and when a main controller node corresponding to the target thin volume fails, taking a controller node corresponding to the target thin volume as the main controller node of the target thin volume.
In an optional embodiment of the invention, the allocating module is configured to:
when a main controller node corresponding to a target thin volume receives a write request, distributing data addresses and metadata addresses of the target thin volume according to address distribution data and capacity statistical data corresponding to the target thin volume; the data address is an address for storing data corresponding to the write request in the target thin volume, and the metadata address is used for indexing to the data address.
In an optional embodiment of the invention, the apparatus further comprises: a forwarding module to:
when the controller node corresponding to the target thin volume receives a write request, the master controller node corresponding to the target thin volume receives the write request forwarded by the controller node corresponding to the target thin volume.
In an optional embodiment of the invention, the apparatus further comprises: a storage module to:
and storing the metadata of the target thin volume according to a B + Tree structure.
In an optional embodiment of the invention, the apparatus further comprises: a reconciliation module to:
and synchronizing the metadata of the target thin volume to a controller node corresponding to the target thin volume.
In an optional embodiment of the invention, the apparatus further comprises: a reading module to:
when a main controller node corresponding to the target thin volume receives a read request, acquiring a target metadata address from the read request;
determining a target data address corresponding to the target metadata address from metadata of the target thin volume;
and reading corresponding data from the target thin volume according to the target data address.
The embodiment of the invention also discloses electronic equipment which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory finish mutual communication through the communication bus;
the memory is used for storing a computer program;
the processor is configured to implement the method according to the embodiment of the present invention when executing the program stored in the memory.
Embodiments of the present invention also disclose a computer program product, which is stored in a storage medium and executed by at least one processor to implement the method according to the embodiments of the present invention.
Also disclosed is a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform a method according to an embodiment of the invention.
The embodiment of the invention has the following advantages:
in the embodiment of the invention, a dual-control cluster and a thin volume are created for a storage system, wherein the thin volume is respectively provided with a corresponding main controller node and a corresponding controller node in the dual-control cluster, when the main controller node corresponding to a target thin volume receives a write request, metadata of the target thin volume is distributed according to address distribution data and capacity statistical data corresponding to the target thin volume, wherein the metadata is used for storing the data corresponding to the write request in the target thin volume, then, the address distribution data and the capacity statistical data corresponding to the target thin volume can be updated according to the metadata of the target thin volume, and the updated address distribution data and the updated capacity statistical data are synchronized to the controller node corresponding to the target thin volume, so that both the main controller node and the controller node of the target thin volume can obtain the address distribution data and the capacity statistical data of the target thin volume. According to the embodiment of the invention, the consistency of the address allocation data and the capacity statistical data of the target reduced volume can be maintained by the master controller node and the slave controller node of the target reduced volume, so that even if a single controller node in the double-control cluster fails, the controller node which does not fail in the double-control cluster can take over the failed controller node, the metadata of the target reduced volume can be continuously and effectively allocated based on the address allocation data and the capacity statistical data corresponding to the target reduced volume, and the accuracy of the address allocation and the capacity statistics of the reduced volume of the storage system is improved.
Drawings
FIG. 1 is a schematic diagram of an application environment provided in an embodiment of the present invention;
FIG. 2 is a flowchart illustrating steps of a method for processing data of a thin volume of a storage system according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating steps of another method for processing data in a thin volume of a storage system according to an embodiment of the present invention;
FIG. 4 is a block diagram of a data processing apparatus for thin volumes in a storage system according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a hardware structure of an electronic device implementing various embodiments of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
In a specific implementation, the reduced volume of the storage system is implemented based on dual control, that is, one controller serves as a master controller node, that is, an Owner end, and the controller node provides a data read-write service, and the other controller serves as a non-Owner end, and the controller node provides a mirror image backup function of data and metadata, so that the non-Owner end can take over a service when a failure occurs. In the normal IO writing process, address allocation and capacity statistics of the thin volume are performed on a single controller node, and when a single controller fails, the problem of how to perform effective address allocation and capacity statistics needs to be considered.
Based on the above problems, the embodiment of the present invention provides a data processing method for a reduced volume of a storage system, where when a controller receives a write IO, a first step is to allocate corresponding metadata at an Owner end, a second step is to insert the metadata into a cache space, a third step is to perform inter-control mirroring on the metadata, and a fourth step is to update updated address allocation data and capacity statistical data to a dual-control cluster after the end of the cache mirroring, where the dual-control cluster uses consistency algorithms such as paxos algorithm to ensure data consistency of each controller node in the dual-control cluster. The storage system can ensure that a controller node at a non-Owner end can take over the responsibility of the controller node at the Owner end for address allocation and capacity statistics under the condition that a single controller node fails by ensuring the data consistency, so that the consistency of the address allocation and the capacity statistics is ensured, the data is not lost, and the reliability of the storage system is improved.
The data processing method for the storage system thin volume provided by the embodiment of the invention can be applied to the application environment shown in fig. 1. The thin volume is provided with a main controller node 101 and a controller node 102 which correspond to each other in the double control cluster; when a main controller node 101 corresponding to a target thin volume receives a write request, distributing metadata of the target thin volume according to address distribution data and capacity statistical data corresponding to the target thin volume; the metadata is used for storing data corresponding to the write request in the target thin volume; updating address allocation data and capacity statistical data corresponding to the target thin volume according to the metadata of the target thin volume; and synchronizing the updated address allocation data and the updated capacity statistical data of the target thin volume to the controller node 102 corresponding to the target thin volume.
In practical applications, the master controller node 101 and the slave controller node 102 may include, but are not limited to, various personal computers, notebook computers, smart phones, tablet computers, vehicle-mounted terminals, portable wearable devices, and servers, where the servers may be servers operating independently or a server cluster composed of a plurality of servers, and the servers may be cloud servers.
Referring to fig. 2, a flowchart illustrating steps of a data processing method for a thin volume of a storage system provided in an embodiment of the present invention is shown, which may specifically include the following steps:
In the embodiment of the present invention, a dual-controlled cluster and a thin volume of a storage system are created, where the dual-controlled cluster may include a plurality of controller nodes, and each thin volume in the control nodes has a corresponding master controller node (Owner end) and a corresponding controller node (Backup end), respectively.
The thin volumes are self-compaction volumes to achieve higher storage capacity utilization of the storage system of the dual-control cluster. In particular, a thin volume refers to a technology for virtualizing the capacity of a volume on a hard disk (i.e., storage space on a hard disk), and the core of the thin volume is address allocation of the thin volume before writing. The thin volume is a virtual volume on the hard disk, so that when the thin volume is created, all the capacity cannot be allocated to a user, the thin volume is allocated before writing only when the user performs writing operation on the thin volume, and an actual physical space is allocated to an address to be written.
In a specific implementation, when a plurality of thin volumes are created in a storage system, each thin volume is allocated while using, and as such, the distribution in the actual storage space of each thin volume is not continuous, and therefore, the implementation of the thin volumes requires the introduction of a concept of metadata. LBA (Logical Block Address, metadata Address/Logical Address) is a general mechanism used for indicating the location of data on a storage system, most commonly a hard disk is used as a metadata Address, and PBA (Physics Block Address, data Address/physical Address) is an actual Address of a storage space in the storage system. Specifically, when data is inserted, there is metadata of LBA (metadata address) > PBA (data address), and when data is read, it is necessary to determine the data address based on the metadata according to the metadata address, and then the data address is further held to read the disk to obtain real data.
Each thin volume has corresponding address allocation data and capacity statistical data, and metadata for allocating the thin volume may be based on the address allocation data and the capacity statistical data, specifically, the address allocation data indicates which metadata addresses and data addresses have been allocated to the thin volume, and the capacity statistical data indicates how much remaining capacity of the thin volume may be allocated.
In the embodiment of the present invention, when the master controller node corresponding to the target thin volume receives the write request sent by the controller node, the metadata of the target thin volume may be allocated according to the address allocation data and the capacity statistics data corresponding to the target thin volume, so that the data corresponding to the write request may be stored in the target thin volume according to the metadata.
In practical application, after the metadata of the target thin volume is allocated according to the address allocation data and the volume statistical data corresponding to the target thin volume, the address allocation data and the volume statistical data corresponding to the target thin volume need to be updated according to the metadata, so that the metadata corresponding to other write requests can be subsequently allocated according to the updated address allocation data and volume statistical data.
And step 204, synchronizing the updated address allocation data and the updated capacity statistical data of the target thin volume to a controller node corresponding to the target thin volume.
In the embodiment of the present invention, after the address allocation data and the volume statistical data are updated by the master controller node corresponding to the target thin volume, the updated address allocation data and volume statistical data may be updated to the dual-control cluster. The double-control cluster adopts paxos algorithm to ensure the data consistency of each controller node in the double-control cluster. The embodiment of the invention can ensure the consistency of other data such as metadata, address allocation data, capacity statistical data and the like of the controller nodes of the double-control cluster by applying the consistency algorithm based on message transmission.
Of course, in the embodiment of the present invention, other manners may also be used to ensure data consistency in the dual control cluster, for example, a Raft algorithm, a ZAB algorithm, a Gossip algorithm, and the like are used, so that after a certain controller node in the dual control cluster fails, other controller nodes can normally take over a service of the failed controller node based on the consistent data, and this embodiment of the present invention does not need to be limited.
In the embodiment of the invention, a dual-control cluster and a thin volume are created for a storage system, wherein the thin volume is respectively provided with a corresponding main controller node and a corresponding controller node in the dual-control cluster, when the main controller node corresponding to a target thin volume receives a write request, metadata of the target thin volume is distributed according to address distribution data and capacity statistical data corresponding to the target thin volume, wherein the metadata is used for storing the data corresponding to the write request in the target thin volume, then, the address distribution data and the capacity statistical data corresponding to the target thin volume can be updated according to the metadata of the target thin volume, and the updated address distribution data and the updated capacity statistical data are synchronized to the controller node corresponding to the target thin volume, so that both the main controller node and the controller node of the target thin volume can obtain the address distribution data and the capacity statistical data of the target thin volume. According to the embodiment of the invention, the consistency of the address allocation data and the capacity statistical data of the target reduced volume can be maintained by the master controller node and the slave controller node of the target reduced volume, so that even if a single controller node in the dual-control cluster fails, the controller node which does not fail in the dual-control cluster can take over the failed controller node, the metadata of the target reduced volume can still be effectively allocated continuously based on the address allocation data and the capacity statistical data corresponding to the target reduced volume, and the accuracy of the address allocation and the capacity statistics of the reduced volume of the storage system is improved.
Referring to fig. 3, a flowchart illustrating steps of another data processing method for a thin volume of a storage system according to an embodiment of the present invention is shown, which specifically includes the following steps:
In the embodiment of the invention, when the main controller node corresponding to the target thin volume receives the write request sent by the controller node, the address allocation data and the capacity statistical data corresponding to the target thin volume can be obtained, so that the metadata of the target thin volume can be accurately allocated according to the address allocation data and the capacity statistical data corresponding to the target thin volume, and the data corresponding to the write request can be stored in the target thin volume according to the metadata.
In an optional embodiment of the present invention, before, in step 302, when the master controller node corresponding to the target thin volume receives the write request, allocating the metadata of the target thin volume according to the address allocation data and the capacity statistics data corresponding to the target thin volume, the method may further include:
when the controller node corresponding to the target thin volume receives a write request, the master controller node corresponding to the target thin volume receives the write request forwarded by the controller node corresponding to the target thin volume.
In the embodiment of the present invention, if the master controller node corresponding to the target thin volume receives the write request, the metadata of the target thin volume may be allocated directly according to the address allocation data and the capacity statistical data corresponding to the target thin volume, and if the controller node corresponding to the target thin volume receives the write request, the controller node corresponding to the target thin volume may forward the write request to the master controller node, so that the master controller node may allocate the metadata of the target thin volume according to the address allocation data and the capacity statistical data corresponding to the target thin volume.
In an optional embodiment of the present invention, in the step 302, when the master controller node corresponding to the target thin volume receives the write request, allocating the metadata of the target thin volume according to the address allocation data and the capacity statistics data corresponding to the target thin volume may include the following steps:
when a main controller node corresponding to a target thin volume receives a write request, distributing data addresses and metadata addresses of the target thin volume according to address distribution data and capacity statistical data corresponding to the target thin volume.
The metadata may include a data address and a metadata address, among others. Specifically, the data address is an address for storing data corresponding to the write request in the target thin volume, and the metadata address is used for indexing to the data address
In the embodiment of the present invention, when a master controller node corresponding to a target thin volume receives a write request, a data address and a metadata address of the target thin volume may be allocated according to address allocation data and capacity statistical data corresponding to the target thin volume, so that data corresponding to the write request may be written into a thin volume corresponding to the data address, and when data needs to be read, a corresponding data address may be found from metadata according to the metadata address, and then corresponding data may be acquired from the thin volume according to the data address.
In an optional embodiment of the present invention, after, in step 302, when a master controller node corresponding to a target thin volume receives a write request, allocating metadata of the target thin volume according to address allocation data and capacity statistics data corresponding to the target thin volume, the method may further include:
and storing the metadata of the target thin volume according to a B + Tree structure.
Wherein, the B + Tree is a B-Tree variant and is a data storage structure. In the embodiment of the invention, the B + Tree structure can be used for storing the metadata of the target thin volume, so that the query efficiency of the metadata can be improved, and the storage efficiency or the reading efficiency of the double-control cluster is improved.
Of course, in practical application, other structures besides the B + Tree structure may be used to store the metadata, and only the efficient query of the metadata needs to be ensured, which is not limited in the embodiment of the present invention.
In the embodiment of the invention, after the master controller node of the target thin volume allocates the allocated data address and the metadata address, the master controller node of the target thin volume inserts the write cache of the metadata, and after the master controller node of the target thin volume inserts the write cache, the write cache is mirrored to the slave controller node of the target thin volume, so that the write cache loss after the single controller node fails is prevented.
And 306, when the main controller node corresponding to the target thin volume fails, taking the controller node corresponding to the target thin volume as the main controller node of the target thin volume.
In an optional embodiment of the present invention, in step 302, after the allocating, when the master controller node corresponding to the target thin volume receives the write request, the metadata of the target thin volume according to the address allocation data and the capacity statistics data corresponding to the target thin volume, the method further includes:
when a main controller node corresponding to the target thin volume receives a read request, acquiring a target metadata address from the read request;
determining a target data address corresponding to the target metadata address from the metadata of the target thin volume;
and reading corresponding data from the target thin volume according to the target data address.
In the embodiment of the present invention, when a controller node in a dual-control cluster wants to read data from a target thin volume, a read request may be sent to a master controller node corresponding to the target thin volume, and when the master controller node corresponding to the target thin volume receives the read request, a target metadata address may be obtained from the read request, and then a target data address corresponding to the target metadata address is determined from metadata of the target thin volume, so that corresponding data may be read from the target thin volume according to the target data address, thereby implementing reading of data of the thin volume.
The key point of the embodiment of the invention is to provide a method for the consistency of the capacity allocation of the multiple controllers of the reduced volume of the storage system, based on the method, the consistency of the address allocation and the capacity of the storage system under a fault scene can be effectively ensured, the capacity waste and the loss are reduced, and the reliability of the storage system is improved.
In order that those skilled in the art will better understand the embodiments of the present invention, a specific example will be described below. Specifically, an embodiment of the present invention provides a method for consistent allocation of capacity of multiple thin volume controllers in a storage system, where a specific implementation process of the method may be performed according to the following steps:
the method comprises the steps that firstly, a double-control cluster is established for a storage system, a thin volume is established, the established thin volume is an Owner end (a main controller node) on one controller node, and a non-Owner end (a controller node) on the other controller node;
secondly, processing a write request of a controller node of the double-control cluster, and when the write request comes from a non-Owner end, forwarding data by an upper layer module of the non-Owner end, and forwarding the data to the Owner end to process the write request;
thirdly, distributing data addresses and metadata addresses of data of the write request from the controller node at the Owner end of the corresponding thin volume;
fourthly, after the data address and the metadata address are distributed, inserting a write cache of the metadata at an Owner end, wherein the organization format of the metadata can be B + Tree;
fifthly, after the Owner end is inserted and the write cache is finished, the write cache is mirrored to the non-Owner end, so that the write cache is prevented from being lost after a single controller node is in fault;
sixthly, after the cache mirror image writing is finished, the Owner end distributes data and capacity statistical data to the cluster updating address;
and seventhly, when a single controller node failure occurs, such as a main controller node failure, the address allocation data and the capacity statistical data which are updated to the double-control cluster are reversely updated to a non-Owner end, so that the consistency of the address allocation and the capacity statistics is ensured.
By applying the embodiment of the invention, the consistency of the address allocation data and the capacity statistical data of the controller node which is taken over when a fault occurs is ensured by ensuring the synchronous update of the address allocation and the capacity statistics in the double-control cluster.
When data is written into the thin volume of the double-control cluster, address allocation and capacity statistics are carried out at an Owner end, after metadata cache synchronization between controller nodes is completed, address allocation data and capacity statistics data of the thin volume are updated, the updated address allocation data and capacity statistics data are refreshed to the double-control cluster, a consistency algorithm of the double-control cluster is used for guaranteeing that all the controller nodes are consistent data, and then the address allocation data and the capacity statistics data of the thin volume are reversely updated from the double-control cluster when a single controller node fails, so that consistency of capacity allocation of multiple controllers of the thin volume of the storage system is guaranteed.
It should be noted that for simplicity of description, the method embodiments are shown as a series of combinations of acts, but those skilled in the art will recognize that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 4, a block diagram of a data processing apparatus for storing a thin volume of a system according to an embodiment of the present invention is shown, which may specifically include the following modules:
a creating module 401, configured to create a dual-control cluster and a thin volume for a storage system; the thin volume is provided with a main controller node and a controller node which correspond to each other in the double-control cluster respectively;
an allocating module 402, configured to allocate, when a master controller node corresponding to a target thin volume receives a write request, metadata of the target thin volume according to address allocation data and capacity statistical data corresponding to the target thin volume; the metadata is used for storing data corresponding to the write request in the target thin volume;
an updating module 403, configured to update the address allocation data and the capacity statistics data corresponding to the target thin volume according to the metadata of the target thin volume;
a synchronizing module 404, configured to synchronize the updated address allocation data and the updated capacity statistical data of the target thin volume to a controller node corresponding to the target thin volume.
In an optional embodiment of the invention, the apparatus further comprises: a take-over module for:
and when the main controller node corresponding to the target thin volume fails, taking the controller node corresponding to the target thin volume as the main controller node of the target thin volume.
In an optional embodiment of the present invention, the allocating module 402 is configured to:
when a main controller node corresponding to a target thin volume receives a write request, distributing data addresses and metadata addresses of the target thin volume according to address distribution data and capacity statistical data corresponding to the target thin volume; the data address is an address for storing data corresponding to the write request in the target thin volume, and the metadata address is used for indexing to the data address.
In an optional embodiment of the invention, the apparatus further comprises: a forwarding module to:
when the controller node corresponding to the target thin volume receives a write request, the master controller node corresponding to the target thin volume receives the write request forwarded by the controller node corresponding to the target thin volume.
In an optional embodiment of the invention, the apparatus further comprises: a storage module to:
and storing the metadata of the target thin volume according to a B + Tree structure.
In an optional embodiment of the invention, the apparatus further comprises: a reconciliation module to:
and synchronizing the metadata of the target thin volume to a controller node corresponding to the target thin volume.
In an optional embodiment of the invention, the apparatus further comprises: a reading module to:
when a main controller node corresponding to the target thin volume receives a read request, acquiring a target metadata address from the read request;
determining a target data address corresponding to the target metadata address from the metadata of the target thin volume;
and reading corresponding data from the target thin volume according to the target data address.
According to the embodiment of the invention, the consistency of the address allocation data and the capacity statistical data of the target reduced volume can be maintained by the master controller node and the slave controller node of the target reduced volume, so that even if a single controller node in the dual-control cluster fails, the controller node which does not fail in the dual-control cluster can take over the failed controller node, the metadata of the target reduced volume can still be effectively allocated continuously based on the address allocation data and the capacity statistical data corresponding to the target reduced volume, and the accuracy of the address allocation and the capacity statistics of the reduced volume of the storage system is improved.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
In addition, an embodiment of the present invention further provides an electronic device, including: the processor, the memory, and the computer program stored in and executable on the memory are executed by the processor to implement the processes of the data processing method embodiment of the reduced volume of the storage system, and can achieve the same technical effects, and are not described herein again to avoid repetition.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the embodiment of the data processing method for a reduced volume of a storage system, and can achieve the same technical effect, and in order to avoid repetition, the computer program is not described herein again. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
Embodiments of the present invention further provide a computer program product, where the program product is stored in a storage medium, and the program product is executed by at least one processor to implement each process of the data processing method embodiment of the reduced volume of the storage system, where the same technical effect can be achieved, and details are not described here to avoid repetition.
Fig. 5 is a schematic diagram of a hardware structure of an electronic device for implementing various embodiments of the present invention.
The electronic device 500 includes, but is not limited to: radio frequency unit 501, network module 502, audio output unit 503, input unit 504, sensor 505, display unit 506, user input unit 507, interface unit 508, memory 509, processor 510, and power supply 511. Those skilled in the art will appreciate that the electronic device configuration shown in fig. 5 does not constitute a limitation of the electronic device, and that the electronic device may include more or fewer components than shown, or some components may be combined, or a different arrangement of components. In the embodiment of the present invention, the electronic device includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted terminal, a wearable device, a pedometer, and the like.
It should be understood that, in the embodiment of the present invention, the radio frequency unit 501 may be used for receiving and sending signals during a process of sending and receiving information or a call, and specifically, receives downlink data from a base station and then processes the downlink data to the processor 510; in addition, the uplink data is transmitted to the base station. In general, radio frequency unit 501 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 501 can also communicate with a network and other devices through a wireless communication system.
The electronic device provides wireless broadband internet access to the user via the network module 502, such as assisting the user in sending and receiving e-mails, browsing web pages, and accessing streaming media.
The audio output unit 503 may convert audio data received by the radio frequency unit 501 or the network module 502 or stored in the memory 509 into an audio signal and output as sound. Also, the audio output unit 503 may also provide audio output related to a specific function performed by the electronic apparatus 500 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 503 includes a speaker, a buzzer, a receiver, and the like.
The input unit 504 is used to receive audio or video signals. The input Unit 504 may include a Graphics Processing Unit (GPU) 5041 and a microphone 5042, and the Graphics processor 5041 processes image data of still pictures or video obtained by an image capturing device (such as a camera) in a video capture mode or an image capture mode. The processed image frames may be displayed on the display unit 506. The image frames processed by the graphics processor 5041 may be stored in the memory 509 (or other storage media) or transmitted via the radio frequency unit 501 or the network module 502. The microphone 5042 may receive sounds and may be capable of processing such sounds into audio data. The processed audio data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 501 in case of the phone call mode.
The electronic device 500 also includes at least one sensor 505, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor includes an ambient light sensor that can adjust the brightness of the display panel 5061 according to the brightness of ambient light, and a proximity sensor that can turn off the display panel 5061 and/or a backlight when the electronic device 500 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration identification related functions (such as pedometer, tapping), and the like; the sensors 505 may also include fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which are not described in detail herein.
The display unit 506 is used to display information input by the user or information provided to the user. The Display unit 506 may include a Display panel 5061, and the Display panel 5061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.
The user input unit 507 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device. Specifically, the user input unit 507 includes a touch panel 5071 and other input devices 5072. Touch panel 5071, also referred to as a touch screen, may collect touch operations by a user on or near it (e.g., operations by a user on or near touch panel 5071 using a finger, stylus, or any suitable object or attachment). The touch panel 5071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 510, and receives and executes commands sent by the processor 510. In addition, the touch panel 5071 may be implemented by various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 5071, the user input unit 507 may include other input devices 5072. Specifically, the other input devices 5072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described herein.
Further, the touch panel 5071 may be overlaid on the display panel 5061, and when the touch panel 5071 detects a touch operation thereon or nearby, the touch operation is transmitted to the processor 510 to determine the type of the touch event, and then the processor 510 provides a corresponding visual output on the display panel 5061 according to the type of the touch event. Although in fig. 5, the touch panel 5071 and the display panel 5061 are two independent components to implement the input and output functions of the electronic device, in some embodiments, the touch panel 5071 and the display panel 5061 may be integrated to implement the input and output functions of the electronic device, and is not limited herein.
The interface unit 508 is an interface for connecting an external device to the electronic apparatus 500. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 508 may be used to receive input (e.g., data information, power, etc.) from external devices and transmit the received input to one or more elements within the electronic apparatus 500 or may be used to transmit data between the electronic apparatus 500 and external devices.
The memory 509 may be used to store software programs as well as various data. The memory 509 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 509 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
The processor 510 is a control center of the electronic device, connects various parts of the whole electronic device by using various interfaces and lines, performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 509 and calling data stored in the memory 509, thereby performing overall monitoring of the electronic device. Processor 510 may include one or more processing units; preferably, the processor 510 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 510.
The electronic device 500 may further include a power supply 511 (e.g., a battery) for supplying power to various components, and preferably, the power supply 511 may be logically connected to the processor 510 via a power management system, so as to implement functions of managing charging, discharging, and power consumption via the power management system.
In addition, the electronic device 500 includes some functional modules that are not shown, and are not described in detail here.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a component of' 8230; \8230;" does not exclude the presence of another like element in a process, method, article, or apparatus that comprises the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the technical solution of the present invention or a part thereof which substantially contributes to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of the changes or substitutions within the technical scope of the present invention, and shall cover the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A data processing method for a thin volume of a storage system, the method comprising:
creating a dual-control cluster and a thin volume for a storage system; the thin volume is provided with a main controller node and a controller node which correspond to each other in the double-control cluster respectively;
when a main controller node corresponding to a target thin volume receives a write request, distributing metadata of the target thin volume according to address distribution data and capacity statistical data corresponding to the target thin volume; the metadata is used for storing data corresponding to the write request in the target thin volume;
updating the address allocation data and the capacity statistical data corresponding to the target thin volume according to the metadata of the target thin volume;
and synchronizing the updated address allocation data and the updated capacity statistical data of the target thin volume to a controller node corresponding to the target thin volume.
2. The method according to claim 1, wherein after said synchronizing the updated address allocation data and the updated capacity statistics data of the target thin volume to the controller node corresponding to the target thin volume, the method further comprises:
and when the main controller node corresponding to the target thin volume fails, taking the controller node corresponding to the target thin volume as the main controller node of the target thin volume.
3. The method according to claim 1, wherein the allocating the metadata of the target thin volume according to the address allocation data and the capacity statistics data corresponding to the target thin volume when the host controller node corresponding to the target thin volume receives a write request comprises:
when a main controller node corresponding to a target thin volume receives a write request, distributing data addresses and metadata addresses of the target thin volume according to address distribution data and capacity statistical data corresponding to the target thin volume; the data address is an address for storing data corresponding to the write request in the target thin volume, and the metadata address is used for indexing to the data address.
4. The method according to claim 1, wherein before the allocating, when the host controller node corresponding to the target thin volume receives the write request, the metadata of the target thin volume according to the address allocation data and the capacity statistics data corresponding to the target thin volume, the method further comprises:
when the controller node corresponding to the target thin volume receives a write request, the master controller node corresponding to the target thin volume receives the write request forwarded by the controller node corresponding to the target thin volume.
5. The method according to claim 1, wherein after the allocating the metadata of the target thin volume according to the address allocation data and the capacity statistics data corresponding to the target thin volume when the host controller node corresponding to the target thin volume receives the write request, the method further comprises:
and storing the metadata of the target thin volume according to a B + Tree structure.
6. The method according to claim 1, wherein after the allocating the metadata of the target thin volume according to the address allocation data and the capacity statistics data corresponding to the target thin volume when the host controller node corresponding to the target thin volume receives the write request, the method further comprises:
and synchronizing the metadata of the target thin volume to a controller node corresponding to the target thin volume.
7. The method according to claim 3, wherein after said allocating the metadata of the target thin volume according to the address allocation data and the capacity statistics data corresponding to the target thin volume when the host controller node corresponding to the target thin volume receives the write request, the method further comprises:
when a main controller node corresponding to the target thin volume receives a read request, acquiring a target metadata address from the read request;
determining a target data address corresponding to the target metadata address from metadata of the target thin volume;
and reading corresponding data from the target thin volume according to the target data address.
8. A data processing apparatus that stores system thin volumes, the apparatus comprising:
the creation module is used for creating the double-control cluster and the thin volume aiming at the storage system; the thin volume is provided with a main controller node and a controller node which correspond to each other in the double-control cluster respectively;
the allocation module is used for allocating the metadata of the target thin volume according to the address allocation data and the capacity statistical data corresponding to the target thin volume when the main controller node corresponding to the target thin volume receives the write request; the metadata is used for storing data corresponding to the write request in the target thin volume;
an updating module, configured to update the address allocation data and the capacity statistics data corresponding to the target thin volume according to the metadata of the target thin volume;
and the synchronization module is used for synchronizing the updated address allocation data and the updated capacity statistical data of the target thin volume to a controller node corresponding to the target thin volume.
9. An electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
the memory is used for storing a computer program;
the processor, when executing a program stored on the memory, implementing the method of any one of claims 1-7.
10. A computer-readable storage medium having instructions stored thereon, which when executed by one or more processors, cause the processors to perform the method recited by any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211293115.5A CN115657943A (en) | 2022-10-21 | 2022-10-21 | Data processing method, device and equipment for thin volume of storage system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211293115.5A CN115657943A (en) | 2022-10-21 | 2022-10-21 | Data processing method, device and equipment for thin volume of storage system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115657943A true CN115657943A (en) | 2023-01-31 |
Family
ID=84988508
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211293115.5A Pending CN115657943A (en) | 2022-10-21 | 2022-10-21 | Data processing method, device and equipment for thin volume of storage system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115657943A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117215499A (en) * | 2023-11-07 | 2023-12-12 | 苏州元脑智能科技有限公司 | Data access method, device, product and multi-control storage system of thin volume |
-
2022
- 2022-10-21 CN CN202211293115.5A patent/CN115657943A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117215499A (en) * | 2023-11-07 | 2023-12-12 | 苏州元脑智能科技有限公司 | Data access method, device, product and multi-control storage system of thin volume |
| CN117215499B (en) * | 2023-11-07 | 2024-02-09 | 苏州元脑智能科技有限公司 | Data access method, device, product and multi-control storage system of thin volume |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109768926B (en) | Data processing method, terminal equipment and computer readable storage medium | |
| EP3531290B1 (en) | Data backup method, apparatus, electronic device, storage medium, and system | |
| CN109885257B (en) | Disk file reading and writing method and device | |
| CN109101363B (en) | Exception recovery method and device based on Android system, intelligent equipment and storage medium | |
| CN110147380B (en) | Data updating method, device, server and storage medium | |
| CN112035298A (en) | Data backup method and device, electronic equipment and storage medium | |
| CN109144723B (en) | Method and terminal for allocating storage space | |
| CN111274160A (en) | Data storage method, electronic device, and medium | |
| CN115617278B (en) | Path device selection method and device, electronic device and readable storage medium | |
| CN111338745A (en) | Deployment method and device of virtual machine and intelligent equipment | |
| CN115657943A (en) | Data processing method, device and equipment for thin volume of storage system and storage medium | |
| CN111131607A (en) | Information sharing method, electronic equipment and computer readable storage medium | |
| CN115543221B (en) | Data migration method and device for solid state disk, electronic equipment and storage medium | |
| CN109889568B (en) | Data export method, server and system | |
| CN115509770B (en) | Micro-service calling method and device, electronic equipment and readable storage medium | |
| CN108319839B (en) | Fingerprint unlocking method and device | |
| CN115167764A (en) | Data read-write processing method and device, electronic equipment and storage medium | |
| CN115437775A (en) | Virtual memory address space allocation method and device and electronic equipment | |
| CN110659329A (en) | Data processing method and device and electronic equipment | |
| CN112698787A (en) | Data writing method and device and computer storage medium | |
| CN115811520B (en) | Method and device for electing master node in distributed system and electronic equipment | |
| WO2022188184A1 (en) | Data storage method and related device | |
| CN115695309B (en) | Access control list rule configuration method and device, electronic equipment and storage medium | |
| CN113127557B (en) | Data persistence method and device based on redis performance and electronic equipment | |
| WO2022151903A1 (en) | Distributed file access method, and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |