CN110018796B - Method and device for processing data request by storage system - Google Patents
Method and device for processing data request by storage system Download PDFInfo
- Publication number
- CN110018796B CN110018796B CN201910290063.8A CN201910290063A CN110018796B CN 110018796 B CN110018796 B CN 110018796B CN 201910290063 A CN201910290063 A CN 201910290063A CN 110018796 B CN110018796 B CN 110018796B
- Authority
- CN
- China
- Prior art keywords
- metadata
- node
- metadata node
- space
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000012545 processing Methods 0.000 title claims abstract description 40
- 230000008859 change Effects 0.000 claims abstract description 59
- 230000008569 process Effects 0.000 claims abstract description 12
- 238000013507 mapping Methods 0.000 claims description 10
- 238000011084 recovery Methods 0.000 claims description 6
- 230000001360 synchronised effect Effects 0.000 claims description 2
- 238000012986 modification Methods 0.000 description 7
- 230000004048 modification Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011010 flushing procedure Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/062—Securing storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Abstract
The invention discloses a method for processing a data request by a storage system, which comprises two metadata nodes, wherein after a client sends a write request, a logic space and a physical space are distributed for data to be written according to an automatic thin provisioning strategy, a metadata change instruction is generated, when the two metadata nodes operate normally, one metadata node executes an update operation according to the metadata change instruction and synchronizes an update result to the other metadata node, and after one metadata node fails, the other metadata node executes the update operation according to the metadata instruction and backs up the update result to a preset storage space. Therefore, the method processes the metadata change instruction and performs backup through the mutual cooperation of the two metadata nodes, avoids the loss of the metadata, and obviously improves the safety of the storage system. In addition, the invention also provides a device for processing the data request by the storage system and the storage system, and the function of the device corresponds to the function of the method.
Description
Technical Field
The present invention relates to the field of storage, and in particular, to a method, an apparatus, and a storage system for processing a data request by a storage system.
Background
The principle of thin provisioning is to provide a storage space from a thin storage pool as needed for a user to use, and compared with the traditional non-thin provisioning, the thin provisioning does not need to allocate unused disk capacity in advance, so that the storage utilization rate is higher, a large amount of metadata needs to be maintained, and user data is lost when the metadata is lost, so that it is very important to ensure the security of the user data metadata.
Disclosure of Invention
The invention aims to provide a method, a device and a storage system for processing a data request by using a storage system, which are used for solving the problem that the security of metadata needs to be improved as a large amount of metadata needs to be maintained by using the storage system with thin provisioning.
To solve the above technical problem, the present invention provides a method for processing a data request by a storage system, where the storage system includes a first metadata node and a second metadata node, and the method includes:
responding to a write request sent by a client, and distributing a logic space and a physical space for data to be written according to an automatic thin provisioning strategy;
generating a metadata change instruction according to the logic space and the physical space;
when a first metadata node and a second metadata node both run normally, sending the metadata change instruction to the first metadata node so that the first metadata node executes an update operation according to the metadata change instruction and synchronizes an update result to the second metadata node, wherein the update operation comprises an operation of inserting target metadata or an operation of modifying pre-stored metadata into the target metadata, and the target metadata comprises a mapping relation from the logical space to the physical space;
and when the first metadata node fails and the second metadata node normally operates, sending the metadata change instruction to the second metadata node so that the second metadata node can execute the updating operation according to the metadata instruction and backup an updating result to a preset storage space.
Optionally, before the sending the metadata change instruction to the second metadata node so that the second metadata node executes the update operation according to the metadata instruction and backs up the update result to a preset storage space, the method further includes:
acquiring an operation state parameter of the first metadata node;
and judging whether the first metadata node fails or not according to the running state parameters.
Optionally, before the responding to the write request sent by the client and allocating a logical space and a physical space for data to be written according to an auto-thin configuration policy, the method further includes:
receiving a data request sent by a client;
judging whether the data request is a write request;
if the metadata is not the write request, searching corresponding metadata from the second metadata node according to the data request when the first metadata node and the second metadata node both run normally, so as to determine the physical space where the data corresponding to the data request is located according to the metadata.
Optionally, after the sending the metadata change instruction to the second metadata node so that the second metadata node executes the update operation according to the metadata instruction and backs up the update result to a preset storage space, the method further includes:
judging whether the first metadata node recovers normal operation or not;
and if the normal operation is recovered, synchronizing the metadata in the second metadata node to the first metadata node, and releasing the preset storage space.
Optionally, after the synchronizing the metadata in the second metadata node to the first metadata node and releasing the preset storage space, the method further includes:
judging whether all dirty data in the cache of the first metadata node are printed to a disk;
and if so, determining that the first metadata node and the second metadata node both operate normally.
Correspondingly, the present invention also provides an apparatus for processing a data request in a storage system, where the storage system includes a first metadata node and a second metadata node, and the apparatus includes:
a space allocation module: the data writing method comprises the steps of responding to a writing request sent by a client, and distributing a logic space and a physical space for data to be written according to an automatic thin provisioning strategy;
a change instruction generation module: generating a metadata change instruction according to the logic space and the physical space;
a first sending module: the metadata updating method comprises the steps that when a first metadata node and a second metadata node both run normally, a metadata changing instruction is sent to the first metadata node, so that the first metadata node can execute updating operation according to the metadata changing instruction, and an updating result is synchronized to the second metadata node, wherein the updating operation comprises the operation of inserting target metadata or the operation of modifying pre-stored metadata into the target metadata, and the target metadata comprises the mapping relation from a logical space to a physical space;
a second sending module: and the metadata change instruction is sent to the second metadata node when the first metadata node fails and the second metadata node normally operates, so that the second metadata node executes the update operation according to the metadata instruction and backs up the update result to a preset storage space.
Optionally, the apparatus includes:
a state parameter acquisition module: the operation state parameter is used for acquiring the operation state parameter of the first metadata node;
a fault judgment module: and the first metadata node is used for judging whether the first metadata node has a fault according to the running state parameters.
Optionally, the apparatus further comprises:
a request receiving module: the data processing system is used for receiving a data request sent by a client;
a request judgment module: the data processing device is used for judging whether the data request is a write request;
a metadata lookup module: and when the data request is not a write request and both the first metadata node and the second metadata node operate normally, searching corresponding metadata from the second metadata node according to the data request so as to determine a physical space where data corresponding to the data request is located according to the metadata.
Optionally, the apparatus further comprises:
and a normal recovery judging module: the first metadata node is used for judging whether the first metadata node recovers normal operation or not;
a releasing module: and the storage device is used for synchronizing the metadata in the second metadata node to the first metadata node and releasing the preset storage space when normal operation is recovered.
Finally, the present invention also provides a storage system comprising: the system comprises a client, a storage server, a first metadata node and a second metadata node;
the client is used for sending a write request;
the storage server is used for responding to the write request, distributing a logic space and a physical space for the data to be written according to an automatic thin provisioning strategy and generating a metadata change instruction;
the first metadata node is used for executing updating operation according to the metadata change instruction when the first metadata node and the second metadata node both run normally, and synchronizing an updating result to the second metadata node, wherein the updating operation comprises an operation of inserting target metadata or an operation of modifying pre-stored metadata into the target metadata, and the target metadata comprises a mapping relation from the logic space to the physical space;
and the second metadata node is used for executing the updating operation according to the metadata instruction when the first metadata node fails and normally operates per se, and backing up the updating result to a preset storage space.
The storage system comprises two metadata nodes, after a client sends a write request, a logic space and a physical space are distributed for data to be written according to an automatic thin provisioning strategy, a metadata change instruction is generated, when the two metadata nodes normally operate, one metadata node executes an update operation according to the metadata change instruction and synchronizes an update result to the other metadata node, and when one metadata node fails, the other metadata node executes the update operation according to the metadata instruction and backs up the update result to a preset storage space. Therefore, the method aims at the storage system with the automatic thin provisioning, the metadata change instruction is processed and backed up through the mutual cooperation of the two metadata nodes, the metadata loss is avoided, and the safety of the storage system is obviously improved.
In addition, the invention also provides a device for processing data request by the storage system and the storage system, and the function of the device corresponds to the function of the method, which is not described again.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
Fig. 1 is a flowchart illustrating a first implementation of a method for processing a data request in a storage system according to a first embodiment of the present invention;
FIG. 2 is a flowchart illustrating a second implementation of a method for processing a data request in a storage system according to a second embodiment of the present invention;
FIG. 3 is a functional block diagram of an embodiment of an apparatus for processing a data request in a storage system according to the present invention;
fig. 4 is a block diagram of a memory system according to an embodiment of the present invention.
Detailed Description
The core of the invention is to provide a method and a device for processing a data request by a storage system and the storage system, aiming at the storage system with automatic thin provisioning, the method and the device realize the mutual cooperation of two metadata nodes to process a metadata change instruction and perform backup, reduce the probability of metadata loss and obviously improve the safety of the storage system.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a first embodiment of a method for processing a data request in a storage system according to the present invention is described below, where the first embodiment includes:
step S101: responding to a write request sent by a client, and distributing a logic space and a physical space for data to be written according to the thin provisioning strategy.
The embodiment is applied to a storage system, which includes a client, a storage server, a first metadata node and a second metadata node, where the client is used to send a data request including but not limited to a write request, the storage server is used to store data, and the first metadata node and the second metadata node are used to store metadata, and there is no essential difference between them, and in a case of normal operation, the metadata on the first metadata node and the metadata on the second metadata node are consistent. The thin provisioning policy refers to that the storage system manages storage resources in a unified manner, and allocates the storage resources to the upper layer application as needed.
Step S102: and generating a metadata change instruction according to the logic space and the physical space.
When a client requests to write data, the metadata needs to be updated, for example, the metadata is inserted or modified, and specifically, at least the mapping relationship between the above logical space and the physical space is maintained in the metadata.
Step S103: and when the first metadata node and the second metadata node both run normally, sending the metadata change instruction to the first metadata node, so that the first metadata node executes update operation according to the metadata change instruction and synchronizes an update result to the second metadata node.
The updating operation comprises an operation of inserting target metadata or an operation of modifying pre-stored metadata into the target metadata, and the target metadata comprises a mapping relation from the logic space to the physical space.
Step S104: and when the first metadata node fails and the second metadata node normally operates, sending the metadata change instruction to the second metadata node so that the second metadata node can execute the updating operation according to the metadata instruction and backup an updating result to a preset storage space.
And only one node executes change operation according to the metadata change instruction between the first metadata node and the second metadata node, and when the node executing the change operation fails, the other node is immediately switched to. And when the other metadata node executes the change operation, the change result is stored in the preset storage space which is obtained by applying in advance, so that when the node also breaks down, the data recovery can be carried out on the metadata node according to the data in the preset storage space. It should be noted that the metadata change instruction may be sent to the first metadata node and the second metadata node at the same time, and then the two metadata nodes determine whether to execute the metadata change instruction by themselves according to the operating states of the two metadata nodes.
In addition, the metadata in the caches of the first metadata node and the second metadata node support power failure protection, that is, power failure is stored to a system disk through a BBU, and after the node failure is recovered, the metadata and user data of the power failure protection of a memory are recovered through a memory management module of a storage system, which is a first safety line.
The embodiment provides a method for processing a data request by a storage system, where the storage system includes two metadata nodes, and after a client sends a write request, a logical space and a physical space are allocated to data to be written according to an auto-thin configuration policy, and a metadata change instruction is generated. Therefore, the method aims at the storage system with the automatic thin provisioning, the metadata change instruction is processed and backed up through the mutual cooperation of the two metadata nodes, the metadata loss is avoided, and the safety of the storage system is obviously improved.
The second embodiment of the method for processing a data request by a storage system provided by the invention is described in detail below, and the second embodiment is implemented based on the first embodiment and is expanded to a certain extent on the basis of the first embodiment.
Specifically, this embodiment uses a winner-backup method to process metadata and data, that is, one of two metadata nodes in a domain is used as a winner, and the other is used as a backup. When two metadata nodes both operate normally, the two metadata nodes are in a caching mode, and when one metadata node fails and the other metadata node operates normally, the two metadata nodes are in a logging mode.
Based on the above description, the implementation process of the second embodiment is described below, and referring to fig. 2, the second embodiment includes:
step S201: and receiving a data request sent by the client.
Step S202: and judging whether the data request is a write request, if so, generating a metadata change instruction, and if not, generating a metadata query instruction.
Step S203: in the caching mode, the owner node executes change operation according to the metadata change instruction, synchronizes a change result to the backup node, and the backup node queries corresponding metadata according to the metadata query instruction.
The so-called caching mode is that the owner node processes modification operation and insertion operation of the metadata node, the query and modification result of the metadata are backed up to the backup node in a transaction mode, and the backup node forwards the metadata change instruction to the owner node for processing. Therefore, when the winner node has a problem, the backup node can continue to process the host IO and the metadata, and simultaneously trigger corresponding fault processing operations, namely, cache flushing, metadata disk dropping and the like, so as to ensure data safety.
Normally, a Caching mode is adopted, namely, the change of metadata of an owner node is backed up to a backup node in a transaction mode, so that the metadata in the caches of the two nodes are kept consistent, and when one node fails, the other node can immediately take over without data inconsistency and data loss.
Step S204: and in the logging mode, the backup node executes change operation according to the metadata change instruction and stores a change result or the metadata change instruction in a Log storage space.
When only one normally operating metadata node exists in the storage system, a winner-backup relationship cannot be formed, and data reliability is reduced, so that the embodiment sets a Logging mode, that is, an independent Log storage space is applied, metadata change instructions or updated metadata are written into the Log storage space in sequence, and after the corresponding metadata is landed on the normally operating metadata node, the Log storage resource is made invalid.
The process of converting the caching mode into the logging mode is as follows: firstly, if the fault occurs in the inner node, the operation of switching the nodes is triggered, and the processing of the metadata instruction is not influenced; and then, the bankup node detects whether the dirty data in the cache of the owner node completely falls, if not, the falling operation is triggered, so that the dirty data before the fault starts to fall, and the data safety is guaranteed. And finally, switching the chachiging mode into a Logging mode, and ensuring that only one node can receive and process IO.
If the owner node recovers normal operation after a fault occurs, data recovery is carried out according to the cache power-down protection data and the metadata recorded in the Log storage space and the data safety is guaranteed. However, at this time, the node and the bankup node do not immediately enter the caching mode, but the node and the node determine whether the dirty data of the owner node completely falls down, and whether the data of the Long storage space is updated to the owner node, and only if the dirty data and the data of the Long storage space are satisfied at the same time, the node and the node return to the caching mode again.
It can be seen that, according to the method for processing a data request by a storage system provided by this embodiment, a metadata processing mode for automatic thin provisioning is designed, security and consistency of metadata are guaranteed in a dual-control caching and single-control Logging manner, and meanwhile, a failure and recovery processing flow of the dual-control system is controlled to adapt to a failure and recovery of a dual-control node, so that when a system failure is unstable, data security of the storage system is guaranteed, reliability and processing capability of a product are improved, and competitiveness of the product is improved.
In the following, a device for processing a data request by a storage system according to an embodiment of the present invention is introduced, and the device for processing a data request by a storage system described below and the method for processing a data request by a storage system described above may be referred to correspondingly.
As shown in fig. 3, the apparatus includes:
the space allocation module 301: the data writing method comprises the steps of responding to a writing request sent by a client, and distributing a logic space and a physical space for data to be written according to a thin provisioning strategy.
Change instruction generation module 302: and generating a metadata change instruction according to the logic space and the physical space.
The first transmission module 303: and the metadata modification unit is used for sending the metadata modification instruction to the first metadata node when both the first metadata node and the second metadata node run normally, so that the first metadata node executes an update operation according to the metadata modification instruction, and synchronizes an update result to the second metadata node, wherein the update operation comprises an operation of inserting target metadata or an operation of modifying pre-stored metadata into the target metadata, and the target metadata comprises a mapping relation from the logical space to the physical space.
The second sending module 304: and the metadata change instruction is sent to the second metadata node when the first metadata node fails and the second metadata node normally operates, so that the second metadata node executes the update operation according to the metadata instruction and backs up the update result to a preset storage space.
As an alternative embodiment, the apparatus comprises:
the state parameter acquisition module 305: the operation state parameter is used for acquiring the operation state parameter of the first metadata node;
the fault judgment module 306: and the first metadata node is used for judging whether the first metadata node has a fault according to the running state parameter.
As an optional implementation, the apparatus further comprises:
the request receiving module 307: the data processing system is used for receiving a data request sent by a client;
the request judging module 308: the data processing device is used for judging whether the data request is a write request;
metadata lookup module 309: and if the metadata is not the write request, searching corresponding metadata from the second metadata node according to the data request when the first metadata node and the second metadata node both operate normally, so as to determine the physical space where the data corresponding to the data request is located according to the metadata.
As an optional implementation, the apparatus further comprises:
the restoration normal determination module 310: the first metadata node is used for judging whether the first metadata node recovers normal operation or not;
the release module 311: and if the normal operation is recovered, synchronizing the metadata in the second metadata node to the first metadata node, and releasing the preset storage space.
Therefore, the specific implementation of the apparatus for a storage system to process a data request in this embodiment may be seen in the foregoing embodiment parts of the method for a storage system to process a data request, for example, the space allocation module 301, the change instruction generation module 302, the first sending module 303, and the second sending module 304, which are respectively used to implement steps S101, S102, S103, and S104 in the method for a storage system to process a data request in this embodiment. Therefore, specific embodiments thereof may be referred to in the description of the corresponding respective partial embodiments, and will not be described herein.
In addition, since the apparatus for processing a data request by a storage system of this embodiment is used to implement the method for processing a data request by a storage system, the role of the apparatus corresponds to that of the method described above, and details are not described here.
In addition, the present invention further provides an embodiment of a storage system, and a storage system described below and a method for processing a data request by the storage system described above may be referred to correspondingly. Referring to fig. 4, the storage system includes: the client 401, the storage server 402, the first metadata node 403, and the second metadata node 404, the functions of each part are as follows:
the client 401 is configured to send a write request;
the storage server 402 is configured to respond to the write request, allocate a logical space and a physical space to data to be written according to an auto-thin configuration policy, and generate a metadata change instruction;
the first metadata node 403 is configured to, when both itself and the second metadata node 404 run normally, execute an update operation according to the metadata change instruction, and synchronize an update result to the second metadata node 404, where the update operation includes an operation of inserting target metadata or an operation of modifying pre-stored metadata into target metadata, and the target metadata includes a mapping relationship from the logical space to the physical space;
the second metadata node 404 is configured to execute the update operation according to the metadata instruction when the first metadata node 403 fails and operates normally, and backup an update result to a preset storage space.
The apparatus for processing a data request by a storage system of this embodiment is used to implement the method for processing a data request by a storage system, so the specific implementation of the apparatus can be found in the embodiment of the method for processing a data request by a storage system in the foregoing, and the function of the apparatus corresponds to the method embodiment, and is not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The method, the device and the storage system for processing the data request of the storage system provided by the invention are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (8)
1. A method of processing a data request by a storage system, the storage system comprising a first metadatum node and a second metadatum node, the method comprising:
responding to a write request sent by a client, and distributing a logic space and a physical space for data to be written according to an automatic thin provisioning strategy;
generating a metadata change instruction according to the logic space and the physical space;
when a first metadata node and a second metadata node both run normally, sending the metadata change instruction to the first metadata node so that the first metadata node executes an update operation according to the metadata change instruction and synchronizes an update result to the second metadata node, wherein the update operation comprises an operation of inserting target metadata or an operation of modifying pre-stored metadata into the target metadata, and the target metadata comprises a mapping relation from the logical space to the physical space;
when the first metadata node fails and the second metadata node normally operates, the metadata change instruction is sent to the second metadata node, so that the second metadata node can execute the updating operation according to the metadata instruction, and an updating result is backed up to a preset storage space;
before the responding to the write request sent by the client and allocating a logical space and a physical space for the data to be written according to the thin provisioning policy, the method further includes:
receiving a data request sent by a client;
judging whether the data request is a write request;
if the metadata is not the write request, searching corresponding metadata from the second metadata node according to the data request when the first metadata node and the second metadata node both operate normally, so as to determine the physical space where the data corresponding to the data request is located according to the metadata.
2. The method of claim 1, wherein before the sending the metadata change instruction to the second metadata node to facilitate the second metadata node to perform the update operation according to the metadata instruction and backup the update result to a preset storage space, further comprising:
acquiring an operation state parameter of the first metadata node;
and judging whether the first metadata node fails or not according to the running state parameters.
3. The method according to any of claims 1-2, wherein after said sending the metadata change instruction to the second metadata node, so that the second metadata node performs the update operation according to the metadata instruction, and backs up the update result to a preset storage space, further comprising:
judging whether the first metadata node recovers normal operation or not;
and if the normal operation is recovered, synchronizing the metadata in the second metadata node to the first metadata node, and releasing the preset storage space.
4. The method of claim 3, after the synchronizing metadata in the second metadata node to the first metadata node and releasing the preset storage space, further comprising:
judging whether all dirty data in the cache of the first metadata node are printed to a disk;
and if so, determining that the first metadata node and the second metadata node both operate normally.
5. An apparatus for a storage system to process a data request, the storage system comprising a first metadatum node and a second metadatum node, the apparatus comprising:
a space allocation module: the data writing method comprises the steps of responding to a writing request sent by a client, and distributing a logic space and a physical space for data to be written according to an automatic thin provisioning strategy;
a change instruction generation module: generating a metadata change instruction according to the logic space and the physical space;
a first transmitting module: the metadata updating method comprises the steps that when a first metadata node and a second metadata node both run normally, a metadata changing instruction is sent to the first metadata node, so that the first metadata node can execute updating operation according to the metadata changing instruction, and an updating result is synchronized to the second metadata node, wherein the updating operation comprises the operation of inserting target metadata or the operation of modifying pre-stored metadata into the target metadata, and the target metadata comprises the mapping relation from a logical space to a physical space;
a second sending module: the metadata change instruction is sent to the second metadata node when the first metadata node fails and the second metadata node normally operates, so that the second metadata node can execute the update operation according to the metadata instruction and backup an update result to a preset storage space;
wherein the apparatus further comprises:
a request receiving module: the data processing system is used for receiving a data request sent by a client;
a request judgment module: the data processing device is used for judging whether the data request is a write request or not;
a metadata lookup module: and when the data request is not a write request and both the first metadata node and the second metadata node operate normally, searching corresponding metadata from the second metadata node according to the data request so as to determine a physical space where data corresponding to the data request is located according to the metadata.
6. The apparatus of claim 5, wherein the apparatus comprises:
a state parameter acquisition module: the operation state parameter is used for acquiring the operation state parameter of the first metadata node;
a fault judgment module: and the first metadata node is used for judging whether the first metadata node has a fault according to the running state parameter.
7. The apparatus of any one of claims 5-6, further comprising:
and a normal recovery judging module: the first metadata node is used for judging whether the first metadata node recovers normal operation or not;
a releasing module: and the storage device is used for synchronizing the metadata in the second metadata node to the first metadata node and releasing the preset storage space when normal operation is recovered.
8. A storage system, comprising: the system comprises a client, a storage server, a first metadata node and a second metadata node;
the client is used for sending a write request;
the storage server is used for responding to the write request, distributing a logic space and a physical space for the data to be written according to an automatic thin provisioning strategy and generating a metadata change instruction;
the first metadata node is used for executing updating operation according to the metadata change instruction when the first metadata node and the second metadata node both run normally, and synchronizing an updating result to the second metadata node, wherein the updating operation comprises an operation of inserting target metadata or an operation of modifying pre-stored metadata into the target metadata, and the target metadata comprises a mapping relation from the logic space to the physical space;
the second metadata node is used for executing the updating operation according to the metadata instruction when the first metadata node fails and normally operates per se, and backing up an updating result to a preset storage space;
the storage server is further configured to: receiving a data request sent by a client before distributing a logic space and a physical space for data to be written according to an automatic thin provisioning strategy in response to a write request sent by the client; judging whether the data request is a write request; if the metadata is not the write request, searching corresponding metadata from the second metadata node according to the data request when the first metadata node and the second metadata node both operate normally, so as to determine the physical space where the data corresponding to the data request is located according to the metadata.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910290063.8A CN110018796B (en) | 2019-04-11 | 2019-04-11 | Method and device for processing data request by storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910290063.8A CN110018796B (en) | 2019-04-11 | 2019-04-11 | Method and device for processing data request by storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110018796A CN110018796A (en) | 2019-07-16 |
| CN110018796B true CN110018796B (en) | 2022-07-08 |
Family
ID=67191092
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910290063.8A Active CN110018796B (en) | 2019-04-11 | 2019-04-11 | Method and device for processing data request by storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110018796B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112578996A (en) * | 2019-09-30 | 2021-03-30 | 华为技术有限公司 | Method for sending metadata of storage system and storage system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102122306A (en) * | 2011-03-28 | 2011-07-13 | 中国人民解放军国防科学技术大学 | Data processing method and distributed file system applying same |
| CN105718217A (en) * | 2016-01-18 | 2016-06-29 | 浪潮(北京)电子信息产业有限公司 | Method and device for maintaining data consistency of thin provisioning database |
-
2019
- 2019-04-11 CN CN201910290063.8A patent/CN110018796B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102122306A (en) * | 2011-03-28 | 2011-07-13 | 中国人民解放军国防科学技术大学 | Data processing method and distributed file system applying same |
| CN105718217A (en) * | 2016-01-18 | 2016-06-29 | 浪潮(北京)电子信息产业有限公司 | Method and device for maintaining data consistency of thin provisioning database |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110018796A (en) | 2019-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11360854B2 (en) | Storage cluster configuration change method, storage cluster, and computer system | |
| CN105933391B (en) | A kind of node expansion method, apparatus and system | |
| US10817478B2 (en) | System and method for supporting persistent store versioning and integrity in a distributed data grid | |
| JP6141189B2 (en) | Provides transparent failover in file systems | |
| CN104679579B (en) | Virtual machine migration method and device in a kind of group system | |
| CN106062717A (en) | Distributed storage replication system and method | |
| CN111651523B (en) | MySQL data synchronization method and system of Kubernetes container platform | |
| CN103207841A (en) | Method and device for data reading and writing on basis of key-value buffer | |
| CN111680015B (en) | File resource processing method, device, equipment and medium | |
| CN103152434A (en) | Leader node replacing method of distributed cloud system | |
| CN103986779B (en) | A kind of application data transmission method, device and terminal device | |
| CN113268472B (en) | Distributed data storage system and method | |
| CN106331081B (en) | Information synchronization method and device | |
| CN105357042A (en) | High-availability cluster system, master node and slave node | |
| CN104753987B (en) | A kind of distributed conversation management method and system | |
| CN105069152A (en) | Data processing method and apparatus | |
| US7315959B2 (en) | Real-time remote backup system and related method | |
| CN110018796B (en) | Method and device for processing data request by storage system | |
| CN110351313A (en) | Data cache method, device, equipment and storage medium | |
| CN113190619B (en) | Data read-write method, system, equipment and medium for distributed KV database | |
| CN107168774B (en) | Virtual machine migration method and system based on local storage | |
| CN105323271B (en) | Cloud computing system and processing method and device thereof | |
| CN111917588A (en) | Edge device management method, device, edge gateway device and storage medium | |
| CN102577249B (en) | The example set of the connection of dynamic addressing main frame | |
| CN115314361B (en) | Server cluster management method and related components thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |