WO2015081521A1 - Procede, appareil et systeme de traitement de donnees dans un systeme de memoire - Google Patents

Procede, appareil et systeme de traitement de donnees dans un systeme de memoire Download PDF

Info

Publication number
WO2015081521A1
WO2015081521A1 PCT/CN2013/088577 CN2013088577W WO2015081521A1 WO 2015081521 A1 WO2015081521 A1 WO 2015081521A1 CN 2013088577 W CN2013088577 W CN 2013088577W WO 2015081521 A1 WO2015081521 A1 WO 2015081521A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
host
snapshot
storage device
file system
Prior art date
Application number
PCT/CN2013/088577
Other languages
English (en)
Chinese (zh)
Inventor
梁莹
张鹏
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201380002234.3A priority Critical patent/CN103842954B/zh
Priority to PCT/CN2013/088577 priority patent/WO2015081521A1/fr
Publication of WO2015081521A1 publication Critical patent/WO2015081521A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1448—Management of the data involved in backup or backup restore
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/82—Solving problems relating to consistency

Definitions

  • the present invention relates to the field of communications technologies, and in particular, to a data processing method, apparatus, and system in a storage system. Background technique
  • the embodiment of the invention provides a data processing method, device and system in a storage system, which can not only perform snapshot and copying of a block storage device and a file system at the same time, but also can uniformly manage data on a block storage device and a file system. Data protection.
  • an embodiment of the present invention provides a data processing method in a storage system, including: establishing a Consistency Group for application snapshot and replication;
  • LUN logical unit number
  • the block storage device logical unit number used in adding the database of the application in the consistency group, and a separate file storing related files of the application System volume including:
  • a separate file system volume of the associated file of the application is obtained from the file system, and the independent file system volume is stored in the consistency group.
  • the snapshot is performed on the block storage device and the file system according to the snapshot policy, including:
  • the sending, by the snapshot policy, the snapshot command to the block storage device and the file system according to the snapshot policy including: by the primary controller Sending a snapshot command to the block storage device and the file system according to the snapshot policy;
  • the blocking host writes data to the disk and creates a protection time point, including: by the primary controller
  • the secondary controller sends a blocking command for instructing the blocking host to write data, so that the secondary controller blocks the data sent by the host in the cache according to the blocking command, and the primary controller sends a command to create a protection time point to the secondary controller. , causing the secondary controller to create a protection time point according to the command to create a protection time point.
  • the determining a replication policy of the consistency group, and performing remote data replication on the block storage device and the file system according to the replication policy including:
  • Remote data replication is performed according to the protection time point.
  • the preventing the host from writing data to the disk according to the replication policy, and creating a protection time point including: by the primary controller Sending, according to the replication policy, a blocking command for instructing the blocking host to write data to the secondary controller, so that the secondary controller blocks the data sent by the host in the cache according to the blocking command;
  • the primary controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command for creating a protection time point.
  • the secondary controller blocks the data sent by the host in the cache according to the blocking command, including:
  • the secondary controller sets a blocking identifier according to the blocking command
  • the secondary controller When the secondary controller receives the data sent by the host, if it is determined that the blocking identifier exists, the data is blocked in the cache;
  • releasing the blocking of the data written to the host includes: deleting the blocking identifier by the secondary controller.
  • an embodiment of the present invention further provides a data processing apparatus in a storage system, including a establishing unit, an adding unit, and a snapshot unit and/or a copying unit;
  • An adding unit a logical unit number of a block storage device used to add a database of the application in the consistency group, and a separate file system volume storing related files of the application; a snapshot unit, configured to determine a snapshot policy of the consistency group, and simultaneously snapshot the block storage device and the file system according to the snapshot policy;
  • a copy unit is configured to determine a replication policy of the consistency group, and perform remote data replication on the block storage device and the file system according to the replication policy.
  • the adding unit is specifically configured to acquire a logical unit number of the block storage device used by the database of the application from the block storage device, add the logical unit number to the consistency group, and, from the file system Obtaining a separate file system volume of the related file of the application, and storing the independent file system volume in the consistency group.
  • the snapshot unit in combination with the first possible implementation manner of the second aspect, includes a snapshot subunit, a first blocking subunit, and a first disarming subunit;
  • the snapshot subunit is configured to send a snapshot command to the block storage device and the file system according to the snapshot policy;
  • the first blocking sub-unit is configured to block the host to write data to the disk and create a protection time point; the first releasing sub-unit is configured to release the blocking of the data written to the host after the protection time point .
  • the snapshot subunit is specifically configured to send a snapshot command to the block storage device and the file system according to the snapshot policy by using the primary controller;
  • the first blocking sub-unit is configured to send, by using the main controller, a blocking command for instructing the blocking host to write data to the secondary controller, so that the secondary controller blocks the data sent by the host in the cache according to the blocking command.
  • the primary controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command for creating a protection time point.
  • the copying unit in combination with the first possible implementation manner of the second aspect, includes a copying subunit, a second blocking subunit, and a second releasing subunit;
  • the second blocking subunit is configured to prevent the host from writing data to the disk according to the copying policy, and creating a protection time point;
  • the second release subunit is configured to cancel writing data to the host after the protection time point Blocking
  • the replicating subunit is configured to perform remote data replication according to the protection time point.
  • the second blocking sub-unit is configured to send, by using the primary controller, a blocking command for instructing the blocking host to write data to the secondary controller according to the replication policy, so that the secondary controller sends the host according to the blocking command.
  • the data is blocked in the cache; the primary controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command to create a protection time point.
  • an embodiment of the present invention further provides a communication system, including a data processing device in any one of the storage systems provided by the embodiments of the present invention.
  • the embodiment of the present invention further provides a storage management server, including a processor, a transceiver device for transmitting and receiving data, and a memory for storing data;
  • the processor is configured to establish a consistency group for application snapshot and replication; add a logical unit number of a block storage used by the application database in the consistency group, and independently store related files of the application a file system volume; determining a snapshot policy of the consistency group, and simultaneously taking snapshots of the block storage and the network shared file system according to the snapshot policy; and/or determining a replication policy of the consistency group, and determining the block according to the replication policy Storage and network shared file systems for remote data replication.
  • the processor is specifically configured to acquire, from a block storage, a logical unit number of a block storage used by the database of the application, add the logical unit number to the consistency group, and acquire the file system from the file system.
  • the processor is specifically configured to send a snapshot command to the block storage and file system according to the snapshot policy; block the host to write data to the disk, and create a protection time point; after the protection time point, release the pair Blocking of data written by the host.
  • the processor is specifically configured to prevent the host from writing data to the disk according to the replication policy, and create a protection time point; after the protection time point, unblock the data written to the host; according to the protection time point Perform remote data replication.
  • the embodiment of the present invention establishes a consistency group for application snapshot and replication, adds a logical unit number of the block storage device used by the application database in the consistency group, and a separate file system volume for storing related files of the application, Determining a snapshot policy of the consistency group, and simultaneously taking snapshots of the block storage device and the file system according to the snapshot policy; and/or determining a replication policy of the consistency group, and remotely controlling the block storage device and the file system according to the replication policy Data replication; because the scheme establishes a consistency group for application snapshots and replication, not only can the consistency group be used for simultaneous snapshot and replication of block storage devices and file systems, but also block storage. Unified management and data protection of data on devices and file systems. DRAWINGS
  • FIG. 1 is a flowchart of a data processing method in a storage system according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of a data processing method in a storage system according to an embodiment of the present invention
  • FIG. 2 is a flowchart of a snapshot in a data processing method in a storage system according to an embodiment of the present invention
  • FIG. 3 is a data processing in a storage system according to an embodiment of the present invention
  • FIG. 3 is a flowchart of a data processing method in a storage system provided by an embodiment of the present invention
  • FIG. 4 is a schematic structural diagram of a data processing apparatus in a storage system preparation according to an embodiment of the present invention
  • FIG. 5 is a schematic structural diagram of a storage management server in a storage system according to an embodiment of the present invention.
  • Embodiments of the present invention provide a data processing method, apparatus, and system in a storage system. The details are described below separately.
  • Embodiment 1
  • the embodiment of the present invention will be described from the perspective of a data processing device in a storage system.
  • the data processing device in the storage system may be integrated into a device such as a storage management server (Storage Management Server).
  • Storage Management Server Storage Management Server
  • a data processing method in a storage system comprising: establishing a consistency group for application snapshot and replication; adding a logical unit number LUN of a block storage device used by the application database in the consistency group, and storing the application a separate file system volume of the related file; determining a snapshot policy of the consistency group, and simultaneously taking snapshots of the block storage device and the file system according to the snapshot policy; and/or determining a replication policy of the consistency group, and according to the replication policy Block storage devices and file systems for remote data replication.
  • the consistency group refers to a collection of data volumes in which multiple data volumes can be saved; since the consistency group established in step 101 is for the snapshot and copy of the application, therefore, The consistency group is primarily used to store data and/or files related to snapshots and replication of the application.
  • step 101 a logical unit number of a block storage device used for adding a database of the application in the consistency group, and a separate file system volume for storing related files of the application are established; for example, the specifics may be as follows:
  • a separate file system volume of the relevant file of the application is obtained from the file system, and the independent file system volume is stored in the consistency group.
  • the file system may be specifically a network shared file system (NFS, Network File System) or a common Internet file system (CIFS, Common Internet File Systems), and the file system may be integrated into a network attached storage (NAS). In the device, it will not be described here.
  • NFS Network File System
  • CIFS Common Internet File Systems
  • steps 103 and/or 104 may be performed as follows:
  • the snapshot policy of the consistency group is determined, and the block storage device and the file system (such as the network shared file system) are simultaneously snapshotd according to the snapshot policy; for example, the details may be as follows:
  • the details may be as follows:
  • the master controller simultaneously sends snapshot commands to the block storage device and the file system according to the determined snapshot policy.
  • the block storage device and the file system can take snapshots of the data or files saved by themselves according to the snapshot command, thereby generating a snapshot.
  • the block storage device and the file system can perform snapshots in various ways.
  • COW Copy on Write
  • COFW Copy on Fist Write
  • Snapshot algorithms such as ROW, Redirect on Write, also called ROFW, Redirect on Fist Write.
  • a snapshot algorithm such as Qtree (Q-tree) or binary tree can be used, and will not be described here.
  • the snapshot policy can be set according to the requirements of the actual application, and is not described here.
  • the data processing device in the storage system includes the primary controller and the secondary controller as an example, the details may be as follows:
  • the primary controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command to create a protection time point.
  • the secondary controller can prevent the host from writing data in various ways. For example, you can indicate whether to prevent the host from writing data by setting a blocking flag. For example, if the blocking identifier exists, it indicates that the host needs to prevent the host from writing data to the disk; otherwise, if the blocking identifier does not exist, it indicates that the host does not need to prevent the host from writing data to the disk, and the host can directly write data to the disk. ,and many more.
  • the received data when it is necessary to prevent the host from writing data to the disk, the received data may be first stored in the cache, that is, the step "the secondary controller blocks the data sent by the host according to the blocking command in the cache" may include: Setting a blocking identifier according to the blocking command, when the secondary controller receives the sending by the host In the case of data, if it is determined that the blocking identifier exists, the data is blocked in the cache.
  • the host Since the host is prevented from writing new data to the disk at this time, the consistency of the data on the block storage device and the file system corresponding to the protection time point can be ensured. Therefore, the snapshots made by the block storage device and the file system will also be consistent.
  • the blocking flag is set in (2) to indicate that the host needs to be prevented from writing data
  • the blocking identifier can be deleted by the secondary controller. In this way, the data can be written to the disk later, for example, the data that was previously stored in the cache can be written to the disk, and so on.
  • 104 Determine a replication policy of the consistency group, and perform remote data replication on the block storage device and the file system (such as a network shared file system) according to the replication policy; for example, the following may be specifically: (1) blocking the host according to the replication policy Write data to disk and create a protection point in time.
  • the data processing device in the storage system includes the primary controller and the secondary controller as an example, the details may be as follows:
  • the primary controller sends a blocking command for instructing the blocking host to write data to the secondary controller according to the replication policy, so that the secondary controller blocks the data sent by the host in the cache according to the blocking command;
  • the command at the time point is given to the secondary controller, so that the secondary controller creates a protection time point according to the command to create a protection time point.
  • the secondary controller can prevent the host from writing data in various ways.
  • the blocking identifier can be set to indicate whether to prevent the host from writing data. For example, if the blocking identifier exists, it indicates that the host needs to prevent the host from writing data to the disk; otherwise, if the blocking identifier does not exist, it indicates that the host does not need to prevent the host from writing data to the disk, and the host can directly write data to the disk. ,and many more.
  • the received data may be first stored in the cache, that is, the step "the secondary controller blocks the data sent by the host according to the blocking command in the cache" may include: The device sets the blocking identifier according to the blocking command. When the secondary controller receives the data sent by the host, if it determines that the blocking identifier exists, the data is blocked in the cache.
  • the host Since the host is prevented from writing new data to the disk at this time, the consistency of the data on the block storage device and the file system corresponding to the protection time point can be ensured. (2) After the protection time point, the blocking of the data written to the host is released.
  • the blocking identifier can be deleted by the secondary controller.
  • the data can be written to the disk later, for example, the data that was previously stored in the cache can be written to the disk, and so on.
  • the embodiment establishes a consistency group for application snapshot and replication, adds a logical unit number of the block storage device used by the application database in the consistency group, and a separate file for storing related files of the application.
  • a system volume which determines a snapshot policy of the consistency group, and simultaneously snapshots the block storage device and the file system (such as a network shared file) system according to the snapshot policy; and/or determines a replication policy of the consistency group, and according to the replication
  • the policy performs remote data replication on block storage devices and file systems, such as network shared file systems.
  • Embodiment 2 Since the solution establishes a consistency group for application snapshot and replication, not only can the consistency group be used to realize the simultaneous snapshot and copy of the block storage device and the file system (such as a network shared file system), but also Unified management and data protection for data on block storage devices and file systems such as network shared file systems. According to the method described in the first embodiment, further details will be exemplified in the second and third embodiments.
  • Embodiment 2 Embodiment 2
  • a snapshot is taken as an example for further detailed description.
  • the data processing device in the storage system is specifically integrated into the storage management server, and the file system is specifically a network shared file system and is installed on a network attached storage device (NAS, Network Attached). Storage is described as an example.
  • NAS network attached storage device
  • the figure is a schematic diagram of a scenario of a data processing method in the storage system.
  • the storage system may include a host, a block storage device, a network attached storage device, and a storage management server.
  • the host, the block storage device, and the network attached storage device can communicate through a storage area network (SAN).
  • the host can also connect the block storage device, the network attached storage device, and the storage management server through a local area network (LAN).
  • Block storage devices and network attached storage devices are respectively composed of multiple storage orders. Yuan composition, no longer repeat here.
  • the functions of each device of the storage system may be specifically as follows:
  • the logical unit number of the block storage device used by the application database may be added to the consistency group of the storage management server, and when the snapshot command issued by the storage management server is received, the stored application according to the snapshot command The database is snapshotted to generate a snapshot of the required data.
  • Used to store related files of the application receive data written by the host, and send data that the host needs to read to the host, and so on.
  • the independent file system volume of the related file of the application may be added to the consistency group of the storage management server, and when the snapshot command sent by the storage management server is received, the related file of the stored application is performed according to the snapshot command. Snapshots to generate a snapshot of the required data.
  • the storage management server includes storage management software, which can be used to establish a consistency group for application snapshots and replications, receive a logical unit number used by a database of applications sent by the block storage device, and receive related files of applications sent by the network attached storage device. a separate file system volume in which the logical unit number used by the application database and the independent file system volume storing the relevant files of the application are added to the consistency group, and the snapshot policy of the consistency group is determined, and according to the snapshot policy pair
  • the block storage device and the network attached storage device that is, the file system, such as the network shared file system
  • perform snapshots at the same time for example, the snapshot command can be issued to the block storage device and the network attached storage device at the same time, and the like.
  • a data processing method in a storage system may be as follows: 201.
  • the storage management server establishes a consistency group for application snapshots and replication.
  • the consistency group refers to a collection of data volumes, in which multiple data volumes can be saved; since the consistency group established in step 201 is for the snapshot and copy of the application, therefore, The consistency group is primarily used to store data and/or files related to snapshots and replication of the application.
  • the block storage device sends the logical unit number of the block storage device used by the application database to the storage management server, and then performs step 203.
  • the storage management server adds the logical unit number of the block storage device used by the database of the received application to the consistency group.
  • the network attached storage device sends a separate file system curl of the related file of the application to the storage management server, and then performs step 205.
  • the storage management server adds the independent file system volume of the related file of the received application to the consistency group.
  • steps 202 and 204 may be in no particular order.
  • the storage management server determines a snapshot policy of the consistency group, and simultaneously snapshots the block storage device and the file system according to the snapshot policy. For example, a snapshot command is issued to a block storage device and a network attached storage device. For example, see Figure 2c for details.
  • the process can be as follows:
  • the primary controller simultaneously sends a snapshot command to the block storage device and the network attached storage device according to the determined snapshot policy.
  • the block storage device and the network attached storage device can take snapshots of the data or files saved by themselves according to the snapshot command, thereby generating a snapshot.
  • the algorithm for performing snapshots on the block storage device and the network attached storage device may be various.
  • a snapshot algorithm such as COW or ROW may be used, and for a network attached storage device (and a file system), Snapshot algorithms such as Qtree or binary tree are not described here.
  • the snapshot policy can be set according to the requirements of the actual application, and is not described here.
  • the primary controller sends a blocking command to the secondary controller to instruct the blocking host to write data.
  • the secondary controller sets a blocking identifier according to the blocking command. For example, the input/output (I/O, Input/Output) may be set to be marked, and a response message indicating completion of the blocking command is returned to the primary controller. 2064.
  • the host continues to write data to the secondary controller, such as writing I/O data.
  • the secondary controller determines whether there is a blocking identifier. If yes, the data written by the host is blocked. If it does not exist, the data does not operate, that is, the host is allowed to continue to write data to the secondary controller.
  • the controller writes I/O data, writes the I/O data to disk, and so on.
  • the main controller sends a command to create a protection time point to the secondary controller.
  • the secondary controller creates a protection time point according to the command for creating a protection time point.
  • the secondary controller deletes the blocking identifier. For example, the previously set I/O lowering brush flag may be canceled, and a response message indicating completion of the command to create the protection time point is returned to the primary controller.
  • the secondary controller writes the I/O data in the cache to the disk, and returns a response message indicating that the host writes successfully to the host.
  • the embodiment establishes a consistency group for application snapshot and replication, adds a logical unit number of the block storage device used by the application database in the consistency group, and a separate file for storing related files of the application.
  • the system volume determines the snapshot policy of the consistency group and simultaneously snapshots the block storage device and the network attached storage device (and file system) according to the snapshot policy. Since the scheme establishes a consistency group for application snapshot and replication, the consistency group can be used not only to realize the simultaneous snapshot operation of the block storage device and the network attached storage device, but also to block storage devices and networks. Data on the attached storage device for unified management and data protection.
  • Embodiment 3 Embodiment 3
  • the storage system may include a host, a block storage device, a network attached storage device, and a storage management server as an example.
  • the second embodiment is different from the second embodiment.
  • the remote data replication will be taken as an example.
  • the structure of the storage system is the same as that of the storage system in the second embodiment, and details are not described herein again.
  • Each device in the storage system can perform the following functions:
  • Mainly used to write or read data to block storage devices and network attached storage devices for example, storing the application database in a block storage device, and storing related files of the application to the network attached Storage devices, and more.
  • the root Used to store related files of the application, receive data written by the host, and send data that the host needs to read to the host, and so on. And when receiving the copy command issued by the storage management server, the root (4) stores the management server;
  • the storage management server is mainly used to establish a consistency group for application snapshot and replication, receive a logical unit number of a block storage device used by a database of an application sent by the block storage device, and receive related files of an application sent by the network attached storage device.
  • a separate file system volume label in which the logical unit number of the block storage device used by the application database and the independent file system volume volume label of the relevant file storing the application are added to the consistency group, and the replication policy of the consistency group is determined.
  • the remote storage data is simultaneously performed on the block storage device and the network attached storage device (that is, the file system, such as the network shared file system). For example, the copy command can be issued to the block storage device and the network attached storage device at the same time, etc. Wait.
  • a data processing method in a storage system may be as follows: 301.
  • a storage management server establishes a consistency group for application snapshot and replication.
  • the consistency group refers to a collection of data volumes in which multiple data volumes can be saved; since the consistency group established in step 301 is for the snapshot and replication of the application, therefore, The consistency group is primarily used to store data and/or files related to snapshots and replication of the application.
  • the block storage device sends the logical unit number of the block storage device used by the application database to the storage management server, and then performs step 303.
  • the logic of the block storage device used by the storage management server to receive the database of the application
  • the unit number is added to the consistency group.
  • the network attached storage device sends a separate file system curl of the related file of the application to the storage management server, and then performs step 305.
  • the storage management server adds the independent file system volume of the related file of the received application to the consistency group.
  • steps 302 and 304 may be in no particular order.
  • the storage management server determines a replication policy of the consistency group, and performs remote data replication on the block storage device and the network attached storage device (that is, a file system, such as a network shared file system) according to the replication policy.
  • a replication policy of the consistency group that is, a file system, such as a network shared file system
  • the process can be as follows:
  • the primary controller of the storage device sends a blocking command for instructing the blocking host to write data to the secondary controller according to the replication policy.
  • the replication policy can be set according to the requirements of the actual application, and is not described here.
  • the secondary controller sets a blocking identifier according to the blocking command. For example, the I/O lowering flag may be set, and a response message indicating completion of the blocking command is returned to the primary controller.
  • the host continues to write data to the secondary controller, such as writing I/O data.
  • the secondary controller determines whether there is a blocking identifier. If yes, the data written by the host is blocked. If it does not exist, the data does not operate, that is, the host is allowed to continue to write data to the secondary controller.
  • the controller writes I/O data, writes the I/O data to disk, and so on.
  • the main controller sends a command to create a protection time point to the secondary controller.
  • the secondary controller creates a protection time point according to the command for creating a protection time point.
  • the secondary controller deletes the blocking identifier. For example, the previously set I/O downstamp flag may be canceled, and a response message indicating completion of the command to create the protection time point is returned to the primary controller.
  • the secondary controller writes the I/O data in the cache to the disk, and returns a response message indicating that the host writes successfully to the host.
  • the secondary controller performs remote data replication according to the created protection time point.
  • this embodiment establishes a consistency group for snapshots and replications of applications, Adding a logical unit number of the block storage device used by the database of the application to the independent file system volume of the relevant file storing the application, determining a replication policy of the consistency group, and according to the replication policy, the block storage device and Network attached storage devices (ie file systems, such as network shared file systems) for remote data replication. Since the scheme establishes a consistency group for application snapshot and replication, the consistency group can be used not only to achieve the purpose of copying the block storage device and the network attached storage device at the same time, but also to block storage devices and networks. Data on the attached storage device for unified management and data protection.
  • the embodiment of the present invention further provides a data processing apparatus in a storage system.
  • the data processing apparatus in the storage system includes an establishing unit 401, an adding unit 402, and a snapshot unit 403. And copy unit 404;
  • the establishing unit 401 is configured to establish a consistency group for application snapshot and replication
  • the consistency group refers to a collection of data volumes in which multiple data volumes can be saved; since the consistency group established by the establishing unit 401 is for the snapshot and replication of the application, therefore, The consistency group is primarily used to store data and/or files related to snapshots and replication of the application.
  • the adding unit 402 is configured to add, in the consistency group established by the establishing unit 401, a logical unit number of the block storage device used by the application database, and a separate file system volume storing related files of the application;
  • a snapshot unit 403 configured to determine a snapshot policy of the consistency group, and simultaneously snapshot the block storage device and the file system according to the snapshot policy; and/or;
  • the copy unit 404 is configured to determine a replication policy of the consistency group, and perform remote data replication on the block storage device and the file system according to the replication policy.
  • the adding unit 402 is specifically configured to obtain a logical unit number of the block storage device used by the database of the application from the block storage device, add the logical unit number to the consistency group, and obtain the file system from the file system.
  • a separate file system volume of the application's associated file, the independent file system volume is placed in the consistency group.
  • the file system may be a network shared file system or a universal Internet file system.
  • the file system may be integrated into a device such as a network attached storage device, and details are not described herein.
  • the snapshot unit 403 may include a snapshot subunit, a first blocking subunit, and a first disarming subunit, as follows:
  • a snapshot subunit configured to send a snapshot command to the block storage device and the file system according to the snapshot policy
  • the snapshot subunit may be specifically configured to use the primary controller to simultaneously send a snapshot command to the block storage device and the file system according to the snapshot policy.
  • the block storage device and the file system can take snapshots of the data or files saved by themselves according to the snapshot command, thereby generating a snapshot.
  • the algorithm for snapshotting the block storage device and the file system may be various.
  • a snapshot algorithm such as COW (ie, COFW) or ROW (ie, ROFW) may be used, and for the file system, Qtree may be adopted. Snapshot algorithms such as binary trees are not described here.
  • a first blocking sub-unit configured to block the host to write data to the disk, and create a protection time point;
  • the first blocking sub-unit may be specifically configured to send, by the primary controller, the secondary controller to indicate that the blocking host writes
  • the data blocking command causes the secondary controller to block the data sent by the host in the cache according to the blocking command, and the primary controller sends a command to create a protection time point to the secondary controller, so that the secondary controller creates the protection time point according to the The command creates a protection time point.
  • the secondary controller can prevent the host from writing data in multiple ways.
  • the blocking identifier can be set to indicate whether to prevent the host from writing data. For example, if the blocking identifier exists, it indicates that the host needs to prevent the host from writing data to the disk. On the contrary, if the blocking identifier does not exist, it means that the host does not need to prevent the host from writing data to the disk. At this time, the host can directly write data to the disk, and the like, wherein when the host needs to prevent the host from writing data to the disk, The data received from the received data is stored in the cache. For details, refer to the previous embodiment, and details are not described herein.
  • the snapshot policy can be set according to the requirements of the actual application, and is not described here.
  • the first release subunit is configured to release the blocking of the data written to the host after the protection time point. For example, if the blocking identifier is used to indicate that the host needs to be prevented from writing data, then the blocking identifier can be deleted by the first releasing subunit, so that the data can be subsequently written to the disk, for example. , you can write data that was previously cached to disk, and so on.
  • the copy unit 404 may include a copy subunit, a second blocker unit, and a second release unit; a second blocking subunit, configured to prevent the host from writing data to the disk according to the replication policy, and creating a protection time point;
  • the second blocking sub-unit may be configured to send, by using the primary controller, a blocking command for instructing the blocking host to write data to the secondary controller according to the replication policy, so that the secondary controller sends the host according to the blocking command.
  • the data is blocked in the cache; the primary controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command to create a protection time point.
  • the secondary controller can prevent the host from writing data in multiple ways.
  • the blocking identifier can be set to indicate whether to prevent the host from writing data. For example, if the blocking identifier exists, it indicates that the host needs to prevent the host from writing data to the disk. On the contrary, if the blocking identifier does not exist, it means that the host does not need to prevent the host from writing data to the disk. At this time, the host can directly write data to the disk, and the like, wherein when the host needs to prevent the host from writing data to the disk, The data received from the received data is stored in the cache. For details, refer to the previous embodiment, and details are not described herein.
  • the second release subunit is configured to release the blocking of the data written to the host after the protection time point.
  • the second disarming unit can delete the blocking identifier, so that the data can be subsequently written to the disk, for example, You can write data that was previously cached to disk, and so on.
  • the data processing device in the storage system may be specifically integrated in a device such as a storage management server.
  • the foregoing units may be implemented as an independent entity, or may be implemented in any combination, and may be implemented as the same entity or a plurality of entities.
  • the foregoing refer to the foregoing embodiments, and details are not described herein.
  • the establishing unit 401 in the data processing apparatus in the storage system of the present embodiment can establish a consistency group for application snapshot and copy, and the adding unit 402 adds the block used by the application database in the consistency group. a logical unit number of the storage device, and a separate file system volume storing related files of the application, and then the snapshot unit 403 determines a snapshot policy of the consistency group, and simultaneously snapshots the block storage device and the file system according to the snapshot policy, and / or, the replication unit 404 determines the replication policy of the consistency group, and performs remote data replication on the block storage device and the file system according to the replication policy; since the scheme establishes a consistency group for application snapshot and replication, Can The consistency group is used to realize the simultaneous snapshot and copy of the block storage device and the file system, and the data on the block storage device and the file system can be uniformly managed and protected.
  • Embodiment 5 Embodiment 5
  • the embodiment of the present invention further provides a communication system, including the data processing device in any one of the storage systems provided by the embodiment of the present invention, where the data processing device in the storage system is specifically referred to the fourth embodiment, for example.
  • the data processing device in the storage system is specifically integrated in the storage management server, and the specifics may be as follows:
  • a storage management server for establishing a consistency group for application snapshots and replications; adding a logical unit number of the block storage device used by the application's database to the consistency group, and a separate file system volume storing related files of the application Determining a snapshot policy of the consistency group, and simultaneously taking snapshots of the block storage device and the file system according to the snapshot policy; and/or determining a replication policy of the consistency group, and performing block storage devices and file systems according to the replication policy
  • the remote data replication refer to the previous embodiment, and details are not described herein.
  • the communication system may also include other devices, such as a host, a block storage device (such as a block storage device, etc.), and a file system (which may be integrated in a network attached storage device), etc., as follows:
  • the host is mainly used for the forward block.
  • the storage device and the file system write or read data, for example, store the application database in a block storage device, and store related files of the application to the file system, and the like.
  • a block storage device is mainly used to store an application database, receive data written by the host, and send data that the host needs to read to the host, and the like.
  • the database of the stored application is snapshot according to the snapshot command; and/or when the copy command issued by the storage management server is received, the storage is performed according to the copy command.
  • the applied database is used for remote data replication.
  • the file system is mainly used to store related files of the application, receive data written by the host, and send data that the host needs to read to the host, and the like. And receiving a snapshot command sent by the storage management server, taking a snapshot of the related file of the stored application according to the snapshot command, thereby generating a snapshot of the required data; and/or receiving the copy command issued by the storage management server Remote data replication of the stored application's database based on the copy command.
  • the embodiment of the present invention further provides a storage management server, including a processor 501, a transceiver device 502 for transmitting and receiving data, and a memory 503 for storing data;
  • a processor 501 configured to establish a consistency group for application snapshot and replication, add a logical unit number of the block storage device used by the application database in the consistency group, and a separate file system volume that stores related files of the application; Determining a snapshot policy of the consistency group, and simultaneously taking snapshots of the block storage device and the file system according to the snapshot policy; and/or determining a replication policy of the consistency group, and remotely controlling the block storage device and the file system according to the replication policy Data replication.
  • the file system may be a network shared file system or a CIFS.
  • the file system may be integrated into a device such as a network attached storage device, and details are not described herein.
  • the processor 501 is specifically configured to obtain, from the block storage device, a logical unit number of the block storage device used by the database of the application, add the logical unit number to the consistency group, and, from the file system Obtain a separate file system volume of the relevant file of the application, and store the independent file system volume in the consistency group.
  • the processor 501 is specifically configured to: prevent the host from writing data to the disk according to the replication policy, and create a protection time point; after the protection time point, unblock the data written to the host; according to the protection time point Perform remote data replication.
  • the blocking identifier can be used to indicate whether to prevent the host from writing data. For example, if the blocking identifier is present, it indicates that the host needs to prevent the host from writing data to the disk. Otherwise, if the blocking identifier does not exist, it indicates that the host does not need to be blocked. Write data to disk. At this time, the host can directly write data to the disk, and so on. When the host needs to prevent the host from writing data to the disk, the received data can be stored in the cache first. For details, see the front. The description in the embodiments is not described herein again. For the specific implementation of the foregoing various devices, refer to the foregoing embodiments, and details are not described herein again.
  • the storage management server of this embodiment can establish a logical unit number of the block storage device used by the database of the application, and a storage unit for storing the application by establishing a consistency group for application snapshot and replication.
  • a separate file system volume of the related file determines a snapshot policy of the consistency group, and simultaneously snapshots the block storage device and the file system according to the snapshot policy, and/or determines a replication policy of the consistency group, and according to the replication policy Block storage device and file system for remote data replication; because the scheme establishes a consistency group for application snapshot and replication, not only can the consistency group be used to achieve simultaneous snapshot and replication of block storage devices and file systems.
  • unified management and data protection can be performed on the data on the block storage device and the file system.
  • the program may be stored in a computer readable storage medium, and the storage medium may include: Read only memory (ROM, Read Only Memory), random access memory (RAM), disk or optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé de traitement de données dans un système de mémoire, qui consiste à : établir un groupe de cohérence pour une copie instantanée d'application et une duplication ; ajouter le nombre d'unités logiques d'un dispositif à blocs mémoire utilisé par une base de données de l'application et le volume d'un système de fichiers indépendants, en vue du stockage d'un fichier correspondant de l'application dans le groupe de cohérence ; déterminer une règle de copie instantanée du groupe de cohérence, et prendre simultanément une copie instantanée du dispositif à blocs mémoire et d'un système de fichiers selon la règle de copie instantanée ; et/ou déterminer une règle de reproduction du groupe de cohérence, et mettre en oeuvre une reproduction à distance des données du dispositif à blocs mémoire et du système de fichiers selon la règle de reproduction. D'autres formes de réalisation de l'invention concernent également un appareil et un système de traitement de données ainsi qu'un dispositif correspondant d'un système de mémoire.
PCT/CN2013/088577 2013-12-05 2013-12-05 Procede, appareil et systeme de traitement de donnees dans un systeme de memoire WO2015081521A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201380002234.3A CN103842954B (zh) 2013-12-05 2013-12-05 一种存储系统中的数据处理方法、装置、系统和服务器
PCT/CN2013/088577 WO2015081521A1 (fr) 2013-12-05 2013-12-05 Procede, appareil et systeme de traitement de donnees dans un systeme de memoire

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2013/088577 WO2015081521A1 (fr) 2013-12-05 2013-12-05 Procede, appareil et systeme de traitement de donnees dans un systeme de memoire

Publications (1)

Publication Number Publication Date
WO2015081521A1 true WO2015081521A1 (fr) 2015-06-11

Family

ID=50804805

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2013/088577 WO2015081521A1 (fr) 2013-12-05 2013-12-05 Procede, appareil et systeme de traitement de donnees dans un systeme de memoire

Country Status (2)

Country Link
CN (1) CN103842954B (fr)
WO (1) WO2015081521A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10241869B2 (en) 2017-03-08 2019-03-26 International Business Machines Corporation Managing a deletion of a volume referenced by a snapshot of a consistency group
EP3796173A4 (fr) * 2018-07-05 2021-08-04 Huawei Technologies Co., Ltd. Procédé d'écriture de données et système de stockage

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107203327B (zh) * 2016-03-16 2020-02-28 伊姆西Ip控股有限责任公司 一种用于存储系统的方法和装置
CN106371767B (zh) * 2016-08-31 2019-08-20 华为技术有限公司 一种处理一致性业务的方法、装置和系统
CN111177260A (zh) * 2019-12-19 2020-05-19 上海上讯信息技术股份有限公司 数据库远程复制方法、装置及电子设备
US20230418961A1 (en) * 2022-06-28 2023-12-28 International Business Machines Corporation Fine granularity read access to generational safeguarded copy data

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7058662B2 (en) * 2000-11-30 2006-06-06 Xythos Software, Inc. Maintenance of data integrity during transfer among computer networks
CN102103530A (zh) * 2009-12-16 2011-06-22 成都市华为赛门铁克科技有限公司 快照方法、设备及系统
CN102722536A (zh) * 2012-05-22 2012-10-10 中兴通讯股份有限公司 基于分布式文件系统的快照实现方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7139887B2 (en) * 2003-12-31 2006-11-21 Veritas Operating Corporation Coordinated storage management operations in replication environment
GB0616257D0 (en) * 2006-08-16 2006-09-27 Ibm Storage management system for preserving consistency of remote copy data
CN101241456B (zh) * 2008-02-28 2011-07-06 成都市华为赛门铁克科技有限公司 一种数据保护方法和装置
CN102289510B (zh) * 2011-08-31 2013-10-09 华为数字技术（成都）有限公司 向生产卷写入数据的方法和装置
CN102819615A (zh) * 2012-08-30 2012-12-12 天津火星科技有限公司 一种基于应用快照的数据库持续数据保护方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7058662B2 (en) * 2000-11-30 2006-06-06 Xythos Software, Inc. Maintenance of data integrity during transfer among computer networks
CN102103530A (zh) * 2009-12-16 2011-06-22 成都市华为赛门铁克科技有限公司 快照方法、设备及系统
CN102722536A (zh) * 2012-05-22 2012-10-10 中兴通讯股份有限公司 基于分布式文件系统的快照实现方法及装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10241869B2 (en) 2017-03-08 2019-03-26 International Business Machines Corporation Managing a deletion of a volume referenced by a snapshot of a consistency group
EP3796173A4 (fr) * 2018-07-05 2021-08-04 Huawei Technologies Co., Ltd. Procédé d'écriture de données et système de stockage
US11500741B2 (en) 2018-07-05 2022-11-15 Huawei Technologies Co., Ltd. Data write method and storage system

Also Published As

Publication number Publication date
CN103842954B (zh) 2017-02-15
CN103842954A (zh) 2014-06-04

Similar Documents

Publication Publication Date Title
WO2015081521A1 (fr) Procede, appareil et systeme de traitement de donnees dans un systeme de memoire
US8170990B2 (en) Integrated remote replication in hierarchical storage systems
US10733060B2 (en) Asynchronous local and remote generation of consistent point-in-time snap copies in consistency groups
US20110055471A1 (en) Apparatus, system, and method for improved data deduplication
US20150220563A1 (en) Method for creating clone file, and file system adopting the same
WO2017215432A1 (fr) Procédé, dispositif et système de rétraction de données
WO2015085530A1 (fr) Procédé de réplication de données et système de stockage
US11797491B2 (en) Inofile management and access control list file handle parity
US20220382484A1 (en) Gransets for managing consistency groups of dispersed storage items
US11797213B2 (en) Freeing and utilizing unused inodes
KR20140106588A (ko) 공유 볼륨의 어플리케이션 일관된 스냅샷 기법
JP2008077288A (ja) データバックアップ方法及び計算機システム
US11449260B2 (en) Persistent hole reservation
US10416921B2 (en) Point-in-time snap copy on asynchronous consistency group management
JP2020502635A (ja) 整合したポイントインタイム・スナップ・コピーの非同期のローカル生成およびリモート生成のためのコンピュータ・プログラム、システム、および方法
US9760457B2 (en) System, method and computer program product for recovering stub files
US9830237B2 (en) Resynchronization with compliance data preservation
US11768624B2 (en) Resilient implementation of client file operations and replication
US20170318093A1 (en) Method and System for Focused Storage Access Notifications from a Network Storage System
US9111598B2 (en) Increased I/O rate for solid state storage
EP2372552B1 (fr) Déplacement automatisé d'un stockage de données protégées multi-sites en cours d'utilisation
US20230259529A1 (en) Timestamp consistency for synchronous replication
US8407301B1 (en) Systems and methods for creating incremental images of cluster volumes
CN111142789A (zh) 一种节点间配置信息同步的方法、装置、设备及存储介质
JP2013120463A (ja) 情報処理方法、情報処理システム、情報処理装置及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13898684

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13898684

Country of ref document: EP

Kind code of ref document: A1