WO2015081521A1

WO2015081521A1 - Data processing method, apparatus and system in storage system

Info

Publication number: WO2015081521A1
Application number: PCT/CN2013/088577
Authority: WO
Inventors: 梁莹; 张鹏
Original assignee: 华为技术有限公司
Priority date: 2013-12-05
Filing date: 2013-12-05
Publication date: 2015-06-11
Also published as: CN103842954B; CN103842954A

Abstract

A data processing method in a storage system comprises: establishing a consistency group for an application snapshot and a replication; adding a logical unit number of a block storage device used by a database of the application and an independent file system volume for storing a related file of the application into the consistency group; determining a snapshot policy of the consistency group, and simultaneously snapshoting the block storage device and a file system according to the snapshot policy; and/or determining a replication policy of the consistency group, and performing remote data replication for the block storage device and the file system according the replication policy. In addition, embodiments also provide a data processing apparatus and system and a related device in a storage system.

Description

Data processing method, device and system in storage system

The present invention relates to the field of communications technologies, and in particular, to a data processing method, apparatus, and system in a storage system. Background technique

With the development of communication technology, network management has gradually penetrated into various fields, and the data involved has become more and more large. Therefore, how to store and manage such data has become an extremely important issue.

More than 4 applications, especially database applications are deployed on block storage, because all mainstream databases have their own storage management layer, so only need to map the disk, but there are many cases where the index is stored in the database. In a block storage device, specific files stored in the database are stored in the file system and accessed through a network file system (NFS, Network File System) or a Common Internet File System (CIFS). That is, the complete database data at this point includes the data stored on the block storage device and the data stored on the file system. In order to improve the security of data and avoid data loss, in the prior art, data is often snapshotted or copied for subsequent operations such as data backup and copy/remote replication disaster recovery. For this way of storing the database data separately, it is often necessary to separately snapshot or copy the data in the block storage device and the file system, and this may result in inconsistent timestamps on both sides, which in turn leads to a block of the snapshot point database. Data and file system data are inconsistent, and eventually the database may not be pulled up, or the corresponding files are inconsistent. Therefore, it must be ensured that the snapshot or copy operations in the block storage device and the file system can be performed simultaneously at the same possible point in time. In the prior art, it can generally be implemented by a backup script, so that the script runs on both sides at the same time, thereby achieving the purpose of simultaneously snapshotting or copying the block storage device and the file system (such as a network shared file system).

In the research and practice of the prior art, the inventors of the present invention have found that although the existing solution can realize the purpose of simultaneous snapshot or copying of the block storage device and the file system, the data on the block storage device and the file system are required. Management separately, data recovery also needs to be separately restored, so it is very inconvenient in management and data protection. Summary of the invention The embodiment of the invention provides a data processing method, device and system in a storage system, which can not only perform snapshot and copying of a block storage device and a file system at the same time, but also can uniformly manage data on a block storage device and a file system. Data protection.

In a first aspect, an embodiment of the present invention provides a data processing method in a storage system, including: establishing a Consistency Group for application snapshot and replication;

Adding, in the consistency group, a logical unit number (LUN) of a block storage device used by the application database, and a separate file system volume storing related files of the application;

Determining a snapshot policy for the consistency group and simultaneously taking snapshots of the block storage device and the file system according to the snapshot policy; and/or,

Determine the replication policy for the consistency group and perform remote data replication on the block storage device and file system based on the replication policy.

In a first possible implementation, in combination with the first aspect, the block storage device logical unit number used in adding the database of the application in the consistency group, and a separate file storing related files of the application System volume, including:

Obtaining, from the block storage device, a logical unit number of the block storage device used by the database of the application, adding the logical unit number to the consistency group; and

A separate file system volume of the associated file of the application is obtained from the file system, and the independent file system volume is stored in the consistency group.

In a second possible implementation manner, in combination with the first possible implementation manner of the first aspect, the snapshot is performed on the block storage device and the file system according to the snapshot policy, including:

Simultaneously sending a snapshot command to the block storage device and the file system according to the snapshot policy; blocking the host to write data to the disk, and creating a protection time point;

After the protection time point, the blocking of the data written to the host is released.

In a third possible implementation, in combination with the second possible implementation manner of the first aspect, the sending, by the snapshot policy, the snapshot command to the block storage device and the file system according to the snapshot policy, including: by the primary controller Sending a snapshot command to the block storage device and the file system according to the snapshot policy;

The blocking host writes data to the disk and creates a protection time point, including: by the primary controller The secondary controller sends a blocking command for instructing the blocking host to write data, so that the secondary controller blocks the data sent by the host in the cache according to the blocking command, and the primary controller sends a command to create a protection time point to the secondary controller. , causing the secondary controller to create a protection time point according to the command to create a protection time point.

In a fourth possible implementation manner, in combination with the first possible implementation manner of the first aspect, the determining a replication policy of the consistency group, and performing remote data replication on the block storage device and the file system according to the replication policy , including:

Blocking the host from writing data to the disk according to the replication policy, and creating a protection time point; after the protection time point, unblocking the data written to the host;

Remote data replication is performed according to the protection time point.

In a fifth possible implementation manner, in combination with the fourth possible implementation manner of the first aspect, the preventing the host from writing data to the disk according to the replication policy, and creating a protection time point, including: by the primary controller Sending, according to the replication policy, a blocking command for instructing the blocking host to write data to the secondary controller, so that the secondary controller blocks the data sent by the host in the cache according to the blocking command;

The primary controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command for creating a protection time point.

In a sixth possible implementation manner, in combination with the fifth possible implementation manner of the first aspect, the secondary controller blocks the data sent by the host in the cache according to the blocking command, including:

The secondary controller sets a blocking identifier according to the blocking command;

When the secondary controller receives the data sent by the host, if it is determined that the blocking identifier exists, the data is blocked in the cache;

After the protection time point, releasing the blocking of the data written to the host includes: deleting the blocking identifier by the secondary controller.

In a second aspect, an embodiment of the present invention further provides a data processing apparatus in a storage system, including a establishing unit, an adding unit, and a snapshot unit and/or a copying unit;

Establishing a unit for establishing a consistency group for application snapshots and replication;

An adding unit, a logical unit number of a block storage device used to add a database of the application in the consistency group, and a separate file system volume storing related files of the application; a snapshot unit, configured to determine a snapshot policy of the consistency group, and simultaneously snapshot the block storage device and the file system according to the snapshot policy;

A copy unit is configured to determine a replication policy of the consistency group, and perform remote data replication on the block storage device and the file system according to the replication policy.

In a first possible implementation, the second aspect is combined, wherein:

The adding unit is specifically configured to acquire a logical unit number of the block storage device used by the database of the application from the block storage device, add the logical unit number to the consistency group, and, from the file system Obtaining a separate file system volume of the related file of the application, and storing the independent file system volume in the consistency group.

In a second possible implementation manner, in combination with the first possible implementation manner of the second aspect, the snapshot unit includes a snapshot subunit, a first blocking subunit, and a first disarming subunit;

The snapshot subunit is configured to send a snapshot command to the block storage device and the file system according to the snapshot policy;

The first blocking sub-unit is configured to block the host to write data to the disk and create a protection time point; the first releasing sub-unit is configured to release the blocking of the data written to the host after the protection time point .

In a third possible implementation, in combination with the second possible implementation of the second aspect, wherein:

The snapshot subunit is specifically configured to send a snapshot command to the block storage device and the file system according to the snapshot policy by using the primary controller;

The first blocking sub-unit is configured to send, by using the main controller, a blocking command for instructing the blocking host to write data to the secondary controller, so that the secondary controller blocks the data sent by the host in the cache according to the blocking command. The primary controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command for creating a protection time point.

In a fourth possible implementation, in combination with the first possible implementation manner of the second aspect, the copying unit includes a copying subunit, a second blocking subunit, and a second releasing subunit;

The second blocking subunit is configured to prevent the host from writing data to the disk according to the copying policy, and creating a protection time point;

The second release subunit is configured to cancel writing data to the host after the protection time point Blocking

The replicating subunit is configured to perform remote data replication according to the protection time point.

In a fifth possible implementation, in combination with the fifth possible implementation of the second aspect, wherein:

The second blocking sub-unit is configured to send, by using the primary controller, a blocking command for instructing the blocking host to write data to the secondary controller according to the replication policy, so that the secondary controller sends the host according to the blocking command. The data is blocked in the cache; the primary controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command to create a protection time point.

In a third aspect, an embodiment of the present invention further provides a communication system, including a data processing device in any one of the storage systems provided by the embodiments of the present invention.

In a fourth aspect, the embodiment of the present invention further provides a storage management server, including a processor, a transceiver device for transmitting and receiving data, and a memory for storing data;

The processor is configured to establish a consistency group for application snapshot and replication; add a logical unit number of a block storage used by the application database in the consistency group, and independently store related files of the application a file system volume; determining a snapshot policy of the consistency group, and simultaneously taking snapshots of the block storage and the network shared file system according to the snapshot policy; and/or determining a replication policy of the consistency group, and determining the block according to the replication policy Storage and network shared file systems for remote data replication.

In a first possible implementation, the second aspect is combined, wherein:

The processor is specifically configured to acquire, from a block storage, a logical unit number of a block storage used by the database of the application, add the logical unit number to the consistency group, and acquire the file system from the file system. A separate file system volume of the associated file of the application, the independent file system volume being stored in the consistency group.

In a second possible implementation, in combination with the first possible implementation of the second aspect, wherein:

The processor is specifically configured to send a snapshot command to the block storage and file system according to the snapshot policy; block the host to write data to the disk, and create a protection time point; after the protection time point, release the pair Blocking of data written by the host.

In a third possible implementation, in combination with the first possible implementation of the second aspect, wherein: The processor is specifically configured to prevent the host from writing data to the disk according to the replication policy, and create a protection time point; after the protection time point, unblock the data written to the host; according to the protection time point Perform remote data replication.

The embodiment of the present invention establishes a consistency group for application snapshot and replication, adds a logical unit number of the block storage device used by the application database in the consistency group, and a separate file system volume for storing related files of the application, Determining a snapshot policy of the consistency group, and simultaneously taking snapshots of the block storage device and the file system according to the snapshot policy; and/or determining a replication policy of the consistency group, and remotely controlling the block storage device and the file system according to the replication policy Data replication; because the scheme establishes a consistency group for application snapshots and replication, not only can the consistency group be used for simultaneous snapshot and replication of block storage devices and file systems, but also block storage. Unified management and data protection of data on devices and file systems. DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. Obviously, the drawings in the following description are only some embodiments of the present invention. Other figures may also be obtained from those skilled in the art from these figures.

1 is a flowchart of a data processing method in a storage system according to an embodiment of the present invention; FIG. 2 is a schematic diagram of a data processing method in a storage system according to an embodiment of the present invention; FIG. 2 is a flowchart of a snapshot in a data processing method in a storage system according to an embodiment of the present invention; FIG. 3 is a data processing in a storage system according to an embodiment of the present invention; FIG. 3 is a flowchart of a data processing method in a storage system provided by an embodiment of the present invention; FIG. 4 is a schematic structural diagram of a data processing apparatus in a storage system preparation according to an embodiment of the present invention; FIG. 5 is a schematic structural diagram of a storage management server in a storage system according to an embodiment of the present invention. detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Embodiments of the present invention provide a data processing method, apparatus, and system in a storage system. The details are described below separately. Embodiment 1

The embodiment of the present invention will be described from the perspective of a data processing device in a storage system. The data processing device in the storage system may be integrated into a device such as a storage management server (Storage Management Server).

A data processing method in a storage system, comprising: establishing a consistency group for application snapshot and replication; adding a logical unit number LUN of a block storage device used by the application database in the consistency group, and storing the application a separate file system volume of the related file; determining a snapshot policy of the consistency group, and simultaneously taking snapshots of the block storage device and the file system according to the snapshot policy; and/or determining a replication policy of the consistency group, and according to the replication policy Block storage devices and file systems for remote data replication.

As shown in Figure 1, the specific process can be as follows:

101. Establish a consistency group for application snapshots and replication.

The consistency group refers to a collection of data volumes in which multiple data volumes can be saved; since the consistency group established in step 101 is for the snapshot and copy of the application, therefore, The consistency group is primarily used to store data and/or files related to snapshots and replication of the application.

102. In step 101, a logical unit number of a block storage device used for adding a database of the application in the consistency group, and a separate file system volume for storing related files of the application are established; for example, the specifics may be as follows:

A separate file system volume of the relevant file of the application is obtained from the file system, and the independent file system volume is stored in the consistency group.

The file system may be specifically a network shared file system (NFS, Network File System) or a common Internet file system (CIFS, Common Internet File Systems), and the file system may be integrated into a network attached storage (NAS). In the device, it will not be described here.

After the consistency group is established, steps 103 and/or 104 may be performed as follows: The snapshot policy of the consistency group is determined, and the block storage device and the file system (such as the network shared file system) are simultaneously snapshotd according to the snapshot policy; for example, the details may be as follows:

(1) Simultaneously send a snapshot command to the block storage device and the file system according to the snapshot policy.

For example, taking the data processing device in the storage system as a main controller and a secondary controller as an example, the details may be as follows:

The master controller simultaneously sends snapshot commands to the block storage device and the file system according to the determined snapshot policy.

In this way, after receiving the snapshot command, the block storage device and the file system can take snapshots of the data or files saved by themselves according to the snapshot command, thereby generating a snapshot. Among them, the block storage device and the file system can perform snapshots in various ways. For example, for a block storage device, COW (Copy on Write), also called COFW (Copy on Fist Write) or redirected write can be used. Snapshot algorithms such as ROW, Redirect on Write, also called ROFW, Redirect on Fist Write. For the file system, a snapshot algorithm such as Qtree (Q-tree) or binary tree can be used, and will not be described here.

The snapshot policy can be set according to the requirements of the actual application, and is not described here.

(2) Block the host to write data to disk and create a protection point in time.

For example, if the data processing device in the storage system includes the primary controller and the secondary controller as an example, the details may be as follows:

Sending, by the primary controller, a blocking command for instructing the blocking host to write data to the secondary controller, so that the secondary controller blocks the data sent by the host in the cache according to the blocking command;

The primary controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command to create a protection time point.

Among them, the secondary controller can prevent the host from writing data in various ways. For example, you can indicate whether to prevent the host from writing data by setting a blocking flag. For example, if the blocking identifier exists, it indicates that the host needs to prevent the host from writing data to the disk; otherwise, if the blocking identifier does not exist, it indicates that the host does not need to prevent the host from writing data to the disk, and the host can directly write data to the disk. ,and many more. Wherein, when it is necessary to prevent the host from writing data to the disk, the received data may be first stored in the cache, that is, the step "the secondary controller blocks the data sent by the host according to the blocking command in the cache" may include: Setting a blocking identifier according to the blocking command, when the secondary controller receives the sending by the host In the case of data, if it is determined that the blocking identifier exists, the data is blocked in the cache.

Conversely, if it is determined that the blocking identifier does not exist, the data is written to the disk.

Since the host is prevented from writing new data to the disk at this time, the consistency of the data on the block storage device and the file system corresponding to the protection time point can be ensured. Therefore, the snapshots made by the block storage device and the file system will also be consistent.

(3) After the protection time point, the blocking of the data written to the host is released.

For example, if the blocking flag is set in (2) to indicate that the host needs to be prevented from writing data, then the blocking identifier can be deleted by the secondary controller. In this way, the data can be written to the disk later, for example, the data that was previously stored in the cache can be written to the disk, and so on.

104. Determine a replication policy of the consistency group, and perform remote data replication on the block storage device and the file system (such as a network shared file system) according to the replication policy; for example, the following may be specifically: (1) blocking the host according to the replication policy Write data to disk and create a protection point in time. For example, if the data processing device in the storage system includes the primary controller and the secondary controller as an example, the details may be as follows:

The primary controller sends a blocking command for instructing the blocking host to write data to the secondary controller according to the replication policy, so that the secondary controller blocks the data sent by the host in the cache according to the blocking command; The command at the time point is given to the secondary controller, so that the secondary controller creates a protection time point according to the command to create a protection time point.

The secondary controller can prevent the host from writing data in various ways. For example, the blocking identifier can be set to indicate whether to prevent the host from writing data. For example, if the blocking identifier exists, it indicates that the host needs to prevent the host from writing data to the disk; otherwise, if the blocking identifier does not exist, it indicates that the host does not need to prevent the host from writing data to the disk, and the host can directly write data to the disk. ,and many more. Wherein, when it is necessary to prevent the host from writing data to the disk, the received data may be first stored in the cache, that is, the step "the secondary controller blocks the data sent by the host according to the blocking command in the cache" may include: The device sets the blocking identifier according to the blocking command. When the secondary controller receives the data sent by the host, if it determines that the blocking identifier exists, the data is blocked in the cache.

Since the host is prevented from writing new data to the disk at this time, the consistency of the data on the block storage device and the file system corresponding to the protection time point can be ensured. (2) After the protection time point, the blocking of the data written to the host is released.

For example, if (1) is mainly to set the blocking identifier to indicate that the host needs to be prevented from writing data, then at this time, the blocking identifier can be deleted by the secondary controller. In this way, the data can be written to the disk later, for example, the data that was previously stored in the cache can be written to the disk, and so on.

(3) Remote data replication is performed according to the protection time point.

As can be seen from the above, the embodiment establishes a consistency group for application snapshot and replication, adds a logical unit number of the block storage device used by the application database in the consistency group, and a separate file for storing related files of the application. a system volume, which determines a snapshot policy of the consistency group, and simultaneously snapshots the block storage device and the file system (such as a network shared file) system according to the snapshot policy; and/or determines a replication policy of the consistency group, and according to the replication The policy performs remote data replication on block storage devices and file systems, such as network shared file systems. Since the solution establishes a consistency group for application snapshot and replication, not only can the consistency group be used to realize the simultaneous snapshot and copy of the block storage device and the file system (such as a network shared file system), but also Unified management and data protection for data on block storage devices and file systems such as network shared file systems. According to the method described in the first embodiment, further details will be exemplified in the second and third embodiments. Embodiment 2

In this embodiment, a snapshot is taken as an example for further detailed description. In addition, for the convenience of description, in the embodiment, the data processing device in the storage system is specifically integrated into the storage management server, and the file system is specifically a network shared file system and is installed on a network attached storage device (NAS, Network Attached). Storage is described as an example.

Referring to FIG. 2a, the figure is a schematic diagram of a scenario of a data processing method in the storage system. As can be seen from FIG. 2a, the storage system may include a host, a block storage device, a network attached storage device, and a storage management server. The host, the block storage device, and the network attached storage device can communicate through a storage area network (SAN). In addition, the host can also connect the block storage device, the network attached storage device, and the storage management server through a local area network (LAN). Block storage devices and network attached storage devices are respectively composed of multiple storage orders. Yuan composition, no longer repeat here. The functions of each device of the storage system may be specifically as follows:

(1) host;

Mainly used to write or read data to block storage devices and network attached storage devices. For example, storing the application's database in a block storage device, storing the application's associated files on a network attached storage device, and so on.

(2) block storage device;

A database for storing applications, receiving data written by the host, and transmitting data that the host needs to read to the host, and so on. In addition, the logical unit number of the block storage device used by the application database may be added to the consistency group of the storage management server, and when the snapshot command issued by the storage management server is received, the stored application according to the snapshot command The database is snapshotted to generate a snapshot of the required data.

(3) Network attached storage devices;

Used to store related files of the application, receive data written by the host, and send data that the host needs to read to the host, and so on. In addition, the independent file system volume of the related file of the application may be added to the consistency group of the storage management server, and when the snapshot command sent by the storage management server is received, the related file of the stored application is performed according to the snapshot command. Snapshots to generate a snapshot of the required data.

(4) a storage management server;

The storage management server includes storage management software, which can be used to establish a consistency group for application snapshots and replications, receive a logical unit number used by a database of applications sent by the block storage device, and receive related files of applications sent by the network attached storage device. a separate file system volume in which the logical unit number used by the application database and the independent file system volume storing the relevant files of the application are added to the consistency group, and the snapshot policy of the consistency group is determined, and according to the snapshot policy pair The block storage device and the network attached storage device (that is, the file system, such as the network shared file system) perform snapshots at the same time, for example, the snapshot command can be issued to the block storage device and the network attached storage device at the same time, and the like.

Based on the structure of the above storage system, the specific execution flow of the storage system will be described below.

As shown in FIG. 2b, a data processing method in a storage system may be as follows: 201. The storage management server establishes a consistency group for application snapshots and replication.

The consistency group refers to a collection of data volumes, in which multiple data volumes can be saved; since the consistency group established in step 201 is for the snapshot and copy of the application, therefore, The consistency group is primarily used to store data and/or files related to snapshots and replication of the application.

202. The block storage device sends the logical unit number of the block storage device used by the application database to the storage management server, and then performs step 203.

203. The storage management server adds the logical unit number of the block storage device used by the database of the received application to the consistency group.

204. The network attached storage device sends a separate file system curl of the related file of the application to the storage management server, and then performs step 205.

205. The storage management server adds the independent file system volume of the related file of the received application to the consistency group.

The execution of steps 202 and 204 may be in no particular order.

206. The storage management server determines a snapshot policy of the consistency group, and simultaneously snapshots the block storage device and the file system according to the snapshot policy. For example, a snapshot command is issued to a block storage device and a network attached storage device. For example, see Figure 2c for details. The process can be as follows:

2061. The primary controller simultaneously sends a snapshot command to the block storage device and the network attached storage device according to the determined snapshot policy.

In this way, after receiving the snapshot command, the block storage device and the network attached storage device can take snapshots of the data or files saved by themselves according to the snapshot command, thereby generating a snapshot.

The algorithm for performing snapshots on the block storage device and the network attached storage device may be various. For example, for a block storage device, a snapshot algorithm such as COW or ROW may be used, and for a network attached storage device (and a file system), Snapshot algorithms such as Qtree or binary tree are not described here.

2062. The primary controller sends a blocking command to the secondary controller to instruct the blocking host to write data.

2063. The secondary controller sets a blocking identifier according to the blocking command. For example, the input/output (I/O, Input/Output) may be set to be marked, and a response message indicating completion of the blocking command is returned to the primary controller. 2064. The host continues to write data to the secondary controller, such as writing I/O data.

2065. The secondary controller determines whether there is a blocking identifier. If yes, the data written by the host is blocked. If it does not exist, the data does not operate, that is, the host is allowed to continue to write data to the secondary controller.

For example, you can check whether there is an I/O under-brush flag. If it exists, the I/O data written by the host is blocked in the cache and not written to the disk. If it does not exist, it does not work, that is, the host is allowed to continue to the secondary. The controller writes I/O data, writes the I/O data to disk, and so on.

2066. The main controller sends a command to create a protection time point to the secondary controller.

2067. The secondary controller creates a protection time point according to the command for creating a protection time point.

2068. The secondary controller deletes the blocking identifier. For example, the previously set I/O lowering brush flag may be canceled, and a response message indicating completion of the command to create the protection time point is returned to the primary controller.

2069. The secondary controller writes the I/O data in the cache to the disk, and returns a response message indicating that the host writes successfully to the host.

As can be seen from the above, the embodiment establishes a consistency group for application snapshot and replication, adds a logical unit number of the block storage device used by the application database in the consistency group, and a separate file for storing related files of the application. The system volume determines the snapshot policy of the consistency group and simultaneously snapshots the block storage device and the network attached storage device (and file system) according to the snapshot policy. Since the scheme establishes a consistency group for application snapshot and replication, the consistency group can be used not only to realize the simultaneous snapshot operation of the block storage device and the network attached storage device, but also to block storage devices and networks. Data on the attached storage device for unified management and data protection. Embodiment 3

In the embodiment of the present invention, the storage system may include a host, a block storage device, a network attached storage device, and a storage management server as an example. The second embodiment is different from the second embodiment. In the middle, the remote data replication will be taken as an example.

The structure of the storage system is the same as that of the storage system in the second embodiment, and details are not described herein again. Each device in the storage system can perform the following functions:

(1) host;

Mainly used to write or read data to block storage devices and network attached storage devices, for example, storing the application database in a block storage device, and storing related files of the application to the network attached Storage devices, and more.

(2) block storage device;

A database for storing applications, receiving data written by the host, and transmitting data that the host needs to read to the host, and so on. And when receiving the copy command issued by the storage management server, remote data copying is performed on the stored application database according to the copy command.

(3) Network attached storage devices;

Used to store related files of the application, receive data written by the host, and send data that the host needs to read to the host, and so on. And when receiving the copy command issued by the storage management server, the root (4) stores the management server;

The storage management server is mainly used to establish a consistency group for application snapshot and replication, receive a logical unit number of a block storage device used by a database of an application sent by the block storage device, and receive related files of an application sent by the network attached storage device. A separate file system volume label, in which the logical unit number of the block storage device used by the application database and the independent file system volume volume label of the relevant file storing the application are added to the consistency group, and the replication policy of the consistency group is determined. . According to the replication policy, the remote storage data is simultaneously performed on the block storage device and the network attached storage device (that is, the file system, such as the network shared file system). For example, the copy command can be issued to the block storage device and the network attached storage device at the same time, etc. Wait.

As shown in FIG. 3a, a data processing method in a storage system may be as follows: 301. A storage management server establishes a consistency group for application snapshot and replication.

The consistency group refers to a collection of data volumes in which multiple data volumes can be saved; since the consistency group established in step 301 is for the snapshot and replication of the application, therefore, The consistency group is primarily used to store data and/or files related to snapshots and replication of the application.

302. The block storage device sends the logical unit number of the block storage device used by the application database to the storage management server, and then performs step 303.

303. The logic of the block storage device used by the storage management server to receive the database of the application The unit number is added to the consistency group.

304. The network attached storage device sends a separate file system curl of the related file of the application to the storage management server, and then performs step 305.

305. The storage management server adds the independent file system volume of the related file of the received application to the consistency group.

The execution of steps 302 and 304 may be in no particular order.

306. The storage management server determines a replication policy of the consistency group, and performs remote data replication on the block storage device and the network attached storage device (that is, a file system, such as a network shared file system) according to the replication policy. For example, refer to FIG. 3b. , the process can be as follows:

3061. The primary controller of the storage device sends a blocking command for instructing the blocking host to write data to the secondary controller according to the replication policy.

The replication policy can be set according to the requirements of the actual application, and is not described here.

3062. The secondary controller sets a blocking identifier according to the blocking command. For example, the I/O lowering flag may be set, and a response message indicating completion of the blocking command is returned to the primary controller.

3063. The host continues to write data to the secondary controller, such as writing I/O data.

3064. The secondary controller determines whether there is a blocking identifier. If yes, the data written by the host is blocked. If it does not exist, the data does not operate, that is, the host is allowed to continue to write data to the secondary controller.

3065. The main controller sends a command to create a protection time point to the secondary controller.

3066. The secondary controller creates a protection time point according to the command for creating a protection time point.

3067. The secondary controller deletes the blocking identifier. For example, the previously set I/O downstamp flag may be canceled, and a response message indicating completion of the command to create the protection time point is returned to the primary controller.

3068. The secondary controller writes the I/O data in the cache to the disk, and returns a response message indicating that the host writes successfully to the host.

3069. The secondary controller performs remote data replication according to the created protection time point.

As can be seen from the above, this embodiment establishes a consistency group for snapshots and replications of applications, Adding a logical unit number of the block storage device used by the database of the application to the independent file system volume of the relevant file storing the application, determining a replication policy of the consistency group, and according to the replication policy, the block storage device and Network attached storage devices (ie file systems, such as network shared file systems) for remote data replication. Since the scheme establishes a consistency group for application snapshot and replication, the consistency group can be used not only to achieve the purpose of copying the block storage device and the network attached storage device at the same time, but also to block storage devices and networks. Data on the attached storage device for unified management and data protection. Embodiment 4

In order to better implement the above method, the embodiment of the present invention further provides a data processing apparatus in a storage system. As shown in FIG. 4, the data processing apparatus in the storage system includes an establishing unit 401, an adding unit 402, and a snapshot unit 403. And copy unit 404;

The establishing unit 401 is configured to establish a consistency group for application snapshot and replication;

The consistency group refers to a collection of data volumes in which multiple data volumes can be saved; since the consistency group established by the establishing unit 401 is for the snapshot and replication of the application, therefore, The consistency group is primarily used to store data and/or files related to snapshots and replication of the application.

The adding unit 402 is configured to add, in the consistency group established by the establishing unit 401, a logical unit number of the block storage device used by the application database, and a separate file system volume storing related files of the application;

a snapshot unit 403, configured to determine a snapshot policy of the consistency group, and simultaneously snapshot the block storage device and the file system according to the snapshot policy; and/or;

The copy unit 404 is configured to determine a replication policy of the consistency group, and perform remote data replication on the block storage device and the file system according to the replication policy.

The adding unit 402 is specifically configured to obtain a logical unit number of the block storage device used by the database of the application from the block storage device, add the logical unit number to the consistency group, and obtain the file system from the file system. A separate file system volume of the application's associated file, the independent file system volume is placed in the consistency group.

The file system may be a network shared file system or a universal Internet file system. The file system may be integrated into a device such as a network attached storage device, and details are not described herein. The snapshot unit 403 may include a snapshot subunit, a first blocking subunit, and a first disarming subunit, as follows:

a snapshot subunit, configured to send a snapshot command to the block storage device and the file system according to the snapshot policy;

For example, the snapshot subunit may be specifically configured to use the primary controller to simultaneously send a snapshot command to the block storage device and the file system according to the snapshot policy.

In this way, after receiving the snapshot command, the block storage device and the file system can take snapshots of the data or files saved by themselves according to the snapshot command, thereby generating a snapshot. The algorithm for snapshotting the block storage device and the file system may be various. For example, for the block storage device, a snapshot algorithm such as COW (ie, COFW) or ROW (ie, ROFW) may be used, and for the file system, Qtree may be adopted. Snapshot algorithms such as binary trees are not described here.

a first blocking sub-unit, configured to block the host to write data to the disk, and create a protection time point; for example, the first blocking sub-unit may be specifically configured to send, by the primary controller, the secondary controller to indicate that the blocking host writes The data blocking command causes the secondary controller to block the data sent by the host in the cache according to the blocking command, and the primary controller sends a command to create a protection time point to the secondary controller, so that the secondary controller creates the protection time point according to the The command creates a protection time point.

The secondary controller can prevent the host from writing data in multiple ways. For example, the blocking identifier can be set to indicate whether to prevent the host from writing data. For example, if the blocking identifier exists, it indicates that the host needs to prevent the host from writing data to the disk. On the contrary, if the blocking identifier does not exist, it means that the host does not need to prevent the host from writing data to the disk. At this time, the host can directly write data to the disk, and the like, wherein when the host needs to prevent the host from writing data to the disk, The data received from the received data is stored in the cache. For details, refer to the previous embodiment, and details are not described herein.

The first release subunit is configured to release the blocking of the data written to the host after the protection time point. For example, if the blocking identifier is used to indicate that the host needs to be prevented from writing data, then the blocking identifier can be deleted by the first releasing subunit, so that the data can be subsequently written to the disk, for example. , you can write data that was previously cached to disk, and so on.

The copy unit 404 may include a copy subunit, a second blocker unit, and a second release unit; a second blocking subunit, configured to prevent the host from writing data to the disk according to the replication policy, and creating a protection time point;

For example, the second blocking sub-unit may be configured to send, by using the primary controller, a blocking command for instructing the blocking host to write data to the secondary controller according to the replication policy, so that the secondary controller sends the host according to the blocking command. The data is blocked in the cache; the primary controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command to create a protection time point.

The second release subunit is configured to release the blocking of the data written to the host after the protection time point.

For example, if the blocking identifier is used to indicate that the host needs to be prevented from writing data, then the second disarming unit can delete the blocking identifier, so that the data can be subsequently written to the disk, for example, You can write data that was previously cached to disk, and so on.

A replication subunit for remote data replication based on the protection time point.

The data processing device in the storage system may be specifically integrated in a device such as a storage management server. For the specific implementation, the foregoing units may be implemented as an independent entity, or may be implemented in any combination, and may be implemented as the same entity or a plurality of entities. For the specific implementation of the foregoing, refer to the foregoing embodiments, and details are not described herein.

It can be seen that the establishing unit 401 in the data processing apparatus in the storage system of the present embodiment can establish a consistency group for application snapshot and copy, and the adding unit 402 adds the block used by the application database in the consistency group. a logical unit number of the storage device, and a separate file system volume storing related files of the application, and then the snapshot unit 403 determines a snapshot policy of the consistency group, and simultaneously snapshots the block storage device and the file system according to the snapshot policy, and / or, the replication unit 404 determines the replication policy of the consistency group, and performs remote data replication on the block storage device and the file system according to the replication policy; since the scheme establishes a consistency group for application snapshot and replication, Can The consistency group is used to realize the simultaneous snapshot and copy of the block storage device and the file system, and the data on the block storage device and the file system can be uniformly managed and protected. Embodiment 5

Correspondingly, the embodiment of the present invention further provides a communication system, including the data processing device in any one of the storage systems provided by the embodiment of the present invention, where the data processing device in the storage system is specifically referred to the fourth embodiment, for example. For example, the data processing device in the storage system is specifically integrated in the storage management server, and the specifics may be as follows:

a storage management server for establishing a consistency group for application snapshots and replications; adding a logical unit number of the block storage device used by the application's database to the consistency group, and a separate file system volume storing related files of the application Determining a snapshot policy of the consistency group, and simultaneously taking snapshots of the block storage device and the file system according to the snapshot policy; and/or determining a replication policy of the consistency group, and performing block storage devices and file systems according to the replication policy For the remote data replication, refer to the previous embodiment, and details are not described herein.

In addition, the communication system may also include other devices, such as a host, a block storage device (such as a block storage device, etc.), and a file system (which may be integrated in a network attached storage device), etc., as follows: The host is mainly used for the forward block. The storage device and the file system write or read data, for example, store the application database in a block storage device, and store related files of the application to the file system, and the like.

A block storage device is mainly used to store an application database, receive data written by the host, and send data that the host needs to read to the host, and the like. In addition, when the snapshot command issued by the storage management server is received, the database of the stored application is snapshot according to the snapshot command; and/or when the copy command issued by the storage management server is received, the storage is performed according to the copy command. The applied database is used for remote data replication.

The file system is mainly used to store related files of the application, receive data written by the host, and send data that the host needs to read to the host, and the like. And receiving a snapshot command sent by the storage management server, taking a snapshot of the related file of the stored application according to the snapshot command, thereby generating a snapshot of the required data; and/or receiving the copy command issued by the storage management server Remote data replication of the stored application's database based on the copy command.

For the specific implementation of the foregoing various devices, refer to the foregoing embodiments, and details are not described herein again. Since the communication system includes the data processing device in any one of the storage systems provided by the embodiments of the present invention, the beneficial effects achieved by the data processing device in any of the storage systems provided by the embodiments of the present invention can be achieved. This will not be repeated here. Embodiment 6

Correspondingly, the embodiment of the present invention further provides a storage management server, including a processor 501, a transceiver device 502 for transmitting and receiving data, and a memory 503 for storing data;

a processor 501, configured to establish a consistency group for application snapshot and replication, add a logical unit number of the block storage device used by the application database in the consistency group, and a separate file system volume that stores related files of the application; Determining a snapshot policy of the consistency group, and simultaneously taking snapshots of the block storage device and the file system according to the snapshot policy; and/or determining a replication policy of the consistency group, and remotely controlling the block storage device and the file system according to the replication policy Data replication.

The file system may be a network shared file system or a CIFS. The file system may be integrated into a device such as a network attached storage device, and details are not described herein.

The processor 501 is specifically configured to obtain, from the block storage device, a logical unit number of the block storage device used by the database of the application, add the logical unit number to the consistency group, and, from the file system Obtain a separate file system volume of the relevant file of the application, and store the independent file system volume in the consistency group.

The processor 501 is specifically configured to send a snapshot command to the block storage device and the file system according to the snapshot policy; block the host to write data to the disk, and create a protection time point; after the protection time point, release Blocking of data written to the host.

The processor 501 is specifically configured to: prevent the host from writing data to the disk according to the replication policy, and create a protection time point; after the protection time point, unblock the data written to the host; according to the protection time point Perform remote data replication.

The blocking identifier can be used to indicate whether to prevent the host from writing data. For example, if the blocking identifier is present, it indicates that the host needs to prevent the host from writing data to the disk. Otherwise, if the blocking identifier does not exist, it indicates that the host does not need to be blocked. Write data to disk. At this time, the host can directly write data to the disk, and so on. When the host needs to prevent the host from writing data to the disk, the received data can be stored in the cache first. For details, see the front. The description in the embodiments is not described herein again. For the specific implementation of the foregoing various devices, refer to the foregoing embodiments, and details are not described herein again. As can be seen from the above, the storage management server of this embodiment can establish a logical unit number of the block storage device used by the database of the application, and a storage unit for storing the application by establishing a consistency group for application snapshot and replication. A separate file system volume of the related file, determines a snapshot policy of the consistency group, and simultaneously snapshots the block storage device and the file system according to the snapshot policy, and/or determines a replication policy of the consistency group, and according to the replication policy Block storage device and file system for remote data replication; because the scheme establishes a consistency group for application snapshot and replication, not only can the consistency group be used to achieve simultaneous snapshot and replication of block storage devices and file systems. Moreover, unified management and data protection can be performed on the data on the block storage device and the file system. A person skilled in the art may understand that all or part of the various steps of the foregoing embodiments may be completed by a program instructing related hardware. The program may be stored in a computer readable storage medium, and the storage medium may include: Read only memory (ROM, Read Only Memory), random access memory (RAM), disk or optical disk.

The foregoing description of the data processing method, apparatus, and system in a storage system provided by the embodiments of the present invention is merely for facilitating understanding of the method and core idea of the present invention. Meanwhile, those skilled in the art The present invention is not limited by the scope of the present invention.

Claims

Rights request

1. A data processing method in a storage system, characterized by including:

Establish consistency groups for application snapshots and replication;

Add to the consistency group the logical unit number of the block storage device used by the database of the application and the independent file system volume that stores the relevant files of the application;

Determine the snapshot policy of the consistency group, and snapshot the block storage device and file system simultaneously according to the snapshot policy; and/or,

Determine the replication policy for the consistency group and perform remote data replication on the block storage device and file system according to the replication policy.

2. The method according to claim 1, characterized in that, adding in the consistency group the logical unit number of the block storage device used by the database of the application and the independent file storing the related files of the application System volumes, including:

Obtain the logical unit number of the block storage device used by the database of the application from the block storage device, and add the logical unit number to the consistency group; and,

Obtain the independent file system volume of the application-related files from the file system, and store the independent file system volume in the consistency group.

3. The method according to claim 2, characterized in that: snapshotting the block storage device and the file system simultaneously according to the snapshot policy includes:

Send snapshot commands to the block storage device and file system simultaneously according to the snapshot policy; Block the host from writing data to the disk and create a protection time point;

After the protection time point, the blocking of writing data by the host is released.

4. The method according to claim 3, characterized in that, sending a snapshot command to the block storage device and the file system simultaneously according to the snapshot policy includes:

The main controller simultaneously sends a snapshot command to the block storage device and the file system according to the snapshot policy;

The blocking host writes data to the disk and creates a protection time point, including: sending a blocking command from the main controller to the secondary controller to instruct the blocking host to write data, so that the secondary controller switches the host to the disk according to the blocking command. The sent data is blocked in the cache, and the main controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time according to the command to create a protection time point. point.

5. The method according to claim 2, characterized in that: determining the replication strategy of the consistency group and performing remote data replication on the block storage device and the file system according to the replication strategy includes: according to the replication strategy Prevent the host from writing data to the disk and create a protection time point; after the protection time point, unblock the host from writing data;

Remote data replication is performed based on the protection time point.

6. The method of claim 5, wherein preventing the host from writing data to disk according to the replication policy and creating a protection time point includes:

The primary controller sends a blocking command to the secondary controller according to the replication policy to instruct the blocking host to write data, so that the secondary controller blocks the data sent by the host in the cache according to the blocking command;

The main controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command to create a protection time point.

7. The method according to claim 4 or 6, characterized in that the secondary controller blocks the data sent by the host in the cache according to the blocking command, including:

The secondary controller sets the blocking flag according to the blocking command;

When the secondary controller receives the data sent by the host, if it is determined that the blocking identifier exists, the data will be blocked in the cache;

After the protection time point, unblocking the host from writing data includes: deleting the blocking identifier by the secondary controller.

8. A data processing device in a storage system, characterized by including:

Creation unit, used to establish consistency groups for application snapshots and replication;

Adding a unit for adding the logical unit number of the block storage device used by the database of the application and the independent file system volume storing the relevant files of the application in the consistency group;

Snapshot unit, used to determine the snapshot policy of the consistency group, and take snapshots of the block storage device and the file system simultaneously according to the snapshot policy; and/or;

The replication unit is used to determine the replication strategy of the consistency group, and perform remote data replication on the block storage device and file system according to the replication strategy.

9. The data processing device in the storage system according to claim 8, characterized in that, The adding unit is specifically configured to obtain the logical unit number of the block storage device used by the database of the application from the block storage device, and add the logical unit number to the consistency group; and, from the file system Obtain the independent file system volume of the relevant files of the application, and store the independent file system volume in the consistency group.

10. The data processing device in the storage system according to claim 9, wherein the snapshot unit includes a snapshot subunit, a first blocking subunit and a first releasing subunit;

The snapshot subunit is configured to simultaneously send snapshot commands to the block storage device and the file system according to the snapshot policy;

The first blocking subunit is used to block the host from writing data to the disk and create a protection time point; the first unblocking subunit is used to unblock the host from writing data after the protection time point. .

11. The data processing device in the storage system according to claim 10, characterized in that the snapshot subunit is specifically configured to use the main controller to simultaneously send data to the block storage device and the file system according to the snapshot policy. snapshot command;

The first blocking subunit is specifically configured to use the main controller to send a blocking command to the secondary controller to instruct the blocking host to write data, so that the secondary controller blocks the data sent by the host in the cache according to the blocking command, The main controller issues a command to create a protection time point to the secondary controller, so that the secondary controller creates a protection time point according to the command to create a protection time point.

12. The data processing device in the storage system according to claim 9, wherein the copy unit includes a copy sub-unit, a second blocking sub-unit and a second releasing sub-unit;

The second blocking subunit is used to prevent the host from writing data to the disk according to the replication policy and create a protection time point;

The second release subunit is used to release the blocking of the host writing data after the protection time point;

The replication subunit is used to perform remote data replication according to the protection time point.

13. The data processing device in the storage system according to claim 12, wherein the second blocking subunit is specifically configured to use the main controller to send a blocking instruction to the secondary controller according to the replication strategy. The blocking command for the host to write data causes the secondary controller to block the data sent by the host in the cache according to the blocking command; the primary controller issues a command to create a protection time point to the secondary controller. The controller causes the secondary controller to create a protection time point according to the command for creating a protection time point.

14. A communication system, characterized by comprising the data processing device in the storage system according to any one of claims 8 to 13.

15. A storage management server, characterized in that it includes a processor, a transceiver device for sending and receiving data, and a memory for storing data;

The processor is configured to establish a consistency group for application snapshots and replication; add to the consistency group the logical unit number of the block storage device used by the database of the application and the location of the relevant files that store the application. Independent file system volume; determine the snapshot policy of the consistency group, and snapshot the block storage device and the file system simultaneously according to the snapshot policy; and/or, determine the replication policy of the consistency group, and snapshot the block storage device according to the replication policy Storage devices and file systems perform remote data replication.

16. The storage management server according to claim 15, characterized in that,

The processor is specifically configured to obtain the logical unit number of the block storage device used by the database of the application from the block storage device, and add the logical unit number to the consistency group; and, from the file system Obtain the independent file system volume of the relevant files of the application, and store the independent file system volume in the consistency group.

17. The storage management server according to claim 16, characterized in that,

The processor is specifically configured to send snapshot commands to the block storage device and the file system simultaneously according to the snapshot policy; block the host from writing data to the disk, and create a protection time point; after the protection time point, release Blocking the host from writing data.

18. The storage management server according to claim 16, characterized in that,

The processor is specifically configured to prevent the host from writing data to the disk according to the replication policy, and create a protection time point; after the protection time point, unblock the host from writing data; according to the protection time point Perform remote data replication.