CN107203331B - Method and device for writing data - Google Patents

Abstract

The invention discloses a device for writing data, which comprises: the first acquisition module is used for acquiring the size information of the write data contained in a write operation command when the write operation command to a source volume is received before the snapshot generating command is not acquired in the process of redirecting the snapshot during writing; and the write operation module is used for applying a storage space matched with the size information from a preset storage area according to the size information of the write data and performing write operation in the applied storage space. The invention also discloses a method for writing data. The invention realizes the purpose of efficiently writing data without reading the data of the original storage block in the process of redirecting the snapshot during writing.

Description

Method and device for writing data

Technical Field

The present invention relates to the field of data storage technologies, and in particular, to a method and an apparatus for writing data.

Background

With the development of information technology, digital resources are more and more, and the security and backup of data become more and more important. At present, data backup is usually realized by using a snapshot technology, and according to the definition of the Storage Network Industry Association (SNIA), a snapshot is a fully available copy related to a set data set and contains an image of corresponding data at the time of copy initialization. The existing snapshot mainstream technology is divided into two types, one is Copy-On-Write (COFW), and the other is Redirect-On-Write (ROW).

In the scenario of distributed storage, the storage of data is performed with blocks (chunks) as the minimum units, however, when write operation is performed in the snapshot technology of redirection during write, a new Chunk space with a Chunk size is usually allocated to perform write operation, when write data in the write operation is smaller than the Chunk size, data of an old Chunk in a source volume is read first, then data is merged, and then the new Chunk space is written.

Disclosure of Invention

The invention mainly aims to provide a method and a device for writing data, and aims to improve the performance of write operation in the process of redirecting snapshots during writing.

In order to achieve the above object, the present invention provides an apparatus for writing data, comprising:

the first acquisition module is used for acquiring the size information of the write data contained in a write operation command when the write operation command to a source volume is received before the snapshot generating command is not acquired in the process of redirecting the snapshot during writing;

and the write operation module is used for applying a storage space matched with the size information from a preset storage area according to the size information of the write data and performing write operation in the applied storage space.

Preferably, the write operation command includes the write data and a first target write address where the write data is to be written to the source volume;

the write operation module includes:

a storage space application unit, configured to apply for a first storage space from the storage area according to size information of the write data, where the size of the first storage space is equal to a sum of the size of the write data and a data size of the first target write address;

a storage unit, configured to store the write data and the first target write address in the first storage space;

and the updating unit is used for updating the corresponding relation between the first target write address and the first storage address of the first storage space into a mapping table, and recording the updating time of the corresponding relation in the mapping table, wherein the mapping table is used for storing the corresponding relation between the address in the source volume and the storage address of the storage area and the updating time of the corresponding relation.

Preferably, the apparatus for writing data further comprises:

the second obtaining module is used for obtaining a target read address in a read operation command when the read operation command for the source volume is received before a snapshot generating command is not obtained in the process of writing and redirecting the snapshot;

a first determining module, configured to determine, if the mapping table includes the target read address, whether update time corresponding to the target read address in the mapping table is after a snapshot is generated based on the target read address for the last time;

a data merging module, configured to, when the mapping table includes the target read address and update time of the target read address in the mapping table is after a snapshot is generated based on the target read address for the last time, search a second storage address corresponding to the target read address in the mapping table, obtain data in the second storage address in the storage area, merge the data in the second storage address and the obtained first data into second data, apply for a second storage space in the storage area to store the second data, and update a correspondence between the target read address and a storage address of the second storage space into the mapping table, where the first data is data in a storage block corresponding to the target read address on the source volume;

and the read operation module is used for reading the data in the storage block corresponding to the target read address on the source volume when the mapping table does not contain the target read address or the mapping table contains the target read address but the corresponding update time of the target read address in the mapping table is not after the snapshot is generated based on the target read address for the last time.

Preferably, the apparatus for writing data further comprises:

the receiving module is used for judging whether the source volume has write operation before receiving the snapshot generating command according to the mapping table if the snapshot generating command to the source volume is received in the process of redirecting the snapshot during writing;

a second judging module, configured to, when there is a write operation before the source volume receives the snapshot generating command, acquire an address of the write operation on the source volume as a third target write address, search, according to the mapping table, an address in the storage area corresponding to the third target write address, acquire data in the address as third data, and acquire a size of the third data at the same time;

and the snapshot generating module is used for generating a snapshot according to the third data, the size of the third data and the third target write address.

Preferably, the snapshot generating module includes:

a determining unit, configured to determine whether a size of the third data is equal to a size of a third storage block corresponding to the source volume at the third target write address;

a write operation unit, configured to write the third data back to a third storage block of the source volume when the size of the third data is equal to the size of the third storage block;

and the snapshot generating unit is configured to, when the size of the third data is not equal to the size of the third storage block, acquire data in the third storage block, merge the third data and the data in the third storage block into fourth data, and write the fourth data back to the third storage block of the source volume.

The invention further provides a method for writing data, which comprises the following steps:

in the process of redirecting snapshots during writing, before a snapshot generating command is not acquired, when a writing operation command for a source volume is received, acquiring the size information of written data contained in the writing operation command;

and according to the size information of the written data, applying a storage space matched with the size information from a preset storage area, and performing write operation in the applied storage space.

Preferably, the write operation command further includes the write data, and the first target write address where the write data is to be written to the source volume;

then, the applying for the storage space matched with the size information from a preset storage area according to the size information of the written data, and performing a write operation in the applied storage space includes:

applying for a first storage space from the storage area according to the size information of the written data, wherein the size of the first storage space is equal to the sum of the size of the written data and the data size of the first target write address;

storing the write data and the first target write address in the first storage space;

and updating the corresponding relation between the first target write address and the first storage address of the first storage space into a mapping table, and recording the updating time of the corresponding relation in the mapping table, wherein the mapping table is used for storing the corresponding relation between the address in the source volume and the storage address of the storage area and the updating time of the corresponding relation.

Preferably, the method for writing data further comprises:

in the process of writing and redirecting the snapshot, before a snapshot generating command is not obtained, when a read operation command of the source volume is received, a target read address in the read operation command is obtained;

if the mapping table contains the target reading address, judging whether the corresponding updating time of the target reading address in the mapping table is after the snapshot is generated based on the target reading address for the last time;

if the mapping table contains the target read address and the update time of the target read address in the mapping table is after a snapshot is generated based on the target read address for the last time, searching a second storage address corresponding to the target read address in the mapping table, acquiring data in the second storage address in the storage area, merging the data in the second storage address and the acquired first data into second data, applying for a second storage space in the storage area to store the second data, and updating the corresponding relation between the target read address and the storage address of the second storage space into the mapping table, wherein the first data is data in a storage block corresponding to the target read address on the source volume;

and if the mapping table does not contain the target read address or the mapping table contains the target read address but the corresponding update time of the target read address in the mapping table is not after the snapshot is generated based on the target read address for the last time, reading the data in the storage block corresponding to the target read address on the source volume.

Preferably, the method for writing data further comprises:

if a snapshot generating command for the source volume is received in the process of redirecting the snapshot during writing, judging whether the source volume has writing operation before receiving the snapshot generating command according to the mapping table;

when the source volume has write operation before receiving the snapshot generating command, acquiring an address of the write operation to the source volume as a third target write address, searching an address in the storage area corresponding to the third target write address according to the mapping table, acquiring data in the address as third data, and acquiring the size of the third data;

and generating a snapshot according to the third data, the size of the third data and the third target write address.

Preferably, the generating the snapshot according to the third data, the size of the third data, and the third target write address includes:

judging whether the size of the third data is equal to the size of a third storage block corresponding to the third target write address in the source volume;

if so, writing the third data back to a third storage block of the source volume;

if not, acquiring the data in the third storage block, merging the third data and the data in the third storage block into fourth data, and writing the fourth data back to the third storage block of the source volume.

According to the method and the device for writing data, in the process of redirecting the snapshot during writing, before a snapshot generating command is not obtained, when a writing operation command for a source volume is received, size information of the written data contained in the writing operation command is obtained; and according to the size information of the written data, applying a storage space matched with the size information from a preset storage area, and performing write operation in the applied storage space. The data space matched with the size information of the write-in data in the write-in operation is applied, the write-in data contained in the write-in operation command and the address to be written in the source volume are written in, the data of the original chunk do not need to be read, and the purpose of efficiently writing the data is achieved.

Drawings

FIG. 1 is a functional block diagram of a first embodiment of an apparatus for writing data according to the present invention;

FIG. 2 is a schematic diagram of a detailed functional block of the write module 20 in the embodiment of FIG. 1;

FIG. 3 is a functional block diagram of a third embodiment of an apparatus for writing data according to the present invention;

FIG. 4 is a functional block diagram of a fourth embodiment of an apparatus for writing data according to the present invention;

FIG. 5 is a functional block diagram of the snapshot generating module 90 in the embodiment shown in FIG. 4 according to the present invention;

FIG. 6 is a flowchart illustrating a first embodiment of a method for writing data according to the present invention;

FIG. 7 is a flowchart illustrating a detailed process of step S20 in the embodiment of FIG. 6 according to the present invention, in which the step S20 applies for a storage space from a preset storage area according to the size of the write data, and performs a write operation in the applied storage space;

FIG. 8 is a flowchart illustrating a data writing method according to a third embodiment of the present invention;

FIG. 9 is a flowchart illustrating a fourth embodiment of a method for writing data according to the present invention;

fig. 10 is a flowchart illustrating a detailed process of generating a snapshot according to the third data, the size of the third data, and the third target write address in step S100 in the embodiment shown in fig. 9.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The present invention provides an apparatus for writing data, and referring to fig. 1, in a first embodiment, the apparatus for writing data includes:

the first obtaining module 10 is configured to, in a process of redirecting a snapshot during writing, before a snapshot generating command is not obtained, obtain size information of write data included in a write operation command when the write operation command for a source volume is received;

and the write operation module 20 is configured to apply a storage space matched with the size information from a preset storage area according to the size information of the write data, and perform write operation in the applied storage space.

The device for writing data provided by the invention is mainly applied to the distributed storage technology, and is a device for writing data when a snapshot technology of redirection during writing is used. In a typical distributed storage scenario, data is stored in a block (Chunk) as a minimum unit, each storage system may divide a storage space into several chunks with the same size as required, and store the data in the chunks, for example, the storage space may be divided into several blocks, where the size of each block may be divided into 4 megabytes or 112 bytes.

In this embodiment, in the process of redirecting snapshots during writing, when an old snapshot has been generated but a new snapshot generation command has not been received, or when a snapshot generation command has not been received in a first snapshot, a write operation command to a source volume may be received. For example, the snapshot is taken at A, B, C three times in the process of redirecting the snapshot during writing, after the snapshot is completed at time a and before the snapshot is generated at time B, that is, before the snapshot generation command is received, a write operation command or a read operation command may be received, and the write operation or the read operation occurring in the process of redirecting the snapshot during writing is specifically a write redirection operation or a read redirection operation. The source volume in the first obtaining module 10 refers to a data storage space to be backed up in a snapshot process, and may snapshot a plurality of blocks as needed. The write operation is to write data to the source volume, and when a write operation command for the source volume is received, size information of the write data included in the write operation command is obtained, where the write data is data to be written to the source volume, and the data is usually stored in a memory in a byte form, so that obtaining the size information of the write data can be obtained by obtaining the number of bytes of the write data.

And after the size information of the written data is acquired, applying for a storage space in a preset storage area according to the size information of the written data instead of applying for the storage space according to the size information of the chunk, and then performing write operation in the newly applied storage space. The preset storage area is other storage space except the source volume in the storage space, and specifically, a new space can be selected for storage according to needs.

In this embodiment, in the process of redirecting a snapshot during writing, before a snapshot generating command is not obtained, when a write operation command to a source volume is received, size information of write data included in the write operation command is obtained; and according to the size information of the written data, applying a storage space matched with the size information from a preset storage area, and performing write operation in the applied storage space. The data space matched with the size information of the write-in data in the write-in operation is applied, the write-in data contained in the write-in operation command and the address to be written in the source volume are written in, the data of the original chunk do not need to be read, and the purpose of efficiently writing the data is achieved.

Further, based on the first embodiment of the apparatus for writing data according to the present invention, in the second embodiment of the apparatus for writing data according to the present invention, the write operation command includes the write data and a first target write address at which the write data is to be written to the source volume;

referring to fig. 2, the detailed functional blocks of the write operation module 20 include:

a storage space applying unit 21, configured to apply for a first storage space from the storage area according to size information of the write data, where the size of the first storage space is equal to a sum of the size of the write data and a data size of the first target write address;

a storage unit 22, configured to store the write data and the first target write address in the first storage space;

an updating unit 23, configured to update a correspondence between the first target write address and the first storage address of the first storage space to a mapping table, and record an update time of the correspondence in the mapping table, where the mapping table is used to store a correspondence between an address in the source volume and a storage address of the storage area and an update time of the correspondence.

In this embodiment, the write data in the obtaining unit is data to be written into the source volume in the write operation command, and the first target write address is an address to be written into the write data, and may be an address of a chunk on the source volume to which the write data is to be written, or an address of a chunk on the source volume to which the write data is to be written and an offset address of the data in the chunk. The first storage space is used for storing written data and addresses of data to be written in the source volume, the size of the first storage space is equal to the sum of the size of the written data and the data size of the first target write address, after the data is written, the corresponding relation between the first target write address of the source volume and the first storage address of the first storage space is updated into the mapping table, the first storage address is the address of the first storage space, and the updating time is recorded in the mapping table. After the mapping table is updated, the new write data stored in the preset storage area can be found through the mapping table, and the chunk address on the source volume corresponding to the new write data can also be found.

For example, when a first chunk in the source volume stores data abcd, and a command for writing m at a position d of the first chunk is received before a snapshot generating command is not received, the preset storage area applies for a new storage space to store the data m to be written and an address to be written to a position of a fourth data of the first block of the source volume, that is, an address of d, where the size of the applied storage space is equal to the sum of the size of the data m and the address length of the source volume d. The addresses of d and m in the mapping table are then updated so that the newly written data m can be found when d is read.

In this embodiment, the size of the storage space applied in the preset storage area is equal to the sum of the size of the write data in the write operation and the size of the write address, and the original chunk data does not need to be read, so that the write efficiency is improved, and the storage space is saved.

Further, referring to fig. 3, in a second embodiment of the apparatus for writing data according to the present invention, in a third embodiment of the apparatus for writing data according to the present invention, the apparatus for writing data further includes:

a second obtaining module 30, configured to, in a process of writing and redirecting a snapshot, obtain a target read address in a read operation command when the read operation command for the source volume is received before a snapshot generating command is not obtained;

a first determining module 40, configured to determine, if the mapping table includes the target read address, whether the corresponding update time of the target read address in the mapping table is after a snapshot is generated based on the target read address for the last time;

a data merging module 50, configured to, when the mapping table includes the target read address and the update time of the target read address in the mapping table is after a snapshot is generated based on the target read address for the last time, search a second storage address corresponding to the target read address in the mapping table, obtain data in the second storage address in the storage area, merge the data in the second storage address and the obtained first data into second data, apply for a second storage space in the storage area to store the second data, and update a correspondence between the target read address and a storage address of the second storage space into the mapping table, where the first data is data in a storage block corresponding to the target read address on the source volume;

a read operation module 60, configured to read data in a storage block corresponding to the target read address on the source volume when the mapping table does not include the target read address or the mapping table includes the target read address but the corresponding update time of the target read address in the mapping table is not after the snapshot is generated based on the target read address for the last time.

In this embodiment, in the process of redirecting snapshots during writing, when an old snapshot has been generated but a new snapshot generation command has not been received, or when the snapshot generation command has not been received by the first snapshot, a read operation command for the source volume may be received, and the read operation command may read the source volume or may read only some chunks. Meanwhile, the write operation command may or may not be received before the read operation command is received, that is, the write operation may or may not be performed before the read operation command is received.

The target read address in the second obtaining module 30 is an address of data to be read in the read operation command. When a read operation command is received, a target read address in the read operation command is obtained, a mapping table is searched according to the address of the read operation command, whether a record of the target read address exists in the mapping table or not is judged, if yes, the target read address is indicated to be used as a target write address to perform write operation recording, whether the corresponding update time of the target read address in the mapping table is after a snapshot is generated for the last time based on the target read address or not is judged according to the update time of the record of the target read address, and the purpose is to judge whether the write operation time is between the generation of the target read address and the receiving of the current time. When the target read address to be read is not recorded in the mapping table, which indicates that the target read address to be read has not been subjected to the write operation, the data on the target read address is not changed, and at this time, the data in the storage block corresponding to the target read address on the source volume can be directly read. When the block or the source volume to be read is subjected to write operation, but the update time of the write operation is judged to be before the snapshot is generated for the last time according to the update time recorded by the target read address, it is indicated that no write operation is performed between the snapshot is generated for the last time and the read operation is received this time, the data on the block or the source volume to be read does not change after the snapshot is generated for the last time, and the data in the storage block corresponding to the target read address on the source volume can be directly read at this time. When the mapping table contains the record of the target read address and the corresponding update time of the target read address in the mapping table is after the snapshot is generated based on the target read address for the last time, it is described that the target read address is subjected to the write operation after the snapshot is generated, that is, it is indicated that the data stored in the storage block corresponding to the target read address is updated after the snapshot is generated for the last time, and because only the data to be actually written is written in the write operation, when the whole block or the source volume is to be subjected to the read operation, the source volume and the data in the write space need to be read in combination.

When the target read address is judged to be written between the last generation time and the current received read operation during the read operation, the target read address is indicated to be used as the target write address for writing. The second storage address is an address for storing write data in a newly applied space in a preset storage area when the write operation is executed at the second target write address, and the corresponding relationship between the second storage address and the target read address can be obtained through a mapping table. The first data is data in a storage block corresponding to the target read address on the source volume. And after the first data and the data in the second storage address are acquired, merging the second storage address and the first data, specifically merging the data in the second storage address to the corresponding position of the first data, wherein the merged data is the second data. The second storage space is a storage space newly applied in the preset storage area and used for storing second data, the second data is the latest data after write operation, and the second storage space can store the second data and the address of the source volume corresponding to the second data. When reading the data, the second data in the second storage space at the moment is read. And when the second storage space is newly applied for storing second data, updating the corresponding relation between the second target write address and the address of the second storage space into the mapping table, and releasing the original space, namely releasing the space applied for storing the write-in data and the storage address when the original write-in data is released.

For example, in the process of redirecting snapshots during writing, snapshots are generated at A, B, C in a sequence, and after the snapshot at time a is generated, if a read operation command for a source volume is obtained, a read address in the read operation command is obtained at this time. And when the read address is the third chunk, judging whether the position has write operation during the period of receiving the read operation command and generating the snapshot for the last time according to the record of the third chunk in the lookup mapping table, and directly reading the data in the third chunk when no record of the third chunk exists or the update time of the record of the third chunk is not behind the time A. If the data stored in the third chunk is abcde, the record of the third chunk is obtained from the mapping table, and the update time is after a time, which indicates that the write operation is performed after the time a, the address in the storage area corresponding to the third chunk is found by searching the mapping table, the data stored in the storage area is found to be m according to the address in the storage area corresponding to the third chunk, the modified position is the fifth offset address in the third chunk, i.e., e in the abcde is modified to be m, at this time, m and abcde are merged, specifically, m is written into e of the third chunk to generate new data abcd, a new space is applied for the new space in the preset storage area to store abcd and the address corresponding to the third chunk, the address of the new space in the source volume and the address of the new space in the storage area for storing abcd in the third chunk in the mapping table are updated, and the space for storing e is released. When data is read, the latest data abcdm is read.

In this embodiment, data is read by determining whether there is a write operation between the received read command and the most recently generated snapshot, so that the read data is the latest real-time data.

Further, referring to fig. 4, based on the above-mentioned embodiment of the apparatus for writing data according to the present invention, in a fourth embodiment of the apparatus for writing data according to the present invention, the apparatus for writing data further includes:

a receiving module 70, configured to, if a snapshot generating command for the source volume is received in a process of redirecting a snapshot during writing, determine, according to the mapping table, whether there is a write operation before the source volume receives the snapshot generating command;

a second determining module 80, configured to, when there is a write operation before the source volume receives the snapshot generating command, acquire an address of the write operation on the source volume as a third target write address, search, according to the mapping table, an address in the storage area corresponding to the third target write address, acquire data in the address as third data, and acquire a size of the third data at the same time;

and a snapshot generating module 90, configured to generate a snapshot according to the third data, the size of the third data, and the third target write address.

In this embodiment, in the process of redirecting a snapshot during writing, when a snapshot generating command for a source volume is received, the source volume may be snapshot for the first time, or a new snapshot generating command received under the condition that an existing old snapshot is generated. Before receiving a snapshot generating command, a source volume may have performed a data writing operation or a data reading operation, and when there is a write redirection, it indicates that data in the source volume has been modified by write data. When no write is performed, the contents of the source volume are the most current data at this point.

The third target write address in the second determining module is an address to which data is to be written in a write operation on the source volume. The third target write address and the corresponding address in the storage area can be queried through the mapping table, the data in the corresponding address in the storage area is obtained as the third data, when the write operation is performed and the read operation is not performed, the write data stored in the storage area when the write data is stored in the corresponding address, and when the write operation is performed and the read operation is performed, the latest data obtained by combining the original write data and the original chunk when the read data is stored in the corresponding address. How to perform data merging is specifically described in the second embodiment of the data writing device in detail, and details are not repeated here.

The method for judging whether the source volume has write operation before receiving the snapshot generating command can be obtained by searching a mapping table. The size of the third data can be obtained by acquiring the byte number of the third data, and then a snapshot is generated according to the size of the third data, the size of the third data and the write address of the third data.

Preferably, referring to fig. 5, in this embodiment, the snapshot generating module 90 includes:

a determining unit 91, configured to determine whether a size of the third data is equal to a size of a third storage block corresponding to the source volume at the third target write address;

a write operation unit 92, configured to write the third data back to a third storage block of the source volume when the size of the third data is equal to the size of the third storage block;

a snapshot generating unit 93, configured to, when the size of the third data is not equal to the size of the third storage block, obtain data in the third storage block, merge the third data and the data in the third storage block into fourth data, and write the fourth data back to the third storage block of the source volume.

In this embodiment, the third storage block refers to a block with a third target write address corresponding to the source volume, that is, a block corresponding to a write operation when the write operation is performed, for example, when the write operation is received to write aaaa in a second chunk of the source volume into aaac, the third target write address is an address of the second chunk, the written data is c, and the third storage block corresponding to the third target write address is the second chunk. And judging whether the size of the third data is equal to that of the third storage block, if so, indicating that the original chunk data is completely rewritten by writing data through the write operation, or generating an over-read operation after the write operation, wherein the write data in the storage space subjected to write redirection is the latest data, and writing the third data back to the third storage block of the source volume, namely writing back to the chunk where the over-write operation is generated.

When the size of the third data is not equal to the size of the third storage block, it indicates that the write operation is performed at this time, but only a part of the data in chunk is rewritten and the read operation is not performed, and the latest data at this time needs to be acquired when the snapshot is generated. And at this time, acquiring data in the third storage block during write operation, merging the third data with the data in the third storage block to generate fourth data, wherein the fourth data is the latest data corresponding to the third storage block of the source volume, and writing the fourth data back to the third storage block of the source volume.

For example, before the snapshot generating command is received, the data stored in the third chunk on the source volume is written for abcde, e is modified to m, and no read operation command is received. At this time, data in the storage space of the write operation, namely m, is acquired, the size of m is compared with the size of chunk, if the size of m is smaller than the size of chunk, the data of the third chunk on the source volume is acquired, and m and abcde in the third chunk are combined to generate new data. Specifically, m is written into the corresponding position of the third chunk and is stored as new data abcdm, at this time, abcdm is the latest data of the third chunk of the source volume, and abcdm is written into the third chunk on the source volume, at this time, the data in the third chunk is the content of the generated snapshot.

In this embodiment, by determining whether a write operation exists before the snapshot generating command is received, acquiring data of the write operation, and generating the snapshot according to the size of the data of the write operation, it is ensured that the data is the latest real-time data when the snapshot is generated, and meanwhile, by selectively writing the data to the size of the data of the write operation, the efficiency of generating the snapshot is improved.

Referring to fig. 6, a first embodiment of the method for writing data of the present invention is proposed, in which the method for writing data includes the steps of:

step S10, in the process of redirecting the snapshot during writing, before the snapshot generating command is not obtained, when a writing operation command to the source volume is received, obtaining the size information of the written data contained in the writing operation command;

and step S20, according to the size information of the written data, applying a storage space matched with the size information from a preset storage area, and performing write operation in the applied storage space.

The method for writing data provided by the invention is mainly applied to the distributed storage technology, and is a method for writing data when a snapshot technology of redirection during writing is used. In a typical distributed storage scenario, data is stored in a block (Chunk) as a minimum unit, each storage system may divide a storage space into several chunks with the same size as required, and store the data in the chunks, for example, the storage space may be divided into several blocks, where the size of each block may be divided into 4 megabytes or 112 bytes.

In this embodiment, in the process of redirecting snapshots during writing, when an old snapshot has been generated but a new snapshot generation command has not been received, or when a snapshot generation command has not been received in a first snapshot, a write operation command to a source volume may be received. For example, the snapshot is taken at A, B, C three times in the process of redirecting the snapshot during writing, after the snapshot is completed at time a and before the snapshot is generated at time B, that is, before the snapshot generation command is received, a write operation command or a read operation command may be received, and the write operation or the read operation occurring in the process of redirecting the snapshot during writing is specifically a write redirection operation or a read redirection operation. The source volume refers to a data storage space to be backed up in a snapshot process, and a plurality of blocks can be snapshot according to needs. The write operation is to write data to the source volume, and when a write operation command for the source volume is received, size information of the write data included in the write operation command is obtained, where the write data is data to be written to the source volume, and the data is usually stored in a memory in a byte form, so that obtaining the size information of the write data can be obtained by obtaining the number of bytes of the write data.

And after the size information of the written data is acquired, applying for a storage space in a preset storage area according to the size information of the written data instead of applying for the storage space according to the size information of the chunk, and then performing write operation in the newly applied storage space. The preset storage area is other storage space except the source volume in the storage space, and specifically, a new space can be selected for storage according to needs.

In this embodiment, in the process of redirecting a snapshot during writing, before a snapshot generating command is not obtained, when a write operation command to a source volume is received, size information of write data included in the write operation command is obtained; and according to the size information of the written data, applying a storage space matched with the size information from a preset storage area, and performing write operation in the applied storage space. The data space matched with the size information of the write-in data in the write-in operation is applied, the write-in data contained in the write-in operation command and the address to be written in the source volume are written in, the data of the original chunk do not need to be read, and the purpose of efficiently writing the data is achieved.

Further, in a second embodiment of the method for writing data according to the present invention, the write operation command further includes the write data, and a first target write address where the write data is to be written to the source volume,

referring to fig. 7, the step of refining of the above step S20 includes:

step S21, applying for a first storage space from the storage area according to the size information of the written data, where the size of the first storage space is equal to the sum of the size of the written data and the data size of the first target write address;

step S22, storing the written data and the first target write address in the first storage space;

step S23, updating the correspondence between the first target write address and the first storage address of the first storage space into a mapping table, and recording the update time of the correspondence in the mapping table, where the mapping table is used to store the correspondence between the address in the source volume and the storage address of the storage area and the update time of the correspondence.

In this embodiment, the write data is data to be written to the source volume in the write operation command, and the first target write address is an address to be written by the write data, and may be an address of a chunk on the source volume to which the write data is to be written, or an address of a chunk on the source volume to which the write data is to be written and an offset address of the data in the chunk. The first storage space is used for storing written data and addresses of the source volume to which the data are written, and the size of the first storage space is equal to the sum of the size of the written data and the data size of the first target writing address. After data is written, a corresponding relationship between a first target write address of a source volume and a first storage address of a first storage space is updated to a mapping table, where the first storage address is an address of the first storage space, and an update time is recorded in the mapping table. After the mapping table is updated, the new write data stored in the preset storage area can be found through the mapping table, and the chunk address on the source volume corresponding to the new write data can also be found.

For example, when a first chunk in the source volume stores data abcd, and a command for writing m at a position d of the first chunk is received before a snapshot generating command is not received, the preset storage area applies for a new storage space to store the data m to be written and an address to be written to a position of a fourth data of the first block of the source volume, that is, an address of d, where the size of the applied storage space is equal to the sum of the size of the data m and the address length of the source volume d. The addresses of d and m in the mapping table are then updated so that the newly written data m can be found when d is read.

In this embodiment, the size of the storage space applied in the preset storage area is equal to the sum of the size of the write data in the write operation and the size of the write address, and the original chunk data does not need to be read, so that the write efficiency is improved, and the storage space is saved.

Further, referring to fig. 8, based on the second embodiment of the method for writing data of the present invention, in the third embodiment of the method for writing data of the present invention, the method for writing data further includes the steps of:

step S30, in the process of writing and redirecting the snapshot, before the snapshot generating command is not obtained, when a read operation command for the source volume is received, a target read address in the read operation command is obtained;

step S40, determining whether the mapping table includes the target read address; if yes, go to step S70; otherwise, executing step S50;

step S50, determining whether the corresponding update time of the target read address in the mapping table is after the snapshot is generated based on the target read address for the last time; if yes, go to step S60; otherwise, go to step S70;

step S60, searching a second storage address corresponding to the target read address in the mapping table, acquiring data in the second storage address in the storage area, merging the data in the second storage address and the acquired first data into second data, applying for a second storage space in the storage area to store the second data, and updating a correspondence between the target read address and a storage address of the second storage space into the mapping table, where the first data is data in a storage block corresponding to the target read address on the source volume;

and step S70, reading the data in the storage block corresponding to the target read address on the source volume.

In this embodiment, in the process of redirecting snapshots during writing, when an old snapshot has been generated but a new snapshot generation command has not been received, or when the snapshot generation command has not been received in the first snapshot, a read operation command for the source volume may be received, and the read operation command may read the source volume or may read only some of the chunks. Meanwhile, the write operation command may or may not be received before the read operation command is received, that is, the write operation may or may not be performed before the read operation command is received.

The target read address is an address where data to be read in the read operation command is acquired. When a read operation command is received, a target read address in the read operation command is obtained, a mapping table is searched according to the address of the read operation command, whether a record of the target read address exists in the mapping table or not is judged, if yes, the target read address is indicated to be used as a target write address to perform write operation recording, whether the corresponding update time of the target read address in the mapping table is after a snapshot is generated for the last time based on the target read address or not is judged according to the update time of the record of the target read address, and the purpose is to judge whether the write operation time is between the generation of the target read address and the receiving of the current time. When the target read address to be read is not recorded in the mapping table, which indicates that the target read address to be read has not been subjected to the write operation, the data on the target read address is not changed, and at this time, the data in the storage block corresponding to the target read address on the source volume can be directly read. When the block or the source volume to be read is subjected to write operation, but the update time of the write operation is judged to be before the snapshot is generated for the last time according to the update time recorded by the target read address, it is indicated that no write operation is performed between the snapshot is generated for the last time and the read operation is received this time, the data on the block or the source volume to be read does not change after the snapshot is generated for the last time, and the data in the storage block corresponding to the target read address on the source volume can be directly read at this time. When the mapping table contains the record of the target read address and the corresponding update time of the target read address in the mapping table is after the snapshot is generated based on the target read address for the last time, it is described that the target read address is subjected to the write operation after the snapshot is generated, that is, it is indicated that the data stored in the storage block corresponding to the target read address is updated after the snapshot is generated for the last time, and because only the data to be actually written is written in the write operation, when the whole block or the source volume is to be subjected to the read operation, the source volume and the data in the write space need to be read in combination.

When the target read address is judged to be written between the last generation time and the current received read operation during the read operation, the target read address is indicated to be used as the target write address for writing. The second storage address is an address for storing write data in a newly applied space in a preset storage area when the write operation is executed at the second target write address, and the corresponding relationship between the second storage address and the target read address can be obtained through a mapping table. The first data is data in a storage block corresponding to the target read address on the source volume. And after the first data and the data in the second storage address are acquired, merging the second storage address and the first data, specifically merging the data in the second storage address to the corresponding position of the first data, wherein the merged data is the second data. The second storage space is a storage space newly applied in the preset storage area and used for storing second data, the second data is the latest data after write operation, and the second storage space can store the second data and the address of the source volume corresponding to the second data. When reading the data, the second data in the second storage space at the moment is read. And when the second storage space is newly applied for storing second data, updating the corresponding relation between the second target write address and the address of the second storage space into the mapping table, and releasing the original space, namely releasing the space applied for storing the write-in data and the storage address when the original write-in data is released.

For example, in the process of redirecting snapshots during writing, snapshots are generated at A, B, C in a sequence, and after the snapshot at time a is generated, if a read operation command for a source volume is obtained, a read address in the read operation command is obtained at this time. And when the read address is the third chunk, judging whether the position has write operation during the period of receiving the read operation command and generating the snapshot for the last time according to the record of the third chunk in the lookup mapping table, and directly reading the data in the third chunk after the record of the third chunk does not exist or the updating time of the record of the third chunk is not at the A moment. If the data stored in the third chunk is abcde, the record of the third chunk is obtained from the mapping table, and the update time is after a time, which indicates that the write operation is performed after the time a, the address in the storage area corresponding to the third chunk is found by searching the mapping table, the data stored in the storage area is found to be m according to the address in the storage area corresponding to the third chunk, the modified position is the fifth offset address in the third chunk, i.e., e in the abcde is modified to be m, at this time, m and abcde are merged, specifically, m is written into e of the third chunk to generate new data abcd, a new space is applied for the new space in the preset storage area to store abcd and the address corresponding to the third chunk, the address of the new space in the source volume and the address of the new space in the storage area for storing abcd in the third chunk in the mapping table are updated, and the space for storing e is released. When data is read, the latest data abcdm is read.

In this embodiment, data is read by determining whether there is a write operation between the received read command and the most recently generated snapshot, so that the read data is the latest real-time data.

Further, referring to fig. 9, based on the above-mentioned embodiment of the method for writing data according to the present invention, in a fourth embodiment of the method for writing data according to the present invention, the method for writing data further includes the steps of:

step S80, if receiving a snapshot generating command to the source volume in the process of redirecting the snapshot during writing, judging whether the source volume has writing operation before receiving the snapshot generating command according to the mapping table;

step S90, when there is a write operation before the snapshot generating command is received by the source volume, acquiring an address of the write operation to the source volume as a third target write address, searching an address in the storage area corresponding to the third target write address according to the mapping table, acquiring data in the address as third data, and acquiring a size of the third data;

and step S100, generating a snapshot according to the third data, the size of the third data and the third target write address.

In this embodiment, in the process of redirecting a snapshot during writing, when a snapshot generating command for a source volume is received, the source volume may be snapshot for the first time, or a new snapshot generating command received under the condition that an existing old snapshot is generated. Before receiving a snapshot generating command, a source volume may have performed a data writing operation or a data reading operation, and when there is a write redirection, it indicates that data in the source volume has been modified by write data. When no write is performed, the contents of the source volume are the most current data at this point.

The third target write address is an address to which data is to be written in a write operation on the source volume. The third target write address and the corresponding address in the storage area can be queried through the mapping table, the data in the corresponding address in the storage area is obtained as the third data, when the write operation is performed and the read operation is not performed, the write data stored in the storage area when the write data is stored in the corresponding address, and when the write operation is performed and the read operation is performed, the latest data obtained by combining the original write data and the original chunk when the read data is stored in the corresponding address. How to perform data merging is specifically described in detail in the second embodiment of the data writing method, and details are not described here.

The method for judging whether the source volume has write operation before receiving the snapshot generating command can be obtained by searching a mapping table. The size of the third data can be obtained by acquiring the byte number of the third data, and then a snapshot is generated according to the size of the third data, the size of the third data and the write address of the third data.

Preferably, referring to fig. 10, in this embodiment, the generating the snapshot according to the third data, the size of the third data, and the third target write address includes:

step S110, determining whether the size of the third data is equal to the size of a third storage block corresponding to the third target write address in the source volume; if yes, go to step S120; otherwise, go to step S130;

step S120, writing the third data back to a third storage block of the source volume;

step S130, acquiring data in the third storage block, merging the third data and the data in the third storage block into fourth data, and writing the fourth data back to the third storage block of the source volume.

In this embodiment, the third storage block refers to a block with a third target write address corresponding to the source volume, that is, a block corresponding to a write operation when the write operation is performed, for example, when the write operation is received to write aaaa in a second chunk of the source volume into aaac, the third target write address is an address of the second chunk, the written data is c, and the third storage block corresponding to the third target write address is the second chunk. And judging whether the size of the third data is equal to that of the third storage block, if so, indicating that the original chunk data is completely rewritten by writing data through the write operation, or generating an over-read operation after the write operation, wherein the write data in the storage space subjected to write redirection at the moment is the latest data, and writing the third data back to the third storage block of the source volume, namely writing back to the chunk where the over-write operation is generated.

When the size of the third data is not equal to the size of the third storage block, it indicates that the write operation is performed at this time, but only a part of the data in chunk is rewritten and the read operation is not performed, and the latest data at this time needs to be acquired when the snapshot is generated. And at this time, acquiring data in the third storage block during write operation, merging the third data with the data in the third storage block to generate fourth data, wherein the fourth data is the latest data corresponding to the third storage block of the source volume, and writing the fourth data back to the third storage block of the source volume.

For example, before the snapshot generating command is received, the data stored in the third chunk on the source volume is written for abcde, e is modified to m, and no read operation command is received. At this time, data in the storage space of the write operation, namely m, is acquired, the size of m is compared with the size of chunk, if the size of m is smaller than the size of chunk, the data of the third chunk on the source volume is acquired, and m and abcde in the third chunk are combined to generate new data. Specifically, m is written into the corresponding position of the third chunk and is stored as new data abcdm, at this time, abcdm is the latest data of the third chunk of the source volume, and abcdm is written into the third chunk on the source volume, at this time, the data in the third chunk is the content of the generated snapshot.

In this embodiment, by judging whether a write operation exists before the snapshot generating command is received, acquiring data of the write operation, and generating the snapshot according to the size of the data of the write operation, the data is ensured to be the latest real-time data when the snapshot is generated, and meanwhile, by selectively writing the data according to the size of the data of the write operation, the efficiency of generating the snapshot is improved.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims (10)

1. An apparatus for writing data, comprising:

the first acquisition module is used for acquiring the size information of the write data contained in a write operation command when the write operation command to a source volume is received before the snapshot generating command is not acquired in the process of redirecting the snapshot during writing;

the write operation module is used for applying a storage space matched with the size information from a preset storage area according to the size information of the written data and performing write operation in the applied storage space;

wherein, the device for writing data further comprises:

the second obtaining module is used for obtaining a target read address in a read operation command when the read operation command for the source volume is received before a snapshot generating command is not obtained in the process of writing and redirecting the snapshot;

a data merging module, configured to, when a mapping table includes the target read address and update time of the target read address in the mapping table is after a snapshot is generated based on the target read address for the last time, search a second storage address corresponding to the target read address in the mapping table, obtain data in the second storage address in the storage area, merge the data in the second storage address and the obtained first data into second data, apply for a second storage space in the storage area to store the second data, and update a correspondence between the target read address and a storage address of the second storage space to the mapping table, where the first data is data in a storage block corresponding to the target read address on the source volume.

2. The apparatus for writing data according to claim 1, wherein the write operation command includes the write data, and the write data is to be written to the first target write address of the source volume;

the write operation module includes:

a storage space application unit, configured to apply for a first storage space from the storage area according to size information of the write data, where the size of the first storage space is equal to a sum of the size of the write data and a data size of the first target write address;

a storage unit, configured to store the write data and the first target write address in the first storage space;

and the updating unit is configured to update the correspondence between the first target write address and the first storage address of the first storage space into the mapping table, and record an update time of the correspondence in the mapping table, where the mapping table is used to store the correspondence between the address in the source volume and the storage address in the storage area and the update time of the correspondence.

3. The apparatus for writing data of claim 1, further comprising:

a first determining module, configured to determine, if the mapping table includes the target read address, whether update time corresponding to the target read address in the mapping table is after a snapshot is generated based on the target read address for the last time;

and the read operation module is used for reading the data in the storage block corresponding to the target read address on the source volume when the mapping table does not contain the target read address or the mapping table contains the target read address but the corresponding update time of the target read address in the mapping table is not after the snapshot is generated based on the target read address for the last time.

4. The apparatus for writing data according to claim 2 or 3, wherein the apparatus for writing data further comprises:

the receiving module is used for judging whether the source volume has write operation before receiving the snapshot generating command according to the mapping table if the snapshot generating command to the source volume is received in the process of redirecting the snapshot during writing;

a second judging module, configured to, when there is a write operation before the source volume receives the snapshot generating command, acquire an address of the write operation on the source volume as a third target write address, search, according to the mapping table, an address in the storage area corresponding to the third target write address, acquire data in the address as third data, and acquire a size of the third data at the same time;

and the snapshot generating module is used for generating a snapshot according to the third data, the size of the third data and the third target write address.

5. The apparatus for writing data of claim 4, wherein the snapshot generation module comprises:

a determining unit, configured to determine whether a size of the third data is equal to a size of a third storage block corresponding to the source volume at the third target write address;

a write operation unit, configured to write the third data back to a third storage block of the source volume when the size of the third data is equal to the size of the third storage block;

and the snapshot generating unit is configured to, when the size of the third data is not equal to the size of the third storage block, acquire data in the third storage block, merge the third data and the data in the third storage block into fourth data, and write the fourth data back to the third storage block of the source volume.

6. A method of writing data, the method comprising the steps of:

in the process of redirecting snapshots during writing, before a snapshot generating command is not acquired, when a writing operation command for a source volume is received, acquiring the size information of written data contained in the writing operation command;

according to the size information of the written data, applying a storage space matched with the size information from a preset storage area, and performing writing operation in the applied storage space;

wherein, the method for writing data further comprises:

in the process of writing and redirecting the snapshot, before a snapshot generating command is not obtained, when a read operation command of the source volume is received, a target read address in the read operation command is obtained;

if the mapping table includes the target read address and the update time of the target read address in the mapping table is after a snapshot is generated based on the target read address for the last time, a second storage address corresponding to the target read address is searched in the mapping table, data in the second storage address in the storage area is obtained, the data in the second storage address and the obtained first data are combined into second data, the second data are applied to a second storage space in the storage area to be stored, the corresponding relation between the target read address and the storage address of the second storage space is updated to the mapping table, and the first data are data in a storage block corresponding to the target read address on the source volume.

7. The method for writing data according to claim 6, wherein the write operation command further includes the write data, and the write data is to be written to the first target write address of the source volume;

then, the applying for the storage space matched with the size information from a preset storage area according to the size information of the written data, and performing a write operation in the applied storage space includes:

applying for a first storage space from the storage area according to the size information of the written data, wherein the size of the first storage space is equal to the sum of the size of the written data and the data size of the first target write address;

storing the write data and the first target write address in the first storage space;

and updating the corresponding relation between the first target write address and the first storage address of the first storage space into the mapping table, and recording the updating time of the corresponding relation in the mapping table, wherein the mapping table is used for storing the corresponding relation between the address in the source volume and the storage address of the storage area and the updating time of the corresponding relation.

8. The method of writing data of claim 7, further comprising:

if the mapping table contains the target reading address, judging whether the corresponding updating time of the target reading address in the mapping table is after the snapshot is generated based on the target reading address for the last time;

and if the mapping table does not contain the target read address, or the mapping table contains the target read address but the corresponding update time of the target read address in the mapping table is not after the snapshot is generated based on the target read address for the last time, reading the data in the storage block corresponding to the target read address on the source volume.

9. The method of writing data of claim 7 or 8, further comprising:

if a snapshot generating command for the source volume is received in the process of redirecting the snapshot during writing, judging whether the source volume has writing operation before receiving the snapshot generating command according to the mapping table;

when the source volume has write operation before receiving the snapshot generating command, acquiring an address of the write operation to the source volume as a third target write address, searching an address in the storage area corresponding to the third target write address according to the mapping table, acquiring data in the address as third data, and acquiring the size of the third data;

and generating a snapshot according to the third data, the size of the third data and the third target write address.

10. The method of writing data of claim 9, wherein the generating a snapshot based on the third data, the size of the third data, and the third target write address comprises:

judging whether the size of the third data is equal to the size of a third storage block corresponding to the third target write address in the source volume;

if so, writing the third data back to a third storage block of the source volume;

if not, acquiring the data in the third storage block, merging the third data and the data in the third storage block into fourth data, and writing the fourth data back to the third storage block of the source volume.

Images