CN114265551B

CN114265551B - Data processing method in storage cluster, storage node and equipment

Info

Publication number: CN114265551B
Application number: CN202111463112.7A
Authority: CN
Inventors: 李舒
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2021-12-02
Filing date: 2021-12-02
Publication date: 2023-10-20
Anticipated expiration: 2041-12-02
Also published as: CN114265551A

Abstract

The embodiment of the application provides a data processing method in a storage cluster, the storage cluster, a storage node and equipment. According to the technical scheme provided by the embodiments of the application, the repeated data in the storage node is eliminated with low delay in the first stage; in order to ensure the performance of the storage cluster, the second stage can be executed in a non-blocking mode, and whether second data which is repeated with the first data are stored in other storage nodes in the storage cluster is checked; when the verification result shows that the storage cluster is stored, the first storage node executes the deduplication operation for the first data, namely, the performance of the storage cluster is not greatly influenced while the deduplication is realized, and the efficiency of infrastructure such as computing, storage and network of the storage cluster is improved.

Description

Data processing method in storage cluster, storage node and equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to a data processing method in a storage cluster, a storage node, and a device.

Background

The cloud storage technology brings convenience storage service to users, and the users can purchase and utilize storage resources according to requirements in the cloud storage system. Cloud service providers (Cloud Storage Provider, CSP) typically provide virtualized environments to decouple hardware differences from the capacity used by cloud users. Meanwhile, with the development trend of CSP expanding outwards, in order to meet the increasing storage capacity and data reliability requirements, distributed storage clusters are mostly adopted as the back end. Storage capacity and bandwidth consumption result in increased costs, which becomes more critical when cloud users create large amounts of data to store from time to time. How to effectively reduce the storage cost has important significance for the development of CSP.

A storage cluster stores a large amount of data, and there must be a large amount of duplicate content. Repeatedly storing the same data consumes not only memory resources, but also computing resources and network resources.

Disclosure of Invention

Embodiments of the present application provide a deduplication scheme to reduce duplicate data in a storage cluster.

In one embodiment of the application, a data processing method is provided that is applicable to a first storage node in a storage cluster. Specifically, the method comprises the following steps:

determining whether the first storage node has stored second data that is repeated with the first data;

when the result is not stored, executing the storage operation of the first data;

applying for verifying whether the second data is stored in the storage nodes except the first storage node in the storage cluster;

and when the verification result is that the storage exists, performing the deduplication operation on the first data.

In another embodiment of the present application, a data processing method is provided, adapted for use with a second storage node in a storage cluster. Specifically, the method comprises the following steps:

receiving a determination result of a first storage node in the storage cluster for first data transmission, wherein the determination result is transmitted after the first storage node determines whether second data which is repeated with the first data is stored on the first storage node;

When the determination result is that the second data is not stored, applying for verifying whether the second data is stored in the storage nodes except the first storage node in the storage cluster;

when the verification result is that the storage exists, feeding back corresponding response information to the first storage node;

the response information is used for preventing the first data from being stored on the self node by the first storage node or triggering the first storage node to delete the first data stored on the first storage node.

In yet another embodiment of the present application, a storage cluster is provided. The storage cluster comprises a plurality of storage nodes, wherein the storage nodes are connected through a network; the plurality of storage nodes comprise a first storage node and a second storage node; wherein,,

the first storage node is used for determining whether the first storage node stores second data which is repeated with the first data or not; when the result is not stored, executing the storage operation of the first data;

the second storage node is used for verifying whether the second data are stored in the storage nodes except the first storage node on the storage cluster or not when the determined result is not stored, and feeding back a verification result to the first storage node;

The first storage node is further configured to perform a deduplication operation with respect to the first data when the verification result is that there is a store.

In yet another embodiment of the present application, a storage node is provided. The storage node includes:

the network card is used for receiving first data to be stored and determining whether the storage node stores second data which is repeated with the first data or not;

the processor is electrically connected with the network card and is used for executing the storage operation of the first data when the determination result is that the first data is not stored;

the network card is further configured to send a verification request for the first data to a target node in a storage cluster, so that the target node verifies whether a node except the storage node in the storage cluster stores the second data;

and the processor is further configured to execute a deduplication operation for the first data when the verification result received by the network card is stored.

the network card is used for receiving a determination result sent by a first storage node in the storage cluster aiming at the first data, wherein the determination result is sent after the first storage node determines whether second data which is repeated with the first data is stored on the first storage node or not;

The processor is electrically connected with the network card and is used for applying to check whether the second data are stored in the storage nodes except the first storage node in the storage cluster when the determined result is not stored;

the network card is further used for feeding back corresponding response information to the first storage node when the verification result is stored;

In yet another embodiment of the present application, an electronic device is provided. The electronic device includes a processor and a memory; the memory is configured to store one or more computer instructions that, when executed by the processor, enable the steps in the above-described embodiments of the data processing method.

The embodiments of the present application also provide a computer-readable storage medium storing a computer program capable of implementing the steps in the embodiments of the data processing method described above when the computer program is executed by a computer.

The embodiment of the application also provides a computer program product. The computer program product comprises a computer program which, when executed by a computer, causes the computer to carry out the steps of the data processing methods described above.

The technical scheme provided by the embodiment of the application designs a two-stage repeated data deleting technology, when first data to be stored exist, a storage node determines whether second data which is repeated with the first data are stored on the storage node in the first stage; and a second stage, determining whether second data which is repeated with the first data is stored on other storage nodes in the storage cluster. The first stage can timely prevent the storage of a certain amount of repeated data; under the condition that the I/O (input/output) performance of the storage cluster is not affected, the second stage can carry out verification in an asynchronous mode, the operation of storing the first data by the storage node is not affected in the second stage verification process, and when the second stage verification result shows that other nodes in the storage cluster store second data which are repeated with the first data, the storage node executes the deduplication operation on the first data.

Therefore, according to the technical scheme provided by the embodiment of the application, the repeated data in the storage node is eliminated at a low delay in the first stage; in order to ensure the performance of the storage cluster, the second stage can be executed in a non-blocking mode, and whether second data which is repeated with the first data are stored in other storage nodes in the storage cluster is checked; when the verification result shows that the storage cluster is stored, the first storage node executes the deduplication operation for the first data, namely, the performance of the storage cluster is not greatly influenced while the deduplication is realized, and the efficiency of infrastructure such as computing, storage and network of the storage cluster is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a solution for providing virtual machines for cloud users and back-end storage support thereof according to an embodiment of the present application;

FIG. 2 is a schematic flow chart of a method for processing data in a storage cluster according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a storage cluster according to an embodiment of the present application;

FIG. 4 is a schematic diagram illustrating a full hash table distributed among multiple master nodes according to an embodiment of the present application;

FIG. 5 is a schematic diagram illustrating a data processing method in a storage cluster according to an embodiment of the present application;

fig. 6 is a schematic diagram of a network card according to an embodiment of the present application to perform a deduplication process;

FIG. 7 is a schematic diagram illustrating a data processing method in a storage cluster according to another embodiment of the present application;

FIG. 8 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a data processing apparatus according to another embodiment of the present application;

fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Before describing embodiments of the present application, the terminology referred to herein will be briefly described.

Repeating data deletion: a technique for eliminating duplicate data. The data de-duplication technique can find duplicate data from already stored data, store no duplicate data, and store only new data.

In addition, what is needed here is: distinction between duplicate data and multiple copy policy. The duplicate data and multiple copy policy differ in that: the repeated data refers to two data with the same content, which appear in the same system, the two stored identical data are equivalent, no disaster recovery function exists between the two identical data, and only one data can be stored. While a multiple copy redundancy strategy is to store another or two copies of the same data on different storage nodes, data redundancy can be achieved between the several copies. When one copy is in error, other copies can be utilized to recover in time. Thus, the duplicate data deletion is found in time: when the same data which can be replaced by the index and does not influence the disaster recovery strategy are stored, only one data need to be stored, so that the utilization rate of the storage space is greatly improved.

Virtual machine: virtualization/emulation of computer systems, virtual machines based on computer architecture, provide the functionality of physical computers.

In order to enable those skilled in the art to better understand the present application, the following description will make clear and complete descriptions of the technical solutions according to the embodiments of the present application with reference to the accompanying drawings.

In some of the flows described in the description of the application, the claims, and the figures described above, a number of operations occurring in a particular order are included, and the operations may be performed out of order or concurrently with respect to the order in which they occur. The sequence numbers of operations such as 101, 102, etc. are merely used to distinguish between the various operations, and the sequence numbers themselves do not represent any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types. Furthermore, the embodiments described below are only some, but not all, embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

FIG. 1 illustrates a schematic diagram of a solution for providing virtual machines for cloud users and back-end storage support therefor. As shown in fig. 1, a hypervisor (hypervisor) is imposed on top of the data center infrastructure consisting of resources (computing, storage, network) and delivers the virtualized environment to meet the needs of the user. On top of the hypervisor (hypervisor), the virtual machines are isolated as independent units, respectively, each with its own operating system (Guest OS) and binary/library files (Bin/lib) installed.

In addition to the data created by the application APP on each virtual machine (e.g., application 1, application 2, … … application n on each independent virtual machine in fig. 1), the installed operating system and binary/library files consume storage resources. Storage clusters located at the back-end do not distinguish between the types of data stored (e.g., operating systems, libraries, etc.). Thus, there is a large amount of duplicate content in the data of different virtual machines. Repeatedly storing the same data consumes not only memory resources, but also computing resources and network resources. The data deleting process is repeated on the stored content, so that the capacity and the bandwidth of the storage cluster can be saved, and the data storage efficiency can be improved. For very large storage clusters, deduplication is particularly important.

The purpose of deduplication is to: when repeated data is present, only one copy is stored, and other data which is repeated with one data replaces the data storage in a simplified manner, such as by using an index instead of the data storage, and in particular, such as by using a pointer to the stored data instead of the repeated data storage. The following embodiments will clearly and fully describe the specific implementation of the deduplication technology provided by the present application.

The technical scheme provided by the application can be realized based on the system architecture shown in fig. 1 and 3. As shown in fig. 1 and 3, a storage cluster includes a plurality of storage nodes connected by a network (e.g., connected to the network by an ethernet switch as shown in fig. 3); the plurality of storage nodes includes a first storage node and a second storage node. The first storage node is used for determining whether the first storage node stores second data which is repeated with the first data or not; and when the determination result is that the first data is not stored, executing the storage operation of the first data. And the second storage node is used for verifying whether the second data are stored in the storage nodes except the first storage node on the storage cluster when the determined result is not stored, and feeding back a verification result to the first storage node. The first storage node is further configured to perform a deduplication operation with respect to the first data when the verification result is that there is a store.

The first storage node in this embodiment may be any one of the storage nodes in the storage cluster in fig. 2. More specifically, the first storage node may be one of the remaining storage nodes in the cluster other than the distributed master node, and the second storage node may be one of the distributed master nodes.

That is, in an achievable system architecture, the storage cluster is a multi-master node architecture, where the plurality of storage nodes further includes a plurality of master nodes, and the second storage node is a master node. As shown in fig. 4, the identification values of the data stored on all the storage nodes in the storage cluster are recorded in a full-scale manner; the full-quantity table is divided into a plurality of sub-tables which are matched with the number of the main nodes; the plurality of sub-tables are free of intersections; the plurality of sub-tables are stored on different main nodes in a scattered manner; one sub-table corresponds to one table range. In specific implementation, the identification value of the data may be a hash value calculated by a hash algorithm. In addition, the data stored in the storage node may be data with full file granularity, data with data block granularity, or the like, which is not limited in this embodiment. The distributed master node in this embodiment is a master node and is also a storage node, and has a read-write function of the storage node.

Correspondingly, in this embodiment, when verifying whether the second data is stored in the storage nodes on the storage cluster except for the first storage node, the second storage node is specifically configured to:

acquiring an identification value of the first data;

checking whether the identification value is in a table range corresponding to the first sub-table; wherein the first sub-table is stored on the second storage node;

when the table range corresponding to the first sub-table is within, checking whether the second data which is repeated with the first data is stored according to the identification value and the table item in the first sub-table;

when the identification value is not in the first sub-table corresponding table range, table ranges corresponding to sub-tables stored by a plurality of main nodes are acquired, and the main node storing the sub-table corresponding to the table range where the identification value is located is determined as the target node; and sending a verification request for the first data to the target node, and waiting for the target node to verify whether a verification result of the second data which is repeated with the first data is stored.

According to the technical scheme provided by the embodiment of the application, the repeated data in the storage node is eliminated at a low delay in the first stage; in order to ensure the performance of the storage cluster, the second stage can be executed in a non-blocking mode, and whether second data which is repeated with the first data are stored in other storage nodes in the storage cluster is checked; when the verification result shows that the storage cluster is stored, the first storage node executes the deduplication operation for the first data, namely, the performance of the storage cluster is not greatly influenced while the deduplication is realized, and the efficiency of infrastructure such as computing, storage and network of the storage cluster is improved.

The functions of each node and the specific implementation of each function in this embodiment will be described in detail in the following method embodiments.

In an embodiment of the method as described in fig. 2, the execution body of the data processing method in the storage cluster may be one storage node in the storage cluster as shown in fig. 1 and 3. Hereinafter, for convenience of description, the method storage node that performs the method provided by this embodiment is referred to as a first storage node. Specifically, as shown in fig. 2, the data processing method includes:

101. determining whether the first storage node has stored second data that is repeated with the first data;

102. when the result is not stored, executing the storage operation of the first data;

103. applying for verifying whether the second data is stored in the storage nodes except the first storage node in the storage cluster;

104. and when the verification result is that the storage exists, performing the deduplication operation on the first data.

The above 101 steps repeat the data discovery process. The data stored in the storage cluster may be a file, or may be one of a plurality of data blocks into which the file is divided, or the like. In specific implementation, a fixed size may be used to divide a file to obtain a plurality of data blocks with the same size, or a variable size may be used to divide a file to obtain a plurality of data blocks with different sizes. In the process of dividing the variable blocks, the boundary of each block is divided by taking the content as a reference in consideration of the content in each block, which causes the size of each block to be inconsistent.

In the repeated data discovery process, if the full file granularity is used, a hash value (hash) is calculated for the entire file, and then the calculated hash value is compared with the hash value of the data stored on the first storage node. If the hash value of the stored data on the first storage node has the same value as the hash value of the file through comparison, the file which is repeated with the content of the file is stored on the first storage node. If the hash value of the stored data on the first storage node is not the same as the hash value of the file through comparison, the file is a new file and can be stored.

In the repeated data discovery process, if the granularity of the data blocks is adopted, calculating a hash value for the content in the data blocks to be stored; the hash value of the stored data block of the first storage node is then looked up. If the hash value of the stored data block of the first storage node has the same value as the hash value of the data block to be stored, the data block to be stored is indicated to be the repeated data block. If the hash value of the stored data of the first storage node has no value identical to the hash value of the file, the data block to be stored is new data and can be stored.

The block-level data deduplication technique is advantageous over file-level data, in that it can eliminate redundant data blocks in different files or applications. For example, if a file is modified only a little bit, the block-level deduplication technique will only save the changed data blocks. In this way, block-level deduplication may provide better compression ratios, particularly in the case of large amounts of data, such as after a database or full backup.

It can be seen that the smaller the granularity of the data, the higher the probability of finding duplicate data, but the correspondingly increased computing resources and storage resources are consumed. Therefore, the granularity of the data needs to be comprehensively considered according to the requirements of the environment.

In practical applications, the hash value of the data stored in the first storage node may be stored in a memory (as shown in fig. 3) of the first storage node; or the index table is stored in the memory of the first storage node, and the index table contains index codes corresponding to hash values of the stored data. Thus, when first data to be stored are needed, a Hash value of the first data can be calculated by utilizing a Hash algorithm; then searching the hash value table or index table stored in the memory. If the hash value table contains the same value as the hash value of the first data (such as the hash value of the second data), determining that the first data is repeated data and not storing the repeated data; if the hash value table does not have the same value as the hash value of the first data, the first data is determined to be new data and can be stored.

The hash algorithm may include MD5 and SHA-X series algorithms, among others. The MD5 Algorithm, which is known as Message-Digest algorism 5, is a mature Hash Digest Algorithm, and the contents of a data block are converted into a 128-bit encrypted Message Digest by a Hash Digest technique. If two different data blocks are operated by MD5 algorithm, the obtained two 128-bit abstract information are different. SHA (Secure Hash algorithm) the secure hash algorithms mainly comprise SHA-1, SHA-256, SHA-512 and the like. In the implementation, the corresponding algorithm can be randomly selected along with the increase of the storage scale so as to reduce the conflict rate of the data.

In 102 above, after determining that the first data is not duplicate data, a store operation for the first data may be performed. A schematic diagram of the hardware architecture of a storage node is shown in fig. 3. The hardware structure of the first storage node in this embodiment may also be as shown in fig. 3, where the first storage node is provided with a hierarchical storage driver, where the hierarchical storage driver includes a log driver and a dump driver. When the first storage node stores data (e.g., first data), the data to be stored may be written to both the log driver and the dump driver. Typically, a log driver (journ driver) will complete a write operation with a shorter delay than a dump driver (dump driver). When at least one of the dump driver and the log driver completes a write operation and is error free, the write operation is deemed to have completed. After the storage operation of the first data is completed (i.e. the writing operation of the first data is completed), the hash value of the first data is stored in the memory, or an index code is created for the hash value of the first data, and the index code is added into an index table of the memory.

If the first data is determined to be repeated data, the first data is not stored; further pointers to the second data may be stored instead of the first data.

The above steps 101 and 102 can be understood as: a single storage node checks the duplication stage. Then the step 103 is a verification stage of global check of the storage cluster, i.e. verifying whether the other nodes in the storage cluster than the first storage node store second data that is repeated with the first data. Step 104 is a remedial action that is performed when the verification result is that the first data is duplicate data.

In the process of deleting the repeated data of the storage cluster, in consideration of the overall performance overhead of the system, global deduplication across storage nodes can seriously reduce the storage performance of the system, so that in the prior art, only data in a single storage node is generally used for carrying out data deduplication. In the embodiment of the present application, the steps 101 and 102 have little influence on the whole system because of the duplicate checking stage of the single storage node, and can be processed in time. In view of the high overall performance overhead of the system caused by global deduplication, the verification of step 103 and the remedial measure of step 104 are adopted, so that steps 103 and 104 can be performed in a non-blocking manner, i.e. can be performed in a delayed manner, such as when the network is idle and the data processing capacity is not high.

Therefore, the technical scheme provided by the embodiment not only realizes the repeated data deletion, but also has little influence on the performance of the storage cluster, thereby being beneficial to improving the efficiency of the storage cluster in the infrastructure such as calculation, storage, network and the like.

In one implementation solution, the storage cluster includes a second storage node. As in the example of fig. 4, the storage cluster includes a distributed master node (Distributed master nodes). Wherein the distributed master node may comprise a plurality of master nodes. The second storage node in this embodiment may be one of the distributed host nodes shown in fig. 4. Specifically, step 103 "the application for verifying whether the storage nodes except the first storage node in the storage cluster store the second data" in the above embodiment may be implemented by the following steps:

1031. acquiring an identification value of the first data;

1032. based on the identification value, sending a verification request to the second storage node to verify by the second storage node whether the second data is stored in the storage nodes except the first storage node in the storage cluster;

1033. and receiving a verification result fed back by the second storage node for the first data.

In 1031, the identification value of the first data has uniqueness, and may be the hash value calculated by using the hash algorithm as mentioned above.

In 1032, the verification request may carry an identification value of the first data. The verification request may be sent to the second storage node when the first storage node determines that the first data is non-duplicate data (i.e., the determination is not stored). Another implementation is to send the verification request to the second storage node regardless of the determination. In this case, the verification request may further carry the determination result in addition to the identification value of the first data. The second storage node receiving the verification request can execute the operation of verifying whether the second data is stored in the storage nodes except the first storage node in the storage cluster when the determination result is that the second data is not stored; the first data may be registered as the duplicate data using the identification value of the first data when the determination result is stored.

For example, in the scheme shown in fig. 5, table a is a registry, and metadata corresponding to the first data (for example, metadata corresponding to the first data to the client node) and a hash value of the first data are added to table a, that is, the registration is completed. After registration is completed, the method is convenient for the subsequent repeated data discovery stage.

The second storage node here may be a master node in a storage cluster. The storage cluster may include one master node or a plurality of master nodes. Such as the architecture diagram shown in fig. 4, the storage cluster is a multi-master architecture. Unlike a master node that uses a separate server as a storage cluster, the distributed master node (Distributed master nodes) operates on storage nodes in the cluster and is not a separate, management-only master node. For example, in one implementation, all hash values corresponding to data stored on a storage cluster may form a full hash table. In implementation, the full hash table can be divided into a plurality of sub-tables according to the range of hash values. The plurality of sub-tables are dispersed into the plurality of master nodes as shown in fig. 4. Wherein, in order to ensure reliability, the master nodes have redundancy, such as standby nodes. The backup node is backed up with data, sub-tables and the like on the main node so as to replace the main node to provide corresponding service when the main node fails or is unavailable.

When the first storage node determines that the second data repeated with the first data is not stored on the own node, after the second storage node (for example, one of the distributed master nodes in fig. 4) receives the verification request sent by the first storage node, when verifying whether the second data is stored in a storage node except for the first storage node in the storage cluster, the method may specifically include: checking whether the hash value (i.e. the identification value) of the first data is in a table range corresponding to a first sub-table stored on the own node; if the hash value of the first data is in the table range corresponding to the first sub-table, checking whether second data which is repeated with the first data is stored or not according to the identification value and the table item in the first sub-table. If the first sub-table (table B shown in fig. 5) contains the same value as the hash value of the first data, it indicates that the second storage node stores second data repeated with the first data; otherwise, the storage nodes except the first storage node in the storage cluster are not used for storing second data repeated with the first data. If the hash value of the first data is not in the table range corresponding to the first sub-table, the second storage node needs to determine a target node (i.e. the main node of the multi-main node, which stores the table range corresponding to the sub-table in which the hash value of the first data is located) in the storage cluster according to the hash value of the first data, and then the target node checks whether the storage nodes except the first storage node in the storage cluster store the second data repeated with the first data.

What is needed here is that: the second storage node needs to store index information of each sub-table scattered on the corresponding main node, so that when the target node is determined, the target node can be determined through the index information, and a verification request carrying the identification value (such as a hash value) of the first data is sent to the target node.

When the first storage node stores first data, a multi-copy disaster recovery strategy can be adopted, namely, the first data are respectively stored on three different storage nodes, so that when one storage node goes wrong, the requirement (such as a data reading requirement of a client side or other data processing requirements) is met by using the copies stored in the other two storage nodes, and the data disaster recovery function is realized. The multi-copy disaster recovery strategy is simple and easy to implement, but needs to consume a large amount of storage space. For this reason, in this embodiment, an erasure coding strategy may be adopted to reduce the usage of the storage space. For example, in the case of 5 data blocks and three copies of disaster recovery, 15 data blocks are required to ensure redundancy of data. And the erasure coding strategy (for example, CRS coding) can also ensure the effect of simultaneous recovery when two data blocks of 5 data blocks are in error, for example, only 5+2 data blocks are needed to ensure the redundancy of the data, thereby greatly improving the utilization rate of the storage space. That is, "performing the storage operation on the first data" in step 102 in this embodiment may include:

1021. When the determined result is not stored, preprocessing the first data;

1022. taking the preprocessed first data as data to be encoded, and adding the data to be encoded into a set to be encoded;

1023. when the quantity of the data in the set to be encoded meets the preset requirement, erasure code encoding is carried out on the plurality of data in the set to be encoded to obtain a plurality of erasure code blocks; wherein, the plurality of data in the set to be encoded and the plurality of erasure code blocks form an erasure code band;

1024. and storing a part of the erasure code bands in the first storage node, and sending the rest of the erasure code bands to one or more storage nodes except the first storage node in the storage cluster for storage.

In 1021, the preprocessing of the first data may include: compression processing, encryption processing, and the like. The specific processing procedure of compression and encryption in this embodiment is not limited, and reference may be made to the related literature.

In the above 1022, in the erasure coding strategy, a plurality of data, such as 5 data (may be a data block), is needed to be encoded together. Therefore, the step of adding the processed first data as the data to be encoded into the set to be encoded is added in this embodiment. The adding step is essentially an accumulating process, such as accumulating 5 pieces of data to be stored, and then encoding the 5 pieces of data (which may be data blocks) using an erasure coding strategy.

The above 1023 process, i.e. erasure code encoding process, is not limited in detail to the erasure code encoding principle and specific encoding process in this embodiment, and may be found in related literature. The basic idea of erasure coding strategy is: and carrying out data encoding on k original data blocks through an erasure coding algorithm to obtain m erasure coding blocks, and storing the k+m data blocks into different storage nodes to finish the establishment of a disaster recovery mechanism. Thus, when any unnecessary m block element in the k+m block elements is in error (including data and redundancy error), the original k block data can be recovered through the corresponding reconstruction algorithm.

For convenience of description, the plurality of data (e.g., the k original data) and the plurality of erasure code blocks (e.g., the m erasure code blocks) in the set to be encoded in the present embodiment constitute an erasure code band (strip). One original data (or original data block) or one erasure code block in an erasure code band may be referred to as a data element (strip). The virtual boxes across multiple storage nodes shown in fig. 2 represent one erasure code band, and the EC band is used in fig. 2 to represent one erasure code band.

In step 1024, a portion of the data elements in the erasure code band may be stored at a first storage node and the remaining data elements may be stored at one or more storage nodes in the storage cluster other than the first storage node. Or, one data element in the erasure code band is stored in the first storage node, and the rest data elements are stored in other storage nodes except the first storage node in the storage cluster in a scattered manner.

What is needed here is that: data elements sent to other storage nodes than the first storage node may skip the local deduplication module. In other words, the storage nodes in the storage cluster are each configured with a deduplication module, which may be an application program installed on the storage node, to implement the steps provided in the present embodiment. The data elements in the EC band that are sent to other storage nodes are disaster-tolerant and do not belong to the object of deduplication. Therefore, the storage node receiving one data element in the EC band does not start the repeated data deleting module, namely does not check the repetition, and directly stores the received one data element in the EC band locally.

Specifically, as shown in FIG. 2, each EC band may be managed by a distributed master node in a storage cluster. For example, the master node manages information about which storage node each data element in the EC band is stored in. This allows a specified number of data elements belonging to the same EC band to be acquired in the storage cluster based on the managed information at the time of data recovery to recover the original data.

In order to improve the utilization rate of the storage space and the network bandwidth, the application of erasure coding and repeated data deleting technology in the cloud storage environment has good practical significance. In a storage system, the erasure coding technology mainly encodes original data by using an erasure algorithm to obtain redundant data, and stores the original data and the redundant data together so as to achieve the purpose of fault tolerance. The repeated data deleting technology is used for reducing repeated data which is not used for disaster tolerance in the storage cluster, so that the space occupied by the repeated data is released, and the storage space utilization rate in the storage cluster is improved.

Based on the above-mentioned performing the storage operation on the first data using the erasure coding policy, the performing, by the embodiment 104", the deduplication operation on the first data when the verification result indicates that there is second data stored in the storage cluster and that the second data is duplicated with the first data may include:

1041. and when the storage operation of the first data is not completed, stopping the storage operation.

After the storage operation is blocked, whether the local stored data needs to be subjected to the deduplication operation or not is determined according to the blocking time.

For example, the first data added to the set to be encoded may be deleted and accumulated when blocked prior to step 1024.

1042. And deleting the first data stored in the first storage node when the storage operation of the first data is completed.

There is also a substantial case where a portion of the erasure code band stored locally, if not containing the first data, may not be subject to a deduplication operation; if first data is contained, the stored first data may be deleted and stored using a pointer to second data in place of the first data.

Because the data duplication technology only saves the changed data and uses the pointer to save the duplicated data, the data storage capacity is reduced, and the user does not need to purchase a large number of magnetic disks as before, thereby reducing the total cost of the user backup system to the minimum. The smaller number of disks also results in a smaller storage cluster size and smaller power and refrigeration requirements, while also reducing the complexity of the storage cluster.

According to the technical scheme provided by the embodiment, the repeated data in the storage nodes are eliminated with low delay in the first stage, in order to ensure the performance of the storage cluster, the second stage can be executed in a non-blocking mode, and whether the second data which is repeated with the first data are stored in other storage nodes in the storage cluster is calibrated; and when the verification result shows that the storage cluster has storage, the first storage node executes the duplicate removal operation for the first data. Therefore, the technical scheme provided by the embodiment of the application not only realizes the aim of repeating data deletion, but also has little influence on the performance of the storage cluster, thereby being beneficial to improving the efficiency of the storage cluster in the infrastructure such as calculation, storage and network.

Fig. 7 is a schematic flow chart of a data processing method in a storage cluster according to another embodiment of the present application. As shown in fig. 7, the method provided in this embodiment is applicable to a second storage node in a storage cluster. The second storage node may be the master node mentioned above, more particularly one of the distributed master nodes. Specifically, the method comprises the following steps:

201. receiving a determination result of a first storage node in the storage cluster for first data transmission, wherein the determination result is transmitted after the first storage node determines whether second data which is repeated with the first data is stored on the first storage node;

202. When the determination result is that the second data is not stored, checking whether the second data is stored in the storage nodes except the first storage node in the storage cluster;

203. when the verification result is that the storage exists, feeding back corresponding response information to the first storage node;

In an implementation solution, the step 202 "checking whether the second data is stored in the storage nodes other than the first storage node in the storage cluster" may be implemented by the following steps:

2021. acquiring an identification value of the first data;

2022. checking whether the identification value is in a table range corresponding to the first sub-table; wherein the first sub-table is stored on the second storage node;

2023. when the table range corresponding to the first sub-table is within, checking whether the second data which is repeated with the first data is stored according to the identification value and the table item in the first sub-table;

2024. And when the first sub-table is not in the range of the first sub-table correspondence table, determining a target node in the storage cluster according to the identification value, sending a verification request for the first data to the target node, and waiting for the target node to verify whether a verification result of the second data which is repeated with the first data is stored.

More specifically, if the storage cluster includes a plurality of master nodes, the second storage node is a master node; the identification values of the data stored on all storage nodes in the storage cluster are recorded in a full-scale mode; the full-quantity table is divided into a plurality of sub-tables which are matched with the number of the main nodes; the plurality of sub-tables are free of intersections; the plurality of sub-tables are stored on different main nodes in a scattered manner; one sub-table corresponds to one table range. Accordingly, the determining a target node in the storage cluster according to the identification value in the step 2024 may include:

s31, obtaining table ranges corresponding to sub-tables stored by a plurality of main nodes;

s32, determining the main node storing the table range corresponding to the sub-table in which the identification value is located as the target node.

Further, the method provided in this embodiment may further include the following steps:

204. And when the determination result is that the storage exists, the first data is registered as repeated data by utilizing the identification value of the first data.

What needs to be explained here is: in a specific implementation, the identification value of the first data may be specifically a hash value of the first data, and the first data may be an entire file or may be a data block segmented from the entire file, which is not specifically limited in this embodiment. In addition, for a more detailed description of each step in this embodiment, reference should be made to the corresponding content above, and details are not repeated here.

The function of each storage node to perform the deduplication mentioned in the above embodiments is implemented through the network card (network interface card NIC). A schematic structure of a storage node is shown in fig. 3. In particular, as shown in fig. 3, the storage node comprises: a network card (NIC) and a processor (CPU). The network card is used for receiving first data to be stored and determining whether the storage node stores second data which is repeated with the first data. And the processor is electrically connected with the network card and is used for executing the storage operation of the first data when the determination result is that the first data is not stored. The network card is further configured to send a verification request for the first data to a target node in a storage cluster, so that the target node verifies whether a node except the storage node in the storage cluster stores the second data; and the processor is further configured to execute a deduplication operation for the first data when the verification result received by the network card is stored.

As shown in fig. 6, the network card in this embodiment may be configured with an unloading computing module based on an existing network card, and the unloading computing module performs a process of deleting repeated data. As shown in fig. 6, the technical solution provided for implementing the present embodiment has a network card customized for the function of deduplication processing. After receiving first data to be stored, a network card on a storage node calculates a hash value of the first data by the unloading calculation module, and then checks whether a hash table stored in a memory of the storage node where the network card is positioned contains a value identical to the hash value; and when the hash value which is the same as the hash value of the first data cannot be found in the hash table, performing compression, encryption and other processing on the first data, adding the first data into a to-be-encoded set to perform erasure code encoding (namely EC encoding) together with other data to be stored, wherein part of erasure code bands obtained after encoding is stored locally, and the rest is dispersed to other storage nodes of a storage cluster for storage. The storage node storing the data in the erasure code band does not perform repeated data checking, deleting and other processing, because the data in the erasure code band is used for disaster recovery and cannot be used as repeated data.

Further, if the storage node in the embodiment is the second storage node mentioned above, the network card is further configured to receive a determination result of the first storage node in the storage cluster for sending the third data, where the determination result is sent after the first storage node determines whether the first storage node stores fourth data that is repeated with the third data on the own node; the processor is further configured to check whether a storage node except the first storage node in the storage cluster stores the fourth data when the determination result is that the fourth data is not stored; and the network card is also used for feeding back corresponding response information to the first storage node when the verification result is stored. The response information is used for preventing the first storage node from storing the third data on the own node or triggering the first storage node to delete the third data stored in the first storage node.

Another embodiment of the present application provides a storage node. The storage node may be the second storage node mentioned above. Also, the storage node includes a network card and a processor. The network card is used for receiving a determination result sent by a first storage node in the storage cluster aiming at the first data, wherein the determination result is sent after the first storage node determines whether second data which is repeated with the first data is stored on the first storage node. And the processor is electrically connected with the network card and is used for applying to check whether the second data are stored in the storage nodes except the first storage node in the storage cluster when the determined result is not stored. And the network card is also used for feeding back corresponding response information to the first storage node when the verification result is stored. The response information is used for preventing the first data from being stored on the self node by the first storage node or triggering the first storage node to delete the first data stored on the first storage node.

The network card in this embodiment may have the functions shown in fig. 6 in addition to the above functions, and for details, reference is made to the above description.

Referring to fig. 3, the storage node includes a memory, etc. in addition to a network card and a processor. Wherein the specific form of the layered storage drive in fig. 3 may be software, a program, an engine, etc. When the processor performs a data storage operation, a tiered storage driver may be invoked to implement the storage operation. The network card in fig. 3 may be a NIC supporting a hardware queue, the network card distributing received data to a plurality of receive queues, the receive queues being associated with a CPU. The NIC distributes the received data to the CPU based on a hash value (hash) and an indirection table (PM).

What needs to be explained here is: in this embodiment, only a part of the functions of each storage node are described, and for a more specific implementation manner of each function, reference is made to the corresponding content above; in addition, each storage node in this embodiment may also implement functions not mentioned in this embodiment, which are mentioned above, and specifically refer to the above, which are not described herein.

The application also provides a data processing device. As shown in fig. 8, the data processing apparatus includes: a determining module 41, a storing module 42, an applying module 43 and a deduplication module 44. Wherein the determining module 41 is configured to determine whether the first storage node already stores second data that is repeated with the first data. The storage module 42 is configured to perform a storage operation on the first data when the determination result of the determination module is that the first data is not stored. The application module 43 is configured to apply for verifying whether the second data is stored in a storage node other than the first storage node in the storage cluster. The deduplication module 44 is configured to perform a deduplication operation with respect to the first data when the verification result is that there is a store.

Further, the deduplication module 44 is configured to prevent the storage operation of the first data when the storage operation is not completed; and deleting the first data stored in the first storage node when the storage operation of the first data is completed.

Further, the storage cluster comprises a second storage node; correspondingly, when the application module 43 applies for verifying whether the second data is stored in a storage node other than the first storage node in the storage cluster, the application module is specifically configured to:

acquiring an identification value of the first data;

based on the identification value, sending a verification request to the second storage node to verify by the second storage node whether the second data is stored in the storage nodes except the first storage node in the storage cluster;

and receiving a verification result fed back by the second storage node for the first data.

Further, the storage module 42 is specifically configured to, when performing the storage operation on the first data: when the determined result is not stored, preprocessing the first data; taking the preprocessed first data as data to be encoded, and adding the data to be encoded into a set to be encoded; when the quantity of the data in the set to be encoded meets the preset requirement, erasure code encoding is carried out on the plurality of data in the set to be encoded to obtain a plurality of erasure code blocks; wherein, the plurality of data in the set to be encoded and the plurality of erasure code blocks form an erasure code band; and storing a part of the erasure code bands in the first storage node, and sending the rest of the erasure code bands to one or more storage nodes except the first storage node in the storage cluster for storage.

Further, the storage cluster includes a second storage node. Correspondingly, the data processing device of the embodiment may further include a sending module. The sending module is configured to send the identification value of the first data and the determination result to the second storage node, so that when the determination result is that the first storage node stores the second data, the second storage node registers the first data as duplicate data by using the identification value.

What needs to be explained here is: the data processing device provided in the foregoing embodiments may implement the technical solutions described in the foregoing method embodiments, and the specific implementation principles of the foregoing modules or units may refer to corresponding contents in the foregoing method embodiments, which are not repeated herein.

As shown in fig. 9, another embodiment of the present application provides a data processing apparatus. The data processing apparatus includes: a receiving module 51, a verification module 52 and a feedback module 53. The receiving module 51 is configured to receive a determination result of a first storage node in the storage cluster for sending first data, where the determination result is sent by the first storage node after determining whether a second data that is repeated with the first data is stored on the first storage node. The verification module 52 is configured to verify whether the second data is stored in a storage node other than the first storage node in the storage cluster when the determination result is that the second data is not stored. The feedback module 53 is configured to feed back corresponding response information to the first storage node when the verification result is stored. The response information is used for preventing the first data from being stored on the first storage node by the first storage node or triggering the first storage node to delete the first data stored in the first storage node.

Further, the verification module 52 is specifically configured to, when verifying whether the second data is stored in a storage node other than the first storage node in the storage cluster:

acquiring an identification value of the first data;

and when the first sub-table is not in the range of the first sub-table correspondence table, determining a target node in the storage cluster according to the identification value, sending a verification request for the first data to the target node, and waiting for the target node to verify whether a verification result of the second data which is repeated with the first data is stored.

Further, the storage cluster comprises a plurality of master nodes, and the second storage node is a master node; the identification values of the data stored on all storage nodes in the storage cluster are recorded in a full-scale mode; the full-quantity table is divided into a plurality of sub-tables which are matched with the number of the main nodes; the plurality of sub-tables are free of intersections; the plurality of sub-tables are stored on different main nodes in a scattered manner; one sub-table corresponds to one table range. Accordingly, the verification module 52 is specifically configured to, when determining a target node in the storage cluster according to the identification value:

Obtaining table ranges corresponding to sub-tables stored by a plurality of main nodes; and determining the main node storing the table range corresponding to the sub-table in which the identification value is located as the target node.

Further, the data processing apparatus provided in this embodiment may further include a registration module. The registration module is used for registering the first data as repeated data by utilizing the identification value of the first data when the determined result is stored.

Fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 10, the electronic device includes: a memory 61 and a processor 62. The memory 61 may be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on an electronic device. The memory 61 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The memory 61 is configured to store one or more computer instructions;

the processor 62 is coupled to the memory 61 and is configured to execute one or more computer instructions stored in the memory 61 to implement the steps in the data processing method provided in the foregoing embodiments.

Further, as shown in fig. 10, the electronic device further includes: communication component 63, power supply component 65, display 66, and other components. Only some of the components are schematically shown in fig. 10, which does not mean that the electronic device only comprises the components shown in fig. 10.

Accordingly, the embodiments of the present application further provide a computer readable storage medium storing a computer program, where the computer program when executed by a computer is capable of implementing the steps or functions in the data processing method in the storage cluster provided in the foregoing embodiments.

Embodiments of the present application also provide a computer program product comprising a computer program which, when executed by a processor, causes the processor to implement the steps or functions in the data processing method in a storage cluster provided in the above embodiments.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present application without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of data processing in a storage cluster, adapted for a first storage node in the storage cluster, the method comprising:

2. The method of claim 1, wherein performing a deduplication operation on the first data comprises:

when the storage operation of the first data is not completed, preventing the storage operation;

and deleting the first data stored in the first storage node when the storage operation of the first data is completed.

3. The method of claim 1, wherein the storage cluster includes a second storage node; and

applying for verifying whether the second data is stored in a storage node except the first storage node in the storage cluster, including:

Acquiring an identification value of the first data;

4. A method according to any one of claims 1 to 3, wherein performing a storage operation for the first data comprises:

when the determined result is not stored, preprocessing the first data;

taking the preprocessed first data as data to be encoded, and adding the data to be encoded into a set to be encoded;

when the quantity of the data in the set to be encoded meets the preset requirement, erasure code encoding is carried out on the plurality of data in the set to be encoded to obtain a plurality of erasure code blocks; wherein, the plurality of data in the set to be encoded and the plurality of erasure code blocks form an erasure code band;

and storing a part of the erasure code bands in the first storage node, and sending the rest of the erasure code bands to one or more storage nodes except the first storage node in the storage cluster for storage.

5. A method according to claim 1 or 2, wherein the storage cluster comprises a second storage node; and

the method further comprises the steps of:

and sending the identification value of the first data and the determination result to the second storage node, so that when the determination result is that the first storage node stores the second data, the second storage node registers the first data as repeated data by using the identification value.

6. A method of data processing in a storage cluster, adapted for use with a second storage node in the storage cluster, the method comprising:

when the determination result is that the second data is not stored, checking whether the second data is stored in the storage nodes except the first storage node in the storage cluster;

7. The method of claim 6, wherein verifying whether the second data is stored by a storage node of the storage cluster other than the first storage node comprises:

acquiring an identification value of the first data;

8. The method of claim 7, wherein the storage cluster includes a plurality of master nodes, and the second storage node is a master node; the identification values of the data stored on all storage nodes in the storage cluster are recorded in a full-scale mode; the full-quantity table is divided into a plurality of sub-tables which are matched with the number of the main nodes; the plurality of sub-tables are free of intersections; the plurality of sub-tables are stored on different main nodes in a scattered manner; one sub-table corresponds to one table range; and

According to the identification value, determining a target node in the storage cluster comprises:

obtaining table ranges corresponding to sub-tables stored by a plurality of main nodes;

and determining the main node storing the table range corresponding to the sub-table in which the identification value is located as the target node.

9. The method according to any one of claims 6 to 8, further comprising:

and when the determination result is that the storage exists, the first data is registered as repeated data by utilizing the identification value of the first data.

10. A storage cluster comprising a plurality of storage nodes, the plurality of storage nodes being connected by a network; the plurality of storage nodes comprise a first storage node and a second storage node; wherein,,

11. The storage cluster of claim 10, wherein the plurality of storage nodes further comprises a plurality of master nodes, the second storage node being a master node;

the identification values of the data stored on all storage nodes in the storage cluster are recorded in a full-scale mode; the full-quantity table is divided into a plurality of sub-tables which are matched with the number of the main nodes; the plurality of sub-tables are free of intersections; the plurality of sub-tables are stored on different main nodes in a scattered manner; one sub-table corresponds to one table range;

the second storage node is specifically configured to, when verifying whether the second data is stored in a storage node on the storage cluster other than the first storage node:

acquiring an identification value of the first data;

When the identification value is not in the first sub-table corresponding table range, table ranges corresponding to sub-tables stored by a plurality of main nodes are acquired, and the main node storing the sub-table corresponding to the table range where the identification value is located is determined to be a target node; and sending a verification request for the first data to the target node, and waiting for the target node to verify whether a verification result of the second data which is repeated with the first data is stored.

12. A storage node, comprising:

13. A storage node, comprising:

the processor is electrically connected with the network card and is used for checking whether the second data are stored in the storage nodes except the first storage node in the storage cluster when the determined result is not stored;

14. An electronic device, comprising a processor and a memory; wherein the memory is configured to store one or more computer instructions which, when executed by the processor, enable the steps in the data processing method of any one of the preceding claims 1 to 5 or the steps in the data processing method of any one of the claims 6 to 9.