CN107102918B - A data processing method and device - Google Patents

Abstract

The invention discloses a data processing method and a device, wherein the method comprises the following steps: after the data flow of the first data reaches a forwarding layer, judging whether the node is a destination node of the data flow of the first data; the destination node comprises a node to which the first storage unit belongs and a cooperative node of the node to which the first storage unit belongs; the first storage unit is a writing unit of the first data; if not, determining a destination node of the data stream of the first data, forwarding the data stream of the first data to the destination node, and sending the data stream of the first data to a cache layer by the destination node; if so, the data stream of the first data is sent to the cache layer, and the consistency of the data is ensured by the invention.

Description

A data processing method and device

technical field

The present invention relates to the technical field of data processing, and more particularly to a data processing method and device.

Background technique

Most of the existing storage systems are dual-node storage systems. When one node fails, another node can be used to cache data to ensure normal business operations.

In recent years, in order to further improve the security of data caching, a multi-node storage system is proposed based on the dual-node storage system. The multi-node storage system still uses the dual-node method for data caching. Specifically, when a certain data is cached, the data stream of the data will be forwarded to any node, thereby causing a technical problem of data inconsistency.

SUMMARY OF THE INVENTION

In view of this, the present invention provides a data processing method and device to solve the technical problem of data inconsistency in the prior art.

To achieve the above object, the present invention provides the following technical solutions:

A data processing method comprising:

After the data stream of the first data reaches the forwarding layer, determine whether the node is the destination node of the data stream of the first data; the destination node includes the node to which the first storage unit belongs and the node to which the first storage unit belongs. A cooperative node; wherein, the first storage unit is a writing unit of the first data;

If not, determine the destination node of the data stream of the first data, forward the data stream of the first data to the destination node, and the destination node will send the data stream of the first data to the cache layer ;

If so, the data stream of the first data is sent to the cache layer.

Preferably, before the judging whether the current node is the destination node of the data stream of the first data, the method further includes:

Determine whether the current node is a node in the node group to which the first storage unit belongs;

If so, trigger the step of judging whether the current node is the destination node of the data stream of the first data;

If not, triggering the step of determining the destination node of the first data stream.

Preferably, after the data stream of the first data is sent to the cache layer, the method further includes:

When the current node is the subordinate node of the first storage unit, mirroring the first data to the cooperative node of the subordinate node;

When the current node is a cooperative node of the node to which the first storage unit belongs, mirroring the first data to the preceding node of the current node;

Wherein, the front node of the current node is the node to which the first storage unit belongs.

Preferably, the determining the destination node of the first data stream includes:

A node with a large data processing capability is selected from the affiliated node and the cooperative nodes of the affiliated node as the destination node.

Preferably, selecting the node with large data processing capability from the affiliated node and the cooperative node of the affiliated node as the destination node, including:

Determine the node that performed data processing last time from the affiliated node and the cooperative nodes of the affiliated node;

Another node other than the node that performed data processing last time is determined as the destination node.

A data processing device, comprising:

The first judgment unit is used to judge whether the current node is the destination node of the data flow of the first data after the data flow of the first data reaches the forwarding layer; the destination node includes the node to which the first storage unit belongs and the first data flow. A cooperative node of a node to which a storage unit belongs; wherein the first storage unit is a writing unit of the first data;

determining a forwarding unit, configured to determine the destination node of the data stream of the first data when the current node is not the destination node of the data stream of the first data, and forward the data stream of the first data to the destination node, the destination node sends the data stream of the first data to the cache layer;

A first sending unit, configured to send the data stream of the first data to the cache layer when the local node is the destination node of the data stream of the first data.

Preferably, it also includes:

a second judging unit, configured to judge whether the current node is a node in the node group to which the first storage unit belongs after the data stream of the first data reaches the forwarding layer;

a first triggering unit, configured to trigger the first judgment unit to perform an operation of judging whether the current node is the destination node of the data stream of the first data when the current node is a node in the node group to which the first storage unit belongs;

The second triggering unit is configured to trigger the determining and forwarding unit to perform an operation of determining the destination node of the first data stream when the current node is not a node in the node group to which the first storage unit belongs.

Preferably, it also includes:

a first mirroring unit, configured to mirror the first data to a cooperative node of the affiliated node when the current node is the affiliated node of the first storage unit;

a second mirroring unit, configured to mirror the first data to the preceding node of the node when the node is a cooperative node of the node to which the first storage unit belongs;

Wherein, the front node of the current node is the node to which the first storage unit belongs.

Preferably, the determining the forwarding unit includes:

a first determining module, configured to select a node with a large data processing capability from the affiliated node and the cooperative node of the affiliated node as the destination node when the current node is not the destination node of the data stream of the first data;

The first forwarding module is configured to forward the data stream of the first data to the destination node, and the destination node sends the data stream of the first data to the cache layer.

Preferably, the first determining module is specifically configured to determine the node that performed data processing last time from the affiliated node and the cooperative nodes of the affiliated node, so as to determine another node other than the node that performed data processing last time node is the destination node.

It can be seen from the above technical solutions that, compared with the prior art, the present invention provides a data processing method. Specifically, after the data stream of the first data reaches the forwarding layer, it is determined whether the node is the data of the first data. The destination node of the flow, the destination node includes the node to which the first storage unit belongs and the cooperative node of the node to which the first storage unit belongs; wherein, the first storage unit is the writing unit of the first data; if not, determine The destination node that outputs the data stream of the first data, forwards the data stream of the first data to the destination node, and finally the destination node sends the data stream of the first data to the cache layer. It can be seen that the data stream of the first data in the present application will be forwarded to the destination node, and the destination node will cache it to the cache layer, which ensures the consistency of the data.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without creative work.

FIG. 1 is a schematic structural diagram of a multi-node storage system disclosed in the present invention;

2 is a schematic flowchart of a data processing method disclosed in Embodiment 1 of the present invention;

3 is a schematic flowchart of a data processing method disclosed in Embodiment 2 of the present invention;

4 is a schematic flowchart of a data processing method disclosed in Embodiment 3 of the present invention;

5 is a schematic structural diagram of a data processing apparatus disclosed in Embodiment 4 of the present invention;

6 is a schematic structural diagram of a data processing apparatus disclosed in Embodiment 5 of the present invention;

FIG. 7 is a schematic structural diagram of a data processing apparatus disclosed in Embodiment 6 of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Example 1

Embodiment 1 of the present invention discloses a data processing method, which is applied to a multi-node storage system. Specifically, FIG. 1 shows a schematic structural diagram of a multi-node storage system disclosed in the present invention, as shown in FIG. 1 . As shown, the client Host is connected to N nodes, and the N nodes form a node group, wherein one node is the cooperative node of the previous node adjacent to it, and the predecessor node of the next node adjacent to it. That is to say, there is a unilateral cooperative relationship between two adjacent nodes, not a mutual cooperative relationship. That is, Node1 is the cooperative node of Node0, and is the pre-node of Node2; Node2 is the cooperative node of Node1, and is the pre-node of Node3, and so on.

It should be noted that the multi-node storage system includes at least one node group, and FIG. 1 takes the client corresponding to one node group as an example. Of course, the client may correspond to multiple node groups, which is not limited in the present invention. And each node group includes at least two nodes.

Figure 2 shows a schematic flowchart of a data processing method, as shown in Figure 2, the method includes the following steps:

Step 201: when the data stream of the first data reaches the forwarding layer, determine whether the current node is the destination node of the data stream of the first data; if not, go to step 202; if so, go to step 203;

The present node refers to the node that receives the data stream of the first data on its forwarding layer.

The destination node includes a node to which the first storage unit belongs and a cooperative node of the node to which the first storage unit belongs.

The first storage unit is a writing unit of the first data. Specifically, the first storage unit is a LUN (Logical Unit Number, logical unit number). A LUN may be composed of one disk, several disks, or may be composed of Partial space composition on a disk, etc.

The node to which the first storage unit belongs is a node that owns the first storage unit, and the node is a designated node of the first storage unit, and is used for writing data to be written into the first storage unit into the first storage unit. The cooperative node of the node to which the first storage unit belongs is a node adjacent to the node to which the first storage unit belongs. From the foregoing description of the multi-node storage system, it can be known that there is a relationship between the node to which the first storage unit belongs and the cooperative node of the node to which it belongs. In a unidirectional cooperative relationship, the node to which the first storage unit belongs is a preceding node of the cooperative node of the node to which the first storage unit belongs.

After the data stream of the first data delivered by the client reaches the forwarding layer of a node, the node will determine whether the node is the destination node of the data stream of the first data.

Specifically, each node in the multi-node storage system stores the node index of the node and the bitmap information of all nodes. Therefore, based on the node index of the node and the bitmap information of all nodes, it can be determined whether the node is the first node. The destination node of a data stream. For example, the bitmap information is a 32-bit integer value, that is, for nodes 0-31. The node index of this node is the Nth bit, then, if the Nth bit of the bitmap information is 1, it means that the data stream of the first storage unit received on the node N needs to be forwarded, that is, the node N is not the data stream of the first data. For the destination node, if the Nth bit of the bitmap information is set to 0, it means that the data stream of the first storage unit received on node N does not need to be forwarded, that is, node N is the destination node of the data stream of the first data. That is to say, in the bitmap information, except for the bit of the node to which the first storage unit of which the data stream of the first data belongs and the position 0 of the cooperative node, all other bits are set to 1.

Step 202: Determine the destination node of the data stream of the first data, forward the data stream of the first data to the destination node, and the destination node sends the data stream of the first data to the cache layer;

When it is determined that the current node is not the destination node of the data stream of the first data, a node may be selected from the node to which the first storage unit belongs and the cooperative node of the node to which the first storage unit belongs, as the destination node, so that the first data The data stream of the first data is forwarded to the forwarding layer of the destination node, and the destination node sends the data stream of the first data to the cache layer of the destination node.

Step 203: Send the data stream of the first data to the cache layer.

When it is determined that the node is the destination node of the data stream of the first data, the data stream of the first data is directly sent by the node to the cache layer of the node. It should be noted that there may be other data processing layers between the forwarding layer and the cache layer of the first node, but the data stream of the first data will eventually be sent to the cache layer, that is, the data stream of the first data Sending to the cache layer does not necessarily mean that it is directly delivered to the cache layer by the forwarding layer, but can refer to being sent by the forwarding layer to other data processing layers and finally sent to the cache layer after data processing.

In this embodiment, after the data stream of the first data reaches the forwarding layer, it is determined whether the current node is the destination node of the data stream of the first data, and the destination node includes the node to which the first storage unit belongs and the node to which the first storage unit belongs. The cooperative node of the node; wherein, the first storage unit is the writing unit of the first data; if not, the destination node of the data stream of the first data is determined, and the data stream of the first data is forwarded to the destination node , and finally the destination node sends the data stream of the first data to the cache layer. It can be seen that the data stream of the first data in the present application will be forwarded to the destination node, and the destination node will cache it to the cache layer, which ensures the consistency of the data.

Embodiment 2

The second embodiment of the present invention also discloses a data processing method, as shown in FIG. 3 , the method includes the following steps:

Step 301: when the data stream of the first data reaches the forwarding layer, determine whether the current node is a node in the node group to which the first storage unit belongs; if so, go to step 302; if not, go to step 303;

The first storage unit is a writing unit of the first data, and has a node to which the first storage unit belongs. In a multi-node storage system, each node belongs to a node group. Correspondingly, the node group to which the first storage unit belongs is a node group having the node to which the first storage unit belongs.

Specifically, each node in the multi-node storage system stores the node index of the node and the bitmap information of all nodes. Therefore, based on the node index of the node and the bitmap information of all nodes, it can be determined whether the node is the first node. A node in a node group to which a storage unit belongs.

Step 302: determine whether the current node is the destination node of the data stream of the first data; if not, go to step 303, if yes, go to step 304;

Step 303: Determine the destination node of the data stream of the first data, forward the data stream of the first data to the destination node, and send the data stream of the first data to the cache layer by the destination node ;

As a specific implementation manner, determining the destination node of the first data stream includes: selecting a node with a large data processing capability as the destination node from the affiliated node and the cooperative nodes of the affiliated node.

Specifically, selecting a node with a large data processing capability from the affiliated node and the cooperative nodes of the affiliated node as the destination node, including: determining the node that performed data processing last time from the affiliated node and the cooperative nodes of the affiliated node, so as to determine the last node except the last time. Another node of the data processing node is the destination node.

Wherein, among the node to which the first storage unit belongs and the cooperative node of the node to which it belongs, the node that performs data processing last time is determined, and if the node that performs data processing last time is the node to which the first storage unit belongs, then the first storage unit is determined. The cooperative node of the node to which the unit belongs is the destination node; if the node that performed data processing last time is the cooperative node of the node to which the first storage unit belongs, then the node to which the first storage unit belongs is determined to be the destination node.

Step 304: Send the data stream of the first data to the cache layer.

In this embodiment, after the data stream of the first data reaches the forwarding layer, it is determined whether the current node is a node in the node group to which the first storage unit belongs. If the current node is a node in the node group to which the first storage unit belongs, then Then it is necessary to judge whether the current node is the destination node of the data stream of the first data. If the current node is not a node in the node group to which the first storage unit belongs, then the destination node of the data stream of the first data can be directly determined, thereby The cache layer to which the data stream of the first data is sent by the destination node is implemented, and the consistency of the data is ensured.

Embodiment 3

The third embodiment of the present invention also discloses a data processing method, as shown in FIG. 4 , the method includes the following steps:

Step 401: when the data stream of the first data reaches the forwarding layer, determine whether the current node is the destination node of the data stream of the first data; if not, go to step 402; if so, go to step 403;

The destination node includes a node to which the first storage unit belongs and a cooperative node of the node to which the first storage unit belongs.

The first storage unit is a writing unit of the first data.

Step 402: Determine the destination node of the data stream of the first data, forward the data stream of the first data to the destination node, and the destination node sends the data stream of the first data to the cache layer;

When it is determined that the current node is not the destination node of the data stream of the first data, a node may be selected from the node to which the first storage unit belongs and the cooperative node of the node to which the first storage unit belongs, as the destination node, so that the first data The data stream of the first data is forwarded to the forwarding layer of the destination node, and the destination node sends the data stream of the first data to the cache layer of the destination node.

Step 403: Send the data stream of the first data to the cache layer;

Step 404: When the current node is a node to which the first storage unit belongs, mirror the first data to a cooperative node of the node to which it belongs;

Step 405: When the current node is a cooperative node of the node to which the first storage unit belongs, mirror the first data to the preceding node of the current node.

Wherein, the front node of this node is the node to which the first storage unit belongs. The node to which the first storage unit belongs and the cooperative node of the node to which the first storage unit belongs are two adjacent nodes, and the node to which the first storage unit belongs is a pre-node of the cooperative node of the node to which the first storage unit belongs, When the current node is a cooperative node of the node to which the first storage unit belongs, after mirroring the first data to the preceding node of the node, it is equivalent to mirroring the first data to the node to which the first storage unit belongs.

In this embodiment, after the data stream reaches the cache layer, when the current node is the node to which the first storage unit belongs, the first data is mirrored to the cooperative node of the node to which it belongs. When the first storage unit is a cooperative node of the node to which the first storage unit belongs, the first data is mirrored to the front node of the node, so as to ensure that when one node fails, the data can be cached by another node. Moreover, since the first data is stored in the node to which the first storage unit belongs and the cooperative node of the node to which it belongs, it can be ensured that the data delivered to the first storage unit through the node to which the first storage unit belongs is the latest data.

Corresponding to the above-mentioned data processing method, the present invention also provides a data processing apparatus, which is applied in a multi-node storage system. Specifically, it is described in detail by the following examples:

Embodiment 4

Embodiment 4 of the present invention discloses a data processing apparatus. As shown in FIG. 5 , the apparatus includes: a first judging unit 501, a determining and forwarding unit 502, and a first sending unit 503; wherein:

The first judgment unit 501 is configured to judge whether the current node is the destination node of the data flow of the first data after the data flow of the first data reaches the forwarding layer;

The destination node includes a node to which the first storage unit belongs and a cooperative node of the node to which the first storage unit belongs; wherein the first storage unit is a writing unit of the first data.

The node to which the first storage unit belongs is a node that owns the first storage unit, and the node is a designated node of the first storage unit, and is used for writing data to be written into the first storage unit into the first storage unit. The cooperative node of the node to which the first storage unit belongs is a node adjacent to the node to which the first storage unit belongs. From the foregoing description of the multi-node storage system, it can be known that there is a relationship between the node to which the first storage unit belongs and the cooperative node of the node to which it belongs. In a unidirectional cooperative relationship, the node to which the first storage unit belongs is a predecessor node of the cooperative node of the node to which the first storage unit belongs.

The first determining unit 501 is configured to determine whether the node is the destination node of the data stream of the first data after the data stream of the first data delivered by the client reaches the forwarding layer of a certain node. Specifically, each node in the multi-node storage system stores information related to the first storage unit, and the information includes the node to which the first storage unit belongs and the cooperative nodes of the node to which the first storage unit belongs. Therefore, the first judgment Based on this information, the unit can determine whether the current node is the destination node.

A determining forwarding unit 502, configured to determine the destination node of the data stream of the first data when the current node is not the destination node of the data stream of the first data, and forward the data stream of the first data to the destination node of the data stream of the first data a destination node, where the destination node sends the data stream of the first data to the cache layer;

When it is determined that the current node is not the destination node of the data stream of the first data, it is determined that the forwarding unit may select a node from the node to which the first storage unit belongs and the cooperative node of the node to which the first storage unit belongs, as the destination node, so as to transfer the The data stream of the first data is forwarded to the forwarding layer of the destination node, and the destination node sends the data stream of the first data to the cache layer of the destination node.

Specifically, as an implementation manner, the determining and forwarding unit 502 includes: a first determining module and a first forwarding module. Wherein, the first determination module is configured to select a node with a large data processing capability as the destination node from the affiliated node and the cooperative nodes of the affiliated node when the current node is not the destination node of the data stream of the first data ; a first forwarding module, configured to forward the data stream of the first data to the destination node, and the destination node sends the data stream of the first data to the cache layer.

Wherein, the first determination module is specifically configured to determine the node that performed data processing last time from the node to which it belongs and its cooperative nodes, so as to determine another node other than the node that performed data processing last time for the purpose of determining node. That is to say, if the node that performed data processing last time is the node to which the first storage unit belongs, then the first determination module determines that the cooperative node of the node to which the first storage unit belongs is the destination node; is a cooperative node of the node to which the first storage unit belongs, then the first determining module determines that the node to which the first storage unit belongs is the destination node.

The first sending unit 503 is configured to send the data stream of the first data to the cache layer when the current node is the destination node of the data stream of the first data.

In this embodiment, after the data stream of the first data reaches the forwarding layer, it is determined whether the current node is the destination node of the data stream of the first data, and the destination node includes the node to which the first storage unit belongs and the node to which the first storage unit belongs. The cooperative node of the node; wherein, the first storage unit is the writing unit of the first data; if not, the destination node of the data stream of the first data is determined, and the data stream of the first data is forwarded to the destination node , and finally the destination node sends the data stream of the first data to the cache layer. It can be seen that the data stream of the first data in the present application will be forwarded to the destination node, and the destination node will cache it to the cache layer, which ensures the consistency of the data.

Embodiment 5

Embodiment 5 of the present invention discloses a data processing apparatus. As shown in FIG. 6 , the apparatus includes: a second judging unit 601 , a first triggering unit 602 , a second triggering unit 603 , a first judging unit 604 , and a determining and forwarding unit 605. The first sending unit 606; wherein:

The second judgment unit 601 is configured to judge whether the current node is a node in the node group to which the first storage unit belongs after the data stream of the first data reaches the forwarding layer;

The first storage unit is a writing unit of the first data, and has a node to which the first storage unit belongs. In a multi-node storage system, each node belongs to a node group. Correspondingly, the node group to which the first storage unit belongs is a node group having the node to which the first storage unit belongs.

Specifically, each node in the multi-node storage system stores information related to the first storage unit, the information includes the node to which the first storage unit belongs and the cooperative node of the node to which the first storage unit belongs, and also includes the first storage unit. The identifier of the node group to which a storage unit belongs; each node in the multi-node storage system also stores the identifier of the node group to which it belongs. Therefore, the second judging unit can determine whether the current node is a node in the node group to which the first storage unit belongs by judging whether the identifier of the node group to which the node belongs is consistent with the identifier of the node group to which the first storage unit belongs.

The first triggering unit 602 is configured to trigger the first judging unit 604 to perform an operation of judging whether the current node is the destination node of the data stream of the first data when the current node is a node in the node group to which the first storage unit belongs;

The second triggering unit 603 is configured to trigger the determining forwarding unit 605 to perform the operation of determining the destination node of the first data stream when the current node is not a node in the node group to which the first storage unit belongs;

The first judgment unit 604 is used for judging whether the current node is the destination node of the data flow of the first data;

The destination node includes a node to which the first storage unit belongs and a cooperative node of the node to which the first storage unit belongs; wherein the first storage unit is a writing unit of the first data.

The determining forwarding unit 605 is configured to determine the destination node of the data stream of the first data when the current node is not the destination node of the data stream of the first data, forward the data stream of the first data to the destination node, and the destination node will forward the data stream of the first data to the destination node. A data stream of data is sent to the cache layer;

The first sending unit 606 is configured to send the data stream of the first data to the cache layer when the current node is the destination node of the data stream of the first data.

In this embodiment, after the data stream of the first data reaches the forwarding layer, it is first determined whether the current node is a node in the node group to which the first storage unit belongs. If the current node is a node in the node group to which the first storage unit belongs, Then, it is then judged whether the current node is the destination node of the data stream of the first data. If the current node is not a node in the node group to which the first storage unit belongs, the destination node of the data stream of the first data is directly determined, thereby realizing The cache layer to which the data stream of the first data is sent by the destination node ensures data consistency.

Embodiment 6

Embodiment 6 of the present invention discloses a data processing apparatus. As shown in FIG. 7 , the apparatus includes: a first judging unit 701 , a determining and forwarding unit 702 , a first sending unit 703 , a first mirroring unit 704 and a second mirroring unit 705; of which:

a first judgment unit 701, configured to judge whether the current node is the destination node of the data flow of the first data after the data flow of the first data reaches the forwarding layer;

The destination node includes a node to which the first storage unit belongs and a cooperative node of the node to which the first storage unit belongs; wherein the first storage unit is a writing unit of the first data.

The determining forwarding unit 702 is configured to determine the destination node of the data stream of the first data when the current node is not the destination node of the data stream of the first data, and forward the data stream of the first data to the destination node, and the destination node will forward the data stream of the first data to the destination node. sending the data stream of the first data to the cache layer;

a first sending unit 703, configured to send the data stream of the first data to the cache layer when the current node is the destination node of the data stream of the first data;

a first mirroring unit 704, configured to mirror the first data to a cooperative node of the belonging node when the current node is the belonging node of the first storage unit;

The second mirroring unit 705 is configured to mirror the first data to the preceding node of the present node when the present node is a cooperative node of the node to which the first storage unit belongs;

Wherein, the front node of this node is the node to which the first storage unit belongs. The node to which the first storage unit belongs and the cooperative node of the node to which the first storage unit belongs are two adjacent nodes, and the node to which the first storage unit belongs is a pre-node of the cooperative node of the node to which the first storage unit belongs, When the current node is a cooperative node of the node to which the first storage unit belongs, after mirroring the first data to the preceding node of the node, it is equivalent to mirroring the first data to the node to which the first storage unit belongs.

In this embodiment, after the data stream reaches the cache layer, when the current node is the node to which the first storage unit belongs, the first data is mirrored to the cooperative node of the node to which it belongs. When the first storage unit is a cooperative node of the node to which the first storage unit belongs, the first data is mirrored to the front node of the node, so as to ensure that when one node fails, the data can be cached by another node. In addition, since the first data is stored in the node to which the first storage unit belongs and the cooperative node of the node to which it belongs, it can be ensured that the data delivered to the first storage unit through the node to which the first storage unit belongs is the latest data. The data.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. a data processing method, is characterized in that, is applied in multi-node storage system, and each node in described multi-node storage system stores the node index of this node and the bitmap information of all nodes, including: After the data stream of the first data reaches the forwarding layer, determine whether the current node is the destination node of the data stream of the first data; wherein, determining whether the current node is the destination node of the data stream of the first data includes: Determine whether the current node is the destination node of the data stream of the first data based on the node index of the current node and the bitmap information of all nodes; the destination node includes the node to which the first storage unit belongs and the node to which the first storage unit belongs The cooperative node; wherein, the first storage unit is the writing unit of the first data; If not, determine the destination node of the data stream of the first data, forward the data stream of the first data to the destination node, and the destination node will send the data stream of the first data to the cache layer ; If so, the data stream of the first data is sent to the cache layer. 2. The method according to claim 1, characterized in that before judging whether the current node is the destination node of the data stream of the first data, the method further comprises: Determine whether the current node is a node in the node group to which the first storage unit belongs; If so, trigger the step of judging whether the current node is the destination node of the data stream of the first data; If not, triggering the step of determining the destination node of the data stream of the first data. 3. The method according to claim 1, wherein after the sending the data stream of the first data to the cache layer, the method further comprises: When the current node is the subordinate node of the first storage unit, mirroring the first data to the cooperative node of the subordinate node; When the current node is a cooperative node of the node to which the first storage unit belongs, mirroring the first data to the preceding node of the current node; Wherein, the front node of the current node is the node to which the first storage unit belongs. 4. The method according to claim 1, wherein the determining the destination node of the data flow of the first data comprises: A node with a large data processing capability is selected from the affiliated node and the cooperative nodes of the affiliated node as the destination node. 5. The method according to claim 4, wherein the selecting a node with a large data processing capability from the affiliated node and the cooperative node of the affiliated node is a destination node, comprising: Determine the node that performed data processing last time from the affiliated node and the cooperative nodes of the affiliated node; Another node other than the node that performed data processing last time is determined as the destination node. 6. A data processing device, characterized in that, when applied to a multi-node storage system, each node in the multi-node storage system stores the node index of the node and the bitmap information of all nodes, including: a first judging unit, configured to judge whether the current node is the destination node of the data flow of the first data after the data flow of the first data reaches the forwarding layer; wherein, judge whether the current node is the data of the first data The destination node of the stream includes: judging whether the current node is the destination node of the data stream of the first data based on the node index of the current node and the bitmap information of all nodes; the destination node includes the node to which the first storage unit belongs and the A cooperative node of the node to which the first storage unit belongs; wherein, the first storage unit is a writing unit of the first data; determining a forwarding unit, configured to determine the destination node of the data stream of the first data when the current node is not the destination node of the data stream of the first data, and forward the data stream of the first data to the destination node, the destination node sends the data stream of the first data to the cache layer; A first sending unit, configured to send the data stream of the first data to the cache layer when the local node is the destination node of the data stream of the first data. 7. The apparatus of claim 6, further comprising: a second judging unit, configured to judge whether the current node is a node in the node group to which the first storage unit belongs after the data stream of the first data reaches the forwarding layer; a first triggering unit, configured to trigger the first judgment unit to perform an operation of judging whether the current node is the destination node of the data stream of the first data when the current node is a node in the node group to which the first storage unit belongs; The second triggering unit is configured to trigger the determining and forwarding unit to perform an operation of determining the destination node of the data flow of the first data when the current node is not a node in the node group to which the first storage unit belongs. 8. The apparatus of claim 6, further comprising: a first mirroring unit, configured to mirror the first data to a cooperative node of the affiliated node when the current node is the affiliated node of the first storage unit; a second mirroring unit, configured to mirror the first data to the preceding node of the node when the node is a cooperative node of the node to which the first storage unit belongs; Wherein, the front node of the current node is the node to which the first storage unit belongs. 9. The apparatus according to claim 6, wherein the determining the forwarding unit comprises: a first determining module, configured to select a node with a large data processing capability from the affiliated node and the cooperative node of the affiliated node as the destination node when the current node is not the destination node of the data stream of the first data; The first forwarding module is configured to forward the data stream of the first data to the destination node, and the destination node sends the data stream of the first data to the cache layer. 10 . The apparatus according to claim 9 , wherein the first determining module is specifically configured to determine the node that performed data processing last time from the affiliated node and the cooperative nodes of the affiliated node, so as to determine the removal of the data. 11 . Another node of the node that performed data processing last time is a destination node.

Images