WO2022196112A1

WO2022196112A1 - Storage system, data processing method, and data processing program

Info

Publication number: WO2022196112A1
Application number: PCT/JP2022/002645
Authority: WO
Inventors: 豊大石; 理貴近藤; 優子宇野; 美咲大塚
Original assignee: 富士フイルム株式会社
Priority date: 2021-03-16
Filing date: 2022-01-25
Publication date: 2022-09-22
Also published as: JPWO2022196112A1

Abstract

This storage system includes a plurality of storage nodes each comprising at least one processor and a storage device. The processors of the plurality of storage nodes receive objects that are identical to each other. Upon receiving an update object which is an update of an existing object stored in the storage device of an own node, each of the processors performs an updating process on the object on the basis of the update object. A processor of at least one storage node of the plurality of storage nodes performs, when an inconsistency has occurred with respect to another storage node in the updating process, the updating process again so as to bring consistency with respect to the other storage node in the updating process.

Description

Storage system, data processing method, and data processing program

The technology disclosed relates to a storage system, a data processing method, and a data processing program.

The following technologies are known as technologies for saving the same data in multiple storage nodes with redundancy. For example, Japanese Unexamined Patent Application Publication No. 2015-88109 describes a plurality of storage nodes each storing the same data, and a backup node connected to the plurality of storage nodes and storing backup data that is a copy of the data stored in the storage nodes. In a storage system having and, when the update time point of data to be replicated stored in the storage node is newer than the update time point of the backup data corresponding to the data, the storage node backs up the object. It is described to send to the node.

Japanese Patent Application Laid-Open No. 2012-528402 discloses that in a set of computers capable of redundantly storing object sets, when an update object is identified in the object set, neighbors in the neighbor set that do not store the update object are identified. It describes determining a swarm node and sending an update object to swarm nodes that contain neighbors.

In the storage system, in order to reduce the risk of data loss and service outage in the event of a failure, multiple storage nodes are placed in different geographical locations, and the same storage nodes are installed at each location. Data is stored with redundancy. In such a storage system, it is required to maintain consistency of data stored among a plurality of storage nodes arranged at each site. For example, it is undesirable for data saved in a certain storage node to be saved in another storage node. As a data transfer method in such a storage system, a method of duplicating data to be saved transmitted from a user terminal and transferring the duplicated data to a plurality of storage nodes can be considered.

On the other hand, object update processing in an object storage system that handles data on an object-by-object basis does not overwrite existing objects, but rather stores the updated objects in the storage node as separate objects from existing objects. Here, it is assumed that a plurality of users update the same object at substantially the same timing. In this case, in each of the plurality of storage nodes arranged at each site, the same object is updated twice in succession. Each storage node manages the object received last among updated objects that are sequentially received as the latest object. The order in which updated objects are received may differ among a plurality of storage nodes due to factors such as the degree of network congestion. In this case, what is managed as the latest object will differ between storage nodes. In other words, inconsistencies arise in stored objects among a plurality of storage nodes that should store the same objects.

The disclosed technique has been made in view of the above points, and ensures the consistency of stored objects even when object update processing continues in a plurality of storage nodes that should store the same objects. for the purpose.

A storage system according to the disclosed technology includes a plurality of storage nodes each having at least one processor and storage device. Each processor of a plurality of storage nodes receives the same object from each other, and when it receives an update object which is an update of an existing object stored in the storage device of its own node, updates the object based on the update object. implement. When inconsistency occurs in update processing with other storage nodes, the processor of at least one storage node among the plurality of storage nodes is updated so that the update processing is consistent with the other storage nodes. Retry the process.

When the processor of at least one storage node among the plurality of storage nodes has performed update processing in its own node, it may transmit update information indicating the results of update processing in its own node to other storage nodes. The processors of the other storage nodes that have received the update information determine whether or not an inconsistency has occurred in the update processing in their own node based on the update information, and determine that an inconsistency has occurred in the update processing. If so, the update process may be performed again so as to match the update information.

A processor of at least one storage node among a plurality of storage nodes issues an instruction to re-execute update processing when the result of update processing in another storage node is different from the result of update processing in its own node. It may be sent to a storage node. Processors of other storage nodes that have received the instruction may perform the update process again.

Multiple storage nodes may be located in different geographical locations.

A data processing method according to technology disclosed herein is a data processing method in a storage system that includes a plurality of storage nodes each having at least one processor and a storage device, in which the same object is received from each other and stored in the storage device of the node itself. When an update object that is an update of an existing object stored in the storage node is received, the processor provided in each of the plurality of storage systems executes the update processing of the object based on the update object, and exchanges with other storage nodes. A processor provided in at least one storage system among a plurality of storage systems performs processing for re-executing update processing so that update processing is consistent with other storage nodes when inconsistency occurs in update processing. to execute.

When the data processing program according to the disclosed technique receives the same object as each other and receives an update object that is an update of an existing object stored in the storage device of its own node, it updates the object based on the update object. At least the storage node is provided with a process of re-executing the update process so that the update process is consistent with the other storage nodes when an inconsistency occurs in the update process with the other storage nodes. It is a program executed by one processor.

According to the disclosed technology, it is possible to ensure the consistency of stored objects even when object update processing continues in a plurality of storage nodes that should store the same objects.

1 is a diagram showing an example of a configuration of a storage system according to an embodiment of technology disclosed herein; FIG. It is a figure which shows an example of the hardware configuration of the server based on embodiment of the disclosed technology. FIG. 2 is a diagram showing an example of an object transfer form in a storage system according to an embodiment of technology disclosed herein; FIG. 4 is a diagram illustrating an example of object update processing according to an embodiment of the disclosed technology; 1 is a functional block diagram of a server acting as a master, according to an embodiment of the disclosed technology; FIG. 1 is a functional block diagram of a server functioning as a slave, according to an embodiment of the disclosed technology; FIG. 4 is a flow chart showing an example of the flow of processing performed by executing a data processing program according to an embodiment of technology disclosed herein; FIG. 4 is a diagram illustrating an example of processing for resolving inconsistencies in update processing according to an embodiment of the disclosed technology; FIG. 4 is a functional block diagram of a server functioning as a master, according to another embodiment of the disclosed technology; FIG. 4 is a functional block diagram of a server functioning as a slave according to another embodiment of the disclosed technology; 7 is a flow chart showing an example of the flow of processing performed by executing a data processing program according to another embodiment of technology disclosed herein;

An example of an embodiment of the disclosed technology will be described below with reference to the drawings. In each drawing, the same or equivalent constituent elements and parts are given the same reference numerals, and overlapping descriptions are omitted as appropriate.

[First Embodiment]
FIG. 1 is a diagram showing an example of the configuration of a storage system 1 according to an embodiment of technology disclosed herein. The storage system 1 constitutes an object storage system that handles data in units of objects. An object includes a data body and metadata about the data body. The storage system 1 includes an information processing device 10 and

servers

20A and 20B. The storage system 1 is connected via a network 40 to

user terminals

50A and 50B. The

user terminals

50A and 50B are computers used by users who use the storage system 1, respectively. The storage system 1 saves the object requested to be saved by the

user terminal

50A or 50B. Further, when receiving an object read request from the

user terminal

50A or 50B, the storage system 1 reads the requested object and transmits it to the

user terminal

50A or 50B.

The information processing device 10 functions as a wrapper that mediates data transmission between the

servers

20A and 20B and the

user terminals

50A and 50B. As shown in FIG. 1, the information processing apparatus 10 duplicates an object to be saved transmitted from the

user terminal

50A or 50B, and transmits (transfers) the same data to the

servers

20A and 20B.

The

servers

20A and 20B each constitute a storage node, and are located at different geographical locations. An object requested to be stored by the

user terminal

50A or 50B is duplicated by the information processing apparatus 10, and the same object is redundantly stored in both the

servers

20A and 20B. In this way, by storing the same object redundantly in multiple storage nodes located at each site, even if some storage nodes fail, the object will be lost and the service will be interrupted. Stopping can be avoided. Note that the storage system 1 may include three or more servers (storage nodes) that store the same object.

Object update processing in the storage system 1 does not overwrite and save an existing object, but an object after update (hereinafter referred to as an updated object) is transferred to the storage node as an object separate from the existing object (hereinafter referred to as an existing object). Saved. That is, the updated object is saved on both

servers

20A and 20B along with the existing object. An existing object is an object already stored in the storage system 1, and an updated object is an object to which the same object key as the existing object is given and for which a storage request is made after the existing object.

FIG. 2 is a diagram showing an example of the hardware configuration of the

servers

20A and 20B.

Servers

20A and 20B have the same hardware configuration as each other. The

servers

20A and 20B each include a CPU (Central Processing Unit) 201, a memory 202 and a storage device 203 as temporary storage areas.

Servers

20A and 20B also include a network interface 204 and an external interface 205 that are connected to the network. CPU 201 , memory 202 , storage device 203 , network interface 204 and external interface 205 are connected to bus 206 .

The storage device 203 is realized by a non-volatile storage medium such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory. A data processing program 210 is stored in the storage device 203 . The CPU 201 reads the data processing program 210 from the storage device 203, develops it in the memory 202, and executes it. Note that the CPU 201 is an example of a processor in technology disclosed herein.

As described above, the object update process in the storage system 1 does not overwrite the existing object, but stores the updated object in the storage node as a different object from the existing object. Here, it is assumed that the user of the user terminal 50A and the user of the user terminal 50B update the same existing object at substantially the same timing. In this case, in each of the

servers

20A and 20B located at each base, the same existing object is updated twice in succession.

In FIG. 3, an existing object (Obj_0) is stored in the storage device 203 of each of the

servers

20A and 20B. A case is exemplified in which the update object (Obj_2) is transmitted from the user terminal 50B at approximately the same timing. The

servers

20A and 20B each successively perform update processing by the update object (Obj_1) and update processing by the update object (Obj_2). The storage device 203 of each of the

servers

20A and 20B stores the existing object (Obj_0) and the updated objects (Obj_1) and (Obj_2).

In the

servers

20A and 20B, among the sequentially received update objects (Obj_1) and (Obj_2), the last received update object is managed as the latest object. The order in which the update objects are received may differ between the server 20A and the server 20B due to factors such as the degree of network congestion. In this case, what is managed as the latest object will differ between servers. In other words, inconsistencies arise in stored objects among a plurality of storage nodes that should store the same objects.

In FIG. 4, after the update process is performed by the update object (Obj_1) in the server 20A, the update process is performed by the update object (Obj_2), and the update process is performed by the update object (Obj_2) in the server 20B. After that, the update process is performed by the update object (Obj_1). In this case, the server 20A manages the update object (Obj_2) as the latest object, and the server 20B manages the update object (Obj_1) as the latest object. That is, the state shown in FIG. 4 is a state in which inconsistency has occurred in update processing between the server 20A and the server 20B.

When a user requests to read an object, the latest managed object will be sent to the user terminal. Therefore, when there is an inconsistency as shown in FIG. 4, when a read request is made to the server 20A and when a read request is made to the server 20B, the user terminal is sent as a response to the read request. An event such as the transmitted object being different occurs.

By executing the data processing program 210, the server 20A and the server 20B perform processing for maintaining consistency in update processing. In this process, either server 20A or server 20B functions as a master, and the other functions as a slave. In the following description, an example will be described in which the server 20A functions as a master and the server 20B functions as a slave.

FIG. 5A is a functional block diagram showing an example of the functional configuration of server 20A that functions as a master. A server 20A functioning as a master includes a receiver 21 , an update processor 22 and a transmitter 23 . By executing the data processing program 210 by the CPU 201 of the server 20A, the server 20A functions as a master and functions as the receiving section 21, the update processing section 22 and the transmitting section .

FIG. 5B is a block diagram showing an example of the functional configuration of the server 20B functioning as a slave. The server 20B functioning as a slave includes a receiving section 25, an update processing section 26 and a reprocessing section 27. FIG. By executing the data processing program 210 by the CPU 201 of the server 20B, the server 20B functions as a slave and functions as the receiving section 25, the update processing section 26, and the reprocessing section 27. FIG. The details of each functional block described above will be described below.

The information processing device 10 duplicates the update object transmitted from the

user terminal

50A or 50B, and transmits (transfers) the same update object to the server 20A and server 20B. The receiving unit 21 of the server 20A and the receiving unit 25 of the server 20B each receive the update object transmitted from the information processing device 10 .

The update processing unit 22 of the server 20A and the update processing unit 26 of the server 20B perform object update processing based on the update objects received by the receiving unit 21 and the receiving unit 25, respectively. That is, the

update processing units

22 and 26 store the update object in the storage device 203 of their own node, and manage the update object as the latest object.

The storage devices 203 of the

servers

20A and 20B store existing objects and updated objects with the same object keys as the existing objects. When a user makes an object read request to the storage system 1, an object key assigned to the object to be read is specified. The

server

20A or 20B that has received the read request reads from the storage device 203 the object managed as the latest object among the objects assigned the object key specified in the read request, and transmits it to the user terminal.

The transmission unit 23 of the server 20A functioning as a master transmits update information indicating the result of update processing in the update processing unit 22 of its own node to the server 20B functioning as a slave. The update information may be, for example, identification information of an object managed as the latest object in the server 20A. Object identification information is information for uniquely identifying an object in the storage system 1, such as a UUID (Universally Unique Identifier).

When the reprocessing unit 27 of the server 20B functioning as a slave receives the update information transmitted from the server 20A functioning as a master, based on the update information, the reprocessing unit 27 detects a mismatch between the server 20A and the server 20A regarding the update process in its own node. Determine whether or not it has occurred. For example, if the object managed as the latest object in its own node is different from the object managed as the latest object in the server 20A indicated by the update information, the reprocessing unit 27 determines that the update process is inconsistent. may be determined to have occurred. When the reprocessing unit 27 determines that the update processing is inconsistent, the reprocessing unit 27 performs the update processing again so as to match the update result in the server 20A indicated by the update information. For example, the reprocessing unit 27 manages the same object as the latest object managed in the server 20A as the latest object in its own node.

The actions of the server 20A and the server 20B will be described below. FIG. 6 is a flow chart showing an example of the flow of processing performed by the CPUs 201 of the

servers

20A and 20B executing the data processing program 210. As shown in FIG. The data processing program 210 is executed, for example, when an update object is transmitted from the information processing apparatus 10 .

In step S1, the receiving unit 21 of the server 20A and the receiving unit 25 of the server 20B each receive the update object transmitted from the information processing device 10.

In step S2, the update processing unit 22 of the server 20A and the update processing unit 26 of the server 20B perform object update processing based on the update object received in step S1. That is, the

update processing units

In step S3, the CPUs 201 of the

servers

20A and 20B respectively determine whether their own nodes are masters or slaves. It is assumed that management information indicating a server functioning as a master and a server functioning as a slave is created by an administrator of the storage system 1, and this management information is stored in the storage device 203 of each server. Here, it is assumed in the management information that the server 20A functions as a master and the server 20B functions as a slave. The CPU 201 of the server 20A determines that its own node is the master based on the management information. The CPU 201 of the server 20B determines that its own node is the slave based on the management information.

In step S4, the transmission unit 23 of the server 20A functioning as a master transmits update information indicating the result of the update processing in its own node to the server 20B functioning as a slave.

In step S5, the reprocessing unit 27 of the server 20B functioning as a slave determines whether update information has been received. When determining that the update information has been received, the reprocessing unit 27 determines in step S6 based on the update information whether or not there is an inconsistency with the server 20A regarding the update processing in its own node. When the reprocessing unit 27 determines that the update processing is inconsistent, the reprocessing unit 27 performs the update processing again so as to match the update result in the server 20A indicated by the update information in step S7. For example, the reprocessing unit 27 manages the same object as the latest object managed in the server 20A as the latest object in its own node.

Here, it is assumed that inconsistencies exemplified in FIG. 4 occur in the update processing in the

servers

20A and 20B. That is, the object managed as the latest object in the server 20A is the update object (Obj_2), and the object managed as the latest object in the server 20B is the update object (Obj_1). In this case, the transmitting unit 23 of the server 20A transmits to the server 20B update information indicating that the object managed as the latest object in its own node is the update object (Obj_2). Based on the update information, the reprocessing unit 27 of the server 20B determines that there is an inconsistency with the server 20A regarding the update process in its own node. The reprocessing unit 27 performs the update process again so as to match the update result in the server 20A. By executing the update process again in the server 20B, as shown in FIG. 7, the object managed as the latest object in the server 20B becomes the same updated object (Obj_2) as the server 20A, and the inconsistency is resolved. be done.

As described above, in the storage system 1 according to this embodiment, each of the plurality of storage nodes receives the same object as each other, and receives an update object that is an update of an existing object stored in the storage device of the node itself. If so, the object update process is performed based on the update object. At least one storage node among a plurality of storage nodes performs update processing so that update processing is consistent with other storage nodes when inconsistency occurs in update processing with other storage nodes. Do it again. More specifically, when at least one storage node among the plurality of storage nodes performs update processing in its own node, it transmits update information indicating the result of the update processing in its own node to other storage nodes. Other storage nodes that have received the update information determine whether or not there is an inconsistency in the update processing of their own node based on the update information. Then, update processing is performed again so as to match the update information.

According to the storage system 1 according to the present embodiment, for example, if different update objects are sent from a plurality of user terminals at substantially the same timing, there is a risk that inconsistency will occur in update processing in each storage node. However, if an inconsistency occurs in the update process, the update process is performed again so that the update process is consistent between the storage nodes, and the inconsistency is resolved. That is, according to the storage system 1 according to the present embodiment, it is possible to ensure the consistency of stored objects even when object update processing continues in a plurality of storage nodes that should store the same objects. becomes.

It should be noted that the disclosed technology may be applied to some of the storage nodes included in the storage system 1 to ensure consistency of objects between the part of the storage nodes.

Also, in the present embodiment, the information processing device 10 exists independently, but the user terminal 50 may also serve as the information processing device 10 . That is, the function as a wrapper may be implemented in the user terminal 50 .

[Second embodiment]
FIG. 8A is a functional block diagram showing an example of a functional configuration of a server 20A functioning as a master according to the second embodiment of technology disclosed herein. A server 20A functioning as a master includes a receiving section 21, an update processing section 22 and an instruction section . By executing the data processing program 210 by the CPU 201 of the server 20A, the server 20A functions as a master and functions as the receiving section 21, the update processing section 22 and the instruction section .

FIG. 8B is a block diagram showing an example of a functional configuration of a server 20B functioning as a slave according to the second embodiment of the disclosed technology. The server 20</b>B functioning as a slave includes a receiver 25 , an update processor 26 , a transmitter 29 and a reprocessor 27 . By executing the data processing program 210 by the CPU 201 of the server 20B, the server 20B functions as a slave and functions as the receiving section 25, the update processing section 26, the transmitting section 29, and the reprocessing section 27. FIG. The details of each functional block described above will be described below. Note that the functions of the receiving

units

21 and 25 and the

update processing units

22 and 26 are the same as those of the first embodiment described above, so description thereof will be omitted.

The transmission unit 29 of the server 20B transmits update information indicating the result of update processing in the update processing unit 26 of its own node to the server 20A. The update information may be, for example, identification information of an object managed as the latest object in the server 20B.

Upon receiving the update information transmitted from the server 20B, the instruction unit 28 of the server 20A determines whether or not there is an inconsistency with the server 20B regarding the update processing in its own node based on the update information. . For example, if the object managed as the latest object in its own node is different from the object managed as the latest object in the server 20B indicated by the update information, the instruction unit 28 determines that there is an inconsistency in the update process. It can be determined that this has occurred. When the instruction unit 28 determines that the update process is inconsistent, the instruction unit 28 transmits to the server 20B a reprocessing instruction indicating that the update process should be performed again.

When the reprocessing unit 27 of the server 20B receives the reprocessing instruction transmitted from the server 20A, it performs the update processing again. The reprocessing unit 27 performs, for example, a process of changing an object managed as the latest object in its own node.

When the server 20B performs the update process again, the transmission unit 29 transmits update information indicating the result of the update process again to the server 20A. Upon receiving the update information transmitted from the server 20B, the instruction unit 28 of the server 20A determines whether or not there is any inconsistency with the server 20B regarding the update processing in its own node. If it is determined that there is, a reprocessing instruction is transmitted to the server 20B. The processing in the reprocessing unit 27, the transmission unit 29, and the instruction unit 28 is repeated until the update processing is consistent between the

servers

20A and 20B.

The operation of the server 20A and the server 20B according to the second embodiment of the disclosed technique will be described below. FIG. 9 is a flow chart showing an example of the flow of processing executed by the CPUs 201 of the

servers

In step S11, the receiving unit 21 of the server 20A and the receiving unit 25 of the server 20B each receive the update object transmitted from the information processing device 10.

In step S12, the update processing unit 22 of the server 20A and the update processing unit 26 of the server 20B perform object update processing based on the update object received in step S11. That is, the

update processing units

In step S13, the CPUs 201 of the

servers

In step S14, the transmission unit 29 of the server 20B functioning as a slave transmits update information indicating the result of the update processing in its own node to the server 20A functioning as a master.

In step S15, the instruction unit 28 of the server 20A functioning as a master determines whether update information has been received. Upon determining that the update information has been received, the instruction unit 28 determines in step S16 based on the update information whether or not there is an inconsistency with the server 20B regarding the update processing in the own node. When the instruction unit 28 determines that there is an inconsistency in the update processing, in step S17, the instruction unit 28 transmits to the server 20B a reprocessing instruction indicating that the update processing should be performed again. After that, the process is returned to step S15. On the other hand, if the instruction unit 28 determines that there is no inconsistency in the update processing, it terminates this routine. In the server 20A, the processes of steps S15, S16 and S17 are repeated until it is determined that there is no inconsistency in the update process among the servers.

In step S18, the reprocessing unit 27 of the server 20B determines whether or not a reprocessing instruction has been received. When the reprocessing unit 27 determines that it has received the reprocessing instruction, it performs the update processing again in step S19. The reprocessing unit 27 performs, for example, a process of changing an object managed as the latest object in its own node. After that, the process is returned to step S14. On the other hand, if the reprocessing unit 27 does not receive a reprocessing instruction within a predetermined period of time after transmitting the update information, the reprocessing unit 27 terminates this routine. In the server 20B, the processes of steps S14, S18 and S19 are repeatedly performed until a reprocessing instruction is not received within a predetermined period.

According to the storage system according to the present embodiment, as in the first embodiment, in a plurality of storage nodes that should store the same object, even if object update processing continues, the consistency of the stored object is ensured. can be secured.

In each of the above embodiments, for example, the receiving

units

21 and 25, the

update processing units

22 and 26, the transmitting

units

23 and 29, the reprocessing unit 27, the instruction unit 28, and the like. As a hardware structure, various processors shown below can be used. As described above, the various processors include, in addition to the CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, a processor such as an FPGA whose circuit configuration can be changed after manufacture. Programmable Logic Device (PLD), ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration specially designed to execute specific processing, such as a dedicated electric circuit.

One processing unit may be composed of one of these various processors, or a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs, a combination of a CPU and an FPGA). combination). Also, a plurality of processing units may be configured by one processor.

As an example of configuring a plurality of processing units with a single processor, first, as represented by computers such as clients and servers, a single processor is configured by combining one or more CPUs and software. There is a form in which a processor functions as multiple processing units. Second, as typified by System on Chip (SoC), etc., there is a form of using a processor that realizes the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip. be. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

Furthermore, as the hardware structure of these various processors, more specifically, an electric circuit combining circuit elements such as semiconductor elements can be used.

Also, in the above embodiment, the data processing program 210 has been pre-stored (installed) in the storage device 203, but the present invention is not limited to this. The data processing program 210 is provided in a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), and a USB (Universal Serial Bus) memory. good too. Also, the data processing program 210 may be downloaded from an external device via a network.

The disclosure of Japanese Patent Application 2021-042932 filed on March 16, 2021 is incorporated herein by reference in its entirety. In addition, all publications, patent applications and technical standards mentioned herein are to the same extent as if each individual publication, patent application and technical standard were specifically and individually noted to be incorporated by reference. , incorporated herein by reference.

Claims

A storage system comprising a plurality of storage nodes each comprising at least one processor and storage device,
each processor of the plurality of storage nodes,
receive the same object as each other,
When receiving an update object that is an update of an existing object stored in the storage device of the node, updating the object based on the update object,
The processor of at least one storage node among the plurality of storage nodes makes the update process consistent with the other storage nodes when the update process is inconsistent with the other storage nodes. A storage system that performs the update processing again as described above.
a processor of at least one storage node among the plurality of storage nodes, when performing update processing in the own node, transmits update information indicating a result of the update processing in the own node to other storage nodes;
The processor of the other storage node that received the update information determines whether or not the update processing in its own node is inconsistent based on the update information. 2. The storage system according to claim 1, wherein when it is determined that the update information is correct, the update processing is performed again so as to match the update information.
The processor of at least one storage node among the plurality of storage nodes issues an instruction to re-execute update processing when the result of update processing in another storage node is different from the result of update processing in its own node. to the storage node of
2. The storage system according to claim 1, wherein processors of other storage nodes that have received said instruction perform said update processing again.
4. The storage system according to any one of claims 1 to 3, wherein said plurality of storage nodes are located at different geographical locations.
A data processing method in a storage system comprising a plurality of storage nodes each comprising at least one processor and a storage device, comprising:
When receiving the same object as each other and receiving an update object that is an update of an existing object stored in a storage device of the self node, each of the plurality of storage systems updates the object based on the update object. A processor comprising
When inconsistency occurs in the update processing with other storage nodes, the plurality of storage systems perform processing for re-executing the update processing so that the update processing is consistent with the other storage nodes. A data processing method executed by a processor provided in at least one storage system of
receive the same object as each other,
When receiving an update object that is an update of an existing object stored in the storage device of the node, updating the object based on the update object,
At least the storage node comprises a process of re-executing the update process so that the update process is consistent with the other storage node when an inconsistency occurs in the update process with another storage node. A data processing program executed by a single processor.