TWI735521B - Method and device for upgrading distributed storage system - Google Patents

Abstract

The embodiments of the present application provide a method and device for upgrading a distributed storage system, which relate to the technical field of distributed computers. The client of this application sends write requests to multiple data servers at the same time for the same data to be written, and then analyzes how many data servers have successfully written the data to be written, and judges whether the number of successful writes is greater than Specify the number, and send the first feedback information or the second feedback information to each data server that writes successfully according to the judgment result. The data server determines whether it is in an upgradeable state or a non-upgradable state according to the received first feedback information or second feedback information. The upgrade control server can choose to notify the data server to perform the upgrade operation in a round-robin manner according to the status of the data server. It can ensure that the response time of the system to the client is low without stopping the upper-layer service, and improve the data The reliability reduces the risk of user data loss.

Description

Method and device for upgrading distributed storage system

This application relates to the field of distributed computer technology, in particular to a distributed storage system upgrade method and a distributed storage system upgrade device.

A highly available distributed storage system is the cornerstone of building highly available services. The distributed storage system consists of distributed data servers, which provide high reliability of data and high availability of access. Among them, high reliability is achieved through data redundancy and multiple backups or erasure codes; high availability is achieved through fast exception handling and failover. To upgrade the version of the distributed storage system, each data server of the system needs to be restarted to complete the version update.

In the prior art, there are several solutions for the upgrade of distributed storage systems:

First, shut down the upper-level services, stop the entire distributed storage system, restart and upgrade all the data servers in the distributed storage system, and restore the upper-level services after all the data servers are upgraded.

However, this kind of solution leads to the overall unavailability of upper-layer services, and the unavailability of its services is very long, which is unacceptable for some services that require high availability.

The second is to keep the upper-level services and restart each data server in a round-robin manner. The client sends write requests to multiple data servers (the number is the number that needs to be backed up by default) to the data server. If the request fails, the client will retry the above-mentioned data server to ensure that the client's access continues.

However, in this solution, if the client waits for the data server to recover and retry, the data server generally restarts and recovers in seconds, and the high real-time data access delay is 10-100MS. The trial strategy will greatly affect the response time of the server.

Third, the upper-level services are not stopped, and each data server is restarted in a round-robin manner. The client sends write requests to multiple data servers (the number is the number that needs to be backed up by default) to the data server. If the request fails to write the data to be written by the server, the client ignores these data servers and retry to ensure that the client's access continues.

However, in this solution, the client temporarily ignores the failed data servers, and the distributed storage system continues the round-robin upgrade of these data servers. If each data server in the distributed storage system restarts too quickly, and if the client writes to only one data server successfully, then only one data server writes successfully. If the disk or machine is damaged, because the client's data to be written is not written to other data servers, the user's data will be lost. Even if the cycle restart time is extended, the large clusters of distributed storage systems will have unexpected disk and machine abnormalities during the cycle for a long time, and the client data will only be successfully written to the data server. Circumstances put user data at great risk of loss. Therefore, although this scheme solves the delay problem, the reliability of the data is insufficient.

In view of the above problems, the embodiments of the present application are proposed to provide a distributed storage system upgrade method and a corresponding distributed storage system upgrade device that overcome the above problems or at least partially solve the above problems.

In order to solve the above problems, this application discloses a distributed storage system upgrade method, applied to the client, including: sending a write request to multiple data servers for the same piece of data to be written; Respond, and determine whether the number of successful writes is greater than the specified number according to the response; if the number of successful writes is greater than the specified number, send the first feedback information to each data server that successfully writes; if write If the number of successful entries is not greater than the specified number, the second feedback information is sent to each data server that has been successfully written; wherein the first feedback information or the second feedback information is used by the data server to determine its own status The status includes an upgradeable status and a non-upgradable status; the status of the data server is for the upgrade control server to notify the data server to perform an upgrade operation in a round-robin manner.

This application also discloses a distributed storage system upgrade method, applied to a data server, including: receiving first feedback information or second feedback information sent by a client; the first feedback information or second feedback information is based on the client After sending a write request to multiple data servers for the same piece of data to be written, the comparison result of the number of successful writes and the specified number is obtained; in the case of not being upgraded, according to the first feedback information or The second feedback information determines the status of the data server itself; the status includes an upgradeable status and an unupgradable status; the status is used by the upgrade control server to select and notify the data server to perform an upgrade operation in a round-robin manner.

The application also discloses a distributed storage system upgrade method, which is applied to upgrade control servers, including: obtaining the status of each data server; the status includes an upgradeable status and an unupgradable status, and each data server has a status The state of the data server is determined according to the first feedback information or the second feedback information; the first feedback information or the second feedback information is sent to multiple data servers based on the client side for the same piece of data to be written After the request is entered, the comparison result of the number of successful writes and the specified number is obtained; at least one data server in an upgradeable state is notified in a round-robin manner to perform the upgrade operation; the data server is upgraded according to the notification operate.

The application also discloses a distributed storage system upgrade device, which is applied to the client, and includes: a request sending module, which is used to send write requests to multiple data servers for the same piece of data to be written; After receiving the response from each data server, and judging whether the number of successful writes is greater than the specified number according to the response; the first feedback module is used for if the number of successful writes is greater than the specified number, then Send the first feedback information to each successfully written data server; the second feedback module is used to send the second feedback to each successfully written data server if the number of successful writes is not greater than the specified number Information; wherein the first feedback information or the second feedback information is used by the data server to determine its own state; the state includes an upgradeable state and a non-upgradable state; the state of the data server is used for the upgrade control server The data server is notified to perform the upgrade operation in a round-robin manner.

The application also discloses a distributed storage system upgrade device, which is applied to a data server, and includes: a feedback information receiving module for receiving first feedback information or second feedback information sent by a client; the first feedback information Or the second feedback information is obtained based on the comparison result of the number of successful writes and the specified number after the client sends a write request to multiple data servers for the same piece of data to be written; In the case of not being upgraded, the state of the data server itself is determined based on the first feedback information or the second feedback information; the state includes an upgradeable state and an unupgradable state; the state is used by the upgrade control server to recycle The data server is selected to notify the data server to perform the upgrade operation in a circular manner.

The application also discloses a distributed storage system upgrade device, which is applied to the upgrade control server, and includes: a status acquisition module for acquiring the status of each data server; the status includes an upgradeable status and a non-upgradable status, each Each data server has a status; the status of the data server is determined according to the first feedback information or the second feedback information; After multiple data servers send write requests, the comparison result of the number of successful writes and the specified number is obtained; the upgrade notification module is used to notify at least one data server that is in an upgradeable state in a round-robin manner. Upgrade operation; the data server performs an upgrade operation according to the notification.

The embodiments of the present application include the following advantages: in the embodiments of the present application, after the distributed storage system starts the upgrade process, for the clients accessing the distributed storage system, each client can simultaneously send multiple messages to multiple clients for the same data to be written. The data server sends the write request for the data to be written, and then analyzes how many data servers have successfully written the data to be written, determines whether the number of successful writes is greater than the specified number, and sends the data to Each successfully written data server sends the first feedback information or the second feedback information. For the data server, according to the received first feedback information or second feedback information, it is determined whether it is in an upgradeable state or a non-upgradable state. For the upgrade control server, the data server can be selected to notify the data server to perform the upgrade operation in a round-robin manner according to the status of the data server. Through the above process, in this embodiment of the application, when the upgrade control server controls each data server in a round-robin manner to upgrade, the client controls the state of the data server and guarantees the data to be written by any client Write at least the specified number of data servers for backup. Therefore, without stopping the upper-layer services, the distributed storage system can not only ensure that the response time of the distributed storage system to the client is low, but also improve the reliability of the data and greatly reduce The risk of user data loss is eliminated.

510‧‧‧Request sending module

520‧‧‧Judgment Module

530‧‧‧First Feedback Module

540‧‧‧Second Feedback Module

610‧‧‧Feedback information receiving module

620‧‧‧Status determination module

710‧‧‧Status Obtaining Module

720‧‧‧Upgrade notification module

810‧‧‧Client

820‧‧‧Data Server

830‧‧‧Upgrade control server

811‧‧‧Request sending module

812‧‧‧Judgment Module

813‧‧‧First Feedback Module

814‧‧‧Second Feedback Module

821‧‧‧Data Storage Module

822‧‧‧Feedback information receiving module

823‧‧‧Status determination module

824‧‧‧Upgrade Module

831‧‧‧Status Obtaining Module

832‧‧‧Upgrade notification module

FIG. 1 is a step flow diagram of an embodiment of a method for upgrading a distributed storage system on the client side of the present application; FIG. 2 is a step flow diagram of an embodiment of a method for upgrading a distributed storage system on the data server side of the present application; 3 is a step flow chart of an embodiment of a distributed storage system upgrade method on the upgrade control server side of this application; FIG. 4 is a step flow chart of an embodiment of a distributed storage system upgrade system of this application; FIG. 5 is this application A structural block diagram of an embodiment of a distributed storage system upgrade device on the client side of the present application; Figure 6 is a structural block diagram of an embodiment of a distributed storage system upgrade device on the client side of the present application; Figure 7 is the data of the present application A structural block diagram of an embodiment of a distributed storage system upgrade device on the server side; FIG. 8 is a structural block diagram of an embodiment of a distributed storage system upgrade device on the upgrade control server side of the present application; FIG. 8A is an implementation of the present application Example of the architecture of a distributed storage system.

In order to make the above objectives, features, and advantages of the application more obvious and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific implementations.

One of the core concepts of the embodiments of the present application is that creative changes are made to the data server and the upgrade control server of the distributed storage system, and new execution logic is also provided for the client accessing the data server. After the distributed storage system starts the upgrade process, for the client accessing the distributed storage system, for the client accessing the distributed storage system, each client is for the same data to be written, and simultaneously sends multiple data The server sends a write request for the data to be written, and then analyzes how many data servers have successfully written the data to be written, judges whether the number of successful writes is greater than the specified number, and writes to each data according to the judgment result The data server that has successfully entered sends the first feedback information or the second feedback information. For the data server, according to the received first feedback information or second feedback information, it is determined whether it is in an upgradeable state or a non-upgradable state. For the upgrade control server, the data server can be selected to notify the data server to perform the upgrade operation in a round-robin manner according to the status of the data server. Through the above process, when the upgrade control server controls the data server in an upgradeable state in a round-robin manner to upgrade, the client controls the state of the data server and ensures that the data to be written by any client is at least written A designated number of data servers are used for backup. Therefore, without stopping the upper-level services, the distributed storage system can not only ensure that the response time of the distributed storage system to the client is low, but also improve the reliability of the data and greatly reduce the user data. The risk of loss; and can tolerate unexpected machine abnormalities during the upgrade process to ensure that the service is not affected.

Example one

The embodiments of this application are applied to the client side.

1, there is shown a step flow chart of an embodiment of a distributed storage system upgrade method of the present application, which may specifically include the following steps: Step 110, for the same piece of data to be written, send write requests to multiple data servers .

In the embodiment of the present application, for a client A, during the upgrade process of the distributed storage system, when the client wants to write data A1 to be written to the data server, the client A will send data to multiple data servers. The device sends a write request for the material A to be written.

In the embodiment of the present application, the number R of multiple data servers may be preset, for example, 10. The specific number R is not limited in the embodiment of the application.

Then, for the data to be written, it can send a write request to the R data server, and then enter step 120.

It can be understood that in practical applications, the client A can first send R write requests to the dispatch server of the distributed storage system, and then the dispatch server controls which R data servers the R write requests are allocated to Device.

It should be noted that different clients send write requests to R data servers for the data to be written, and the respective R data servers may be the same or different.

In another preferred embodiment of the present application, before step 110, steps 101-102 are further included: step 101, when accessing a data server, receive a second upgrade notification sent by the data server.

Step 102: Enter an upgrade preparation state according to the second upgrade notification.

In the embodiment of the present application, the upgrade control server of the distributed storage system first notifies each data server to enter the upgrade preparation state. Then each data server can send a second upgrade notification to the client accessing the data server, so that the client enters the upgrade preparation state.

Correspondingly, for the client A, after it accesses a data server that has entered the upgrade preparation state, the data server returns a second upgrade notification to the client. After receiving the second upgrade preparation notification, the client enters the upgrade preparation state according to the second upgrade preparation notification.

In the embodiment of this application, if the client accesses the data server through the webpage of the browser, this application can transmit an upgrade script to the browser through the webpage opened by the browser. After the browser receives the script, Then you can execute the script to make the client enter the upgrade preparation state. If the client accesses the data server through the Alipay APP, etc., this application can add upgrade processing logic to the above APP in advance. After the APP receives the second upgrade notification, the upgrade processing logic will be activated to enable the client to enter the upgrade Ready state. Of course, the specific manner is not limited by the embodiment of the present application.

When the client enters the upgrade preparation state, it can enter steps 110, 120, 130, 140 and other steps executed by the client in the embodiment of the present application.

Step 120: Receive the response returned by each data server, and determine whether the number of successful writes is greater than the specified number according to the response; if the number of successful writes is greater than the specified number, go to step 130; If the number of successful entries is not greater than the specified number, then step 140 is entered.

In the embodiment of the present application, under normal circumstances, after each data server receives a write request for the same piece of data to be written from the client, it will return a response to the client accordingly. Of course, if a data server is being upgraded or crashed, no response will be sent back to the client.

In the embodiment of the present application, the client can periodically check the received response after sending a write request to the R data servers for the data A1 to be written. For example, periodically check whether a response from the data server or servers is received. If a response from a certain data server or several data servers is not received within a specified time period, it means that the data to be written A1 has not been successfully written into the certain data server or several data servers. If you receive a response from one or several data servers within a specified time period, you can analyze whether the response is a successful write response or a write failed response; if it is a successful write response, the write is successful The data server responds to the success of the backup of the data to be written to A1, that is, the write is successful; if it is a write failure response, it means that the data server corresponding to the write failure response fails to backup the data to be written to A1, that is, the write Entry failed.

It should be noted that, for the responses of R data servers, client A periodically checks whether M is greater than N. The period is for example 1ms (microseconds). Of course, the period can be set as needed, and this application does not impose restrictions on it.

Then, this application can count the number of successful writing of the data A1 to the data server based on the above judgment.

In the embodiment of the present application, a designated number of successful writes can be preset, and the designated number is the lowest number of successful writes. It can also be understood as the number of data servers for which the data to be written is successfully backed up by a client. Number, the designated number, such as N, where N<R, N and R are both positive integers. In practice, N=3 and R=10 can be preset. Of course, the values of N and R can be set according to actual needs, and this application does not limit them.

N，則進入步驟140。其中，M

0，且為整數。 Then, after the client A sends its current data A1 to be written to the R data servers, if there are M data servers returning a write success response, then it is judged whether M>N. If M>N, go to step 130; if M

N, go to step 140. Among them, M

0, and is an integer.

Of course, in practical applications, since the data servers are placed according to racks, the R data servers can be distributed on different racks to further improve the effect of disaster recovery and backup. The optimal situation is that the R data servers are completely distributed on different racks.

Step 130: Send the first feedback information to each data server that has been successfully written.

Since the write request from the client can be successfully received and the data to be written can be backed up, it means that the M data servers are available. Then the client can send the first feedback message to the M data servers. The first feedback message may be an OK message, and the OK message tells the data server that it can set its own state to an upgradeable state.

Of course, in practice, how the data server sets its own state to the upgradeable state according to the first feedback message is described in detail in the second embodiment of this application.

Step 140: Send second feedback information to each data server that has been successfully written.

Wherein, the first feedback information or the second feedback information is used for the data server to determine its own state; the state includes an upgradeable state and a non-upgradable state; the state of the data server is used by the upgrade control server to determine its own status; The data server is notified to perform the upgrade operation in a circular manner.

N，由於這M個資料伺服器是可用的，那麼客戶端可以向該M個資料伺服器發送第二反饋消息。該第二反饋消息可以如HOLD心跳消息，該HOLD消息告訴資料伺服器可以將自身的狀態設置為不可升級狀態。當然，實際中資料伺服器如何根據該第二反饋消息設置自身的狀態為不可升級狀態，本申請在實施例二中進行詳細描述。 If M

N. Since the M data servers are available, the client can send a second feedback message to the M data servers. The second feedback message may be a HOLD heartbeat message, and the HOLD message tells the data server that it can set its own state to a non-upgradeable state. Of course, in practice, how the data server sets its own state to the non-upgradeable state according to the second feedback message is described in detail in the second embodiment of this application.

It should be noted that for the case of M<N, the client re-sends a write request to the new data server until it is guaranteed that M is at least equal to N.

For each unupgraded data service in the distributed storage system, when it receives the first feedback information or the second feedback information, it can determine that it is in an upgradeable state based on the first feedback information or the second feedback information Still in a non-upgradeable state. Among them, the upgradeable state is such as OK, and the non-upgradable state is such as HOLD. For example, if a certain data server receives the second feedback information, the data server can set its own status to a non-upgradeable status. If a certain data server receives the first feedback information, it can set its own status to an upgradeable status.

Of course, in actual applications, since the upper-layer services are not stopped and the data servers are constantly being upgraded, there may also be data servers that have been upgraded among the above R data servers. Then, for the upgraded data server, any of the above feedback information it receives will not be processed further, and its status will remain upgraded.

The specific process of how the data server determines its own state according to the received first feedback information or the second feedback information is described in detail in the second embodiment of this application.

Further, the status of each data server can be notified by the upgrade control server in a round-robin manner to perform the upgrade operation. The specific process of how the upgrade control server notifies the data server to perform the upgrade operation in a round-robin manner is described in the third embodiment of this application.

In another preferred embodiment of the present application, step 140 includes sub-steps 141-144: sub-step 141, if the number of successful writes is equal to the specified number, send to each data server that successfully writes The second feedback information.

For example, the aforementioned client A sends the data to be written A1 to R data servers for the first time, then if the number of successful write responses M=N is monitored, it can directly write to the M data servers that have successfully written Send the second feedback information.

In sub-step 142, if the number of successful writes is less than the specified number, send a write request to at least one data server other than the multiple data servers for the data to be written.

Sub-step 143, receiving the response returned by the at least one data server, and judging whether the current number of successful writes is equal to the specified number based on the number of successful writes. If the current number of successful writes is equal to the specified number, then go to sub-step 144; if the current number of successful writes is less than the specified number, then go to sub-step 142; sub-step 144, if the current number of successful writes is If the number is equal to the specified number, the second feedback information is sent to each data server that has been successfully written.

For sub-steps 142-144, as the aforementioned client A sends the data A1 to be written for the first time to R data servers, then if the number of successful responses M<N is monitored, it will send to at least NM data servers again Write request. Then receive the response of the NM data servers, if the number of successful write responses returned from the NM data servers plus the number of successful write responses of the previous R data servers, M is obtained. If this When M=N, the second feedback information is sent to each data server that has been successfully written. At this time, the value of M changes, but if M<N at this time, the write request is sent to at least NM data servers again, and so on, until M=N, and then the first data server is sent to each successfully written data server. 2. Feedback information.

Further, for example, R=10, N=5, client A sends the data A1 to be written to 10 data servers for the first time, then if the number of successful write responses M=3 is monitored, then at least the data will be sent to 5-3=2 data servers send write requests at a time, continue to monitor the number of successful write responses of the two data servers, and find that only one successful write response is received, then add the previously recorded 3 , Get M=4. Then send write requests to at least 5-4=1 data servers again. Continue to monitor and judge according to the above method, when M=N=5, then send the second feedback information to each data server that has been successfully written. Of course, in practical applications, when M<N, in the above process, after each write request is sent, the data server that writes successfully can directly send the second feedback message, instead of waiting for all of M=N and then send it together. Of course, in practical applications, since the response of the data server is periodically checked, if the specified period T, such as 3*T, it is determined that M<N, then at least NM data servers will be sent again Write request. In each cycle, the judgment of M and N is the same. Under normal circumstances, M=N can be determined in a few cycles, and the response time is generally very short, which will not affect the response delay of the client too much. The response delay of the user to the above situation is also lower than that of waiting for the restart. a lot of.

In this way, it can ensure that the client's to-be-backed-up data is successfully backed up in multiple databases without affecting the normal use of the upload service.

In another preferred embodiment of the present application, after step 140, step 150 is further included: step 150, when new data to be written appears, target multiple data servers including the data server that has been successfully written before ,Go to step 120.

It can be understood that for the case of M=N, since the client sends back HOLD heartbeat information to each successfully written data server, these data servers are set to a non-upgradeable state, such as the HOLD state. In order to more conveniently clear the client's restrictions on these data servers, in the embodiment of the present application, when the client subsequently sends a write request to the new data to be written, it continues to send to the previous R data servers.

For example, client A's data to be written A1, previously sent write requests to the 10 data servers U1, U2...U10, and then judged that M=N, then for client A's pending data Write data A2, you can continue to send write requests to the 10 data servers U1, U2...U10, and continue with the judgment in step 120.

It can be understood that, for the case of sub-step 142, the R data servers include M successfully written, and the remaining R-M data servers are selected from the data servers that failed to write. For example, R=10, N=3, for client A's data A1 to be written, the write request is sent to the 10 data servers U1, U2...U10 for the first time, and only U1, U2 writes The input is successful, and then a write request is sent to U11, U12 for the second time, and U11 is written successfully; then for the client's data to be written A2, you can choose U1, U2, U11, U4, U5...U10 These 10 data servers send write requests to these 10 data servers. Of course, you can also choose to include U1, U2, U11 and other data servers to form 10 data servers.

Then, when the number of successfully written data A2 of client A to be written is M>N, an OK message can be sent to each successfully written data server, and the data server can determine whether to write its own data based on the OK message. The status is changed to an upgradeable status.

In this way, it is possible to write data to be written to the same R data servers as the last time through a client under the condition of M<N, so as to actively update the status of the data server, which can reduce the data server from being unupgradable. The time of the state.

In another preferred embodiment of the present application, the first feedback information and the second feedback information include a client identifier.

For example, for the client A, when sending the first feedback information or the second feedback information to the server, both the first feedback information and the second feedback information include the client identifier "Client A".

Then the first feedback information or the second feedback information for the data server to determine its own state includes: the second feedback information for the data server to send the second feedback information after receiving the second feedback information The client identifier in is written into the non-upgradeable list and marks its own status as the non-upgradeable state; in the embodiment of the present application, the non-upgradeable list is set on the side of the data server. Taking client A as an example, after receiving the second feedback information from client A, the data server sets its own state to a non-upgradeable state, and writes the client A into the non-upgradeable list.

The first feedback information is used by the data server to delete the client identifier in the first feedback information from the unupgradeable list after receiving the first feedback information, and mark its own status after determining that the unupgradable list is empty It is in an upgradeable state.

In the embodiment of the present application, after the client A sends the second feedback information to each successfully written data server, new data to be written appears, so as to include a plurality of data servers written successfully before. Data server is the target. Then, for the data server that receives the second feedback message sent by the client A, it will subsequently receive the message sent by the client A. Then, when the data server continues to receive the message from the client A as the first feedback message, it can delete the record of the client A from the unupgradeable list.

Of course, in actual applications, if the data server receives the first feedback message of a certain client, the corresponding client ID may not be recorded in the unupgradeable list, so there is no need to perform the deletion process.

Then, when the data server determines that the non-upgradeable list is empty, it sets its own status to the upgradeable status.

Of course, in the embodiment of the present application, the client can also receive the exit notification for exiting the upgrade preparation state sent by the data server, and exit the upgrade preparation state according to the exit notification. Then the client sends a write request to the data server according to the normal request sending logic.

In the embodiment of this application, a distributed storage system upgrade method of this application is introduced from the client side, so that each client can simultaneously send the information for the data to be written to multiple data servers for the same data to be written. Write the request, and then analyze how many data servers have been successfully written, determine whether the number of successful writes is greater than the specified number, and send the first feedback information or the first feedback information to each successfully written data server based on the judgment result. 2. Feedback information. The first feedback information is for the data server to delete the client identifier in the first feedback information from the unupgradeable list after receiving the first feedback information, and mark its own after determining that the unupgradable list is empty The status is upgradeable. . Therefore, in the embodiment of the present application, when the upgrade control server controls each data server to upgrade in a round-robin manner, since the client controls the state of the data server, it is ensured that the data to be written by any client is written at least A designated number of data servers are used for backup. Therefore, without stopping the upper-layer services, the distributed storage system can not only ensure that the response time of the distributed storage system to the client is low, but also improve the reliability of the data and greatly reduce the user data. Risk of loss.

Example two

The embodiments of this application are applied to the data server side of a distributed storage server.

Referring to FIG. 2, there is shown a step flow chart of an embodiment of a distributed storage system upgrade method of the present application, which may specifically include the following steps: step 210, receiving first feedback information or second feedback information sent by a client; The first feedback information or the second feedback information is obtained based on the comparison result of the number of successful writes and the specified number after the client sends write requests to multiple data servers for the same piece of data to be written.

With reference to the description on the client side of the first embodiment, after the client sends a write request to R data servers for the data A1 to be written, when the number of successful writes M is greater than the specified number N, it will send a write request to the M data servers. The data server sends the first feedback message; when M is equal to N, the second feedback message is sent to the M data servers. When M<N, send a write request to at least one data server other than the multiple data servers for the data A1 to be written, and then receive a response from the at least one data server, and combine it The number of successful writes determines whether the current number of successful writes M is equal to the specified number N, and if M=N, the second feedback information is sent to the M data servers. Of course, the specific process for the client to return the first feedback information and the second feedback information can refer to the description of the first embodiment, which will not be repeated here.

In contrast, for each data server of the distributed storage system, it may receive the first feedback information returned by each client, such as an OK message. It is also possible to receive a second feedback message returned by each client, such as a HOLD message.

Preferably, before step 210, the method further includes: step 201, receiving the first upgrade notification sent by the upgrade control server.

Step 202: Enter an upgrade preparation state according to the first upgrade notification, and send a second upgrade notification to the client after receiving the client's access request, so that the client enters the upgrade preparation state.

In the embodiment of the present application, the upgrade control server of the distributed storage system first sends a first upgrade notification to each data server, and each corresponding data server receives the first upgrade notification, and then enters the upgrade according to the first upgrade notification. Ready state.

Then, for accessing the data server that enters the upgrade preparation state, after receiving an access request from a certain client, it returns a second upgrade notification to the client. The client enters the upgrade preparation state according to the second upgrade notification.

When the data server enters the upgrade preparation state, it can enter steps 210 and 220 of the embodiment of the present application, which are executed by the data server side.

Step 220, in the case where the self is not upgraded, determine the state of the data server itself according to the first feedback information or the second feedback information; the state includes an upgradeable state and an unupgradable state.

Wherein, the status is for the upgrade control server to select and notify the data server to perform the upgrade operation in a round-robin manner.

In actual applications, except that all data servers have just entered the upgrade preparation state, each data server is in a state of not being upgraded. As some data servers continue to complete the upgrade process, these data servers will be in an upgraded state. For example, a data server that has been successfully upgraded sets its own status to DONE. At this time, the data server that has been successfully upgraded is also It will receive the first feedback information or the second feedback information, but it will no longer process the first feedback information or the second feedback information, and its status will remain upgraded.

Only the data server in the unupgraded state determines the state of the data server itself based on the first feedback information or the second feedback information.

It can be understood that for each unupgraded data service in the distributed cluster, after it receives the first feedback information or the second feedback information, it can determine whether it is in a state of availability based on the first feedback information or the second feedback information. The upgrade state is still in a non-upgradeable state. Among them, the upgradeable state is such as OK, and the non-upgradable state is such as HOLD. For example, if a certain data server receives the second feedback information, the data server can set its own status to a non-upgradeable status. If a certain data server receives the first feedback information, it can set its own status to an upgradeable status.

In another preferred embodiment of the present application, the first feedback information and the second feedback information include a client identifier.

Then based on step 150 of the first embodiment, preferably, step 220 includes sub-steps 221-222: sub-step 221, when the second feedback information is received, the client identifier in the second feedback information is written into Unupgradeable list, and mark its own status as unupgradable; in the embodiment of the present application, an unupgradable list is preset in each data service to record information such as the identification of the client that sent the HOLD message.

For example, for client A, it sends write requests to multiple data servers for the data A1 to be written. When the data servers U1, U2, and U3 are successfully written, and M=N, it will send a write request to the corresponding M The data server sends a HOLD message. The HODL message includes the HOLD instruction and the client identification. Then the M data servers receive the HOLD message, and record the client A in the non-upgradeable list. Of course, you can also record the current time accordingly.

For example, the client B sends a write request to multiple data servers for the data B1 to be written, and the multiple data servers also include the data server U1, and M=N. Then the client B is also recorded in the unupgradable list of the data server U1. Of course, the current time can also be recorded.

An example of the non-upgradable list record of the data server U1 is shown in Table 1:

Of course, for the HOLD message sent again by the same client, only one client ID corresponding to the client may be recorded in the unupgradeable list, or multiple client IDs corresponding to the client may be recorded. In practical applications, preferably, for the HOLD message sent by the same client, only one client ID can be recorded, and then the time of each time is recorded in the time byte.

After receiving the HOLD message, the state of the data server itself is modified to the HOLD state, indicating that the data service cannot be upgraded.

Preferably, after sub-step 221, it further includes sub-step B21: sub-step B21, for the client identifiers in the non-upgradeable list, it is determined whether the second feedback message of the corresponding client is not received within a specified number of time periods, If the second feedback message of the corresponding client is not received within a specified number of time periods, the client identifier is deleted from the non-upgradeable list.

In this embodiment of the application, a time period T1 may be preset. For the client corresponding to the client identifier in the non-upgradeable list, if the data server does not receive the HOLD message sent by the client again, the non-upgradeable list will be removed. The client ID record is deleted.

For example, in Table 1 of the aforementioned data server U1, if the HOLD message of Client A is not received within 3*T1, the record of Client A in Table 1 will be deleted.

This step can prevent the data server from being in the non-upgradable state after being in the non-upgradable state.

In sub-step 222, the first feedback information is received, the client identifier in the first feedback information is deleted from the non-upgradeable list, and the status of itself is marked as an upgradable state after it is determined that the non-upgradeable list is empty.

For example, the aforementioned data server U1, which receives the OK message sent by the client A again, deletes the record of the client A from Table 1. When the data server deletes all the records in Table 1 and Table 1 is empty, it sets its own state to an upgradeable state. In another preferred embodiment of the present application, sub-step 222 includes sub-step B11 -B14: Sub-step B11, judging whether the non-upgradeable list has the client identifier in the first feedback information; if the non-upgradeable list has the client identifier in the first feedback information, enter the sub-step B12.

In sub-step B12, the client identifier is deleted from the unupgradeable list.

Since a data server may be allocated to different clients, it may receive the first feedback information of different clients. Then, for the received first feedback information of a certain client, it is determined whether the non-upgradeable list is If there is an identifier of the client with the first feedback information, and the unupgradeable list has an identifier of the client with the first feedback information, the client identifier is deleted from the unupgradable list; the unupgradable list has nothing If the client identification of the first feedback information is mentioned, subsequent operations may not be performed.

For example, for the aforementioned data server U1, if it receives the ok message sent by client A again, and the ok message includes the client ID, the client ID can be used to match the client ID in Table 1, and it is found that there is client A , Then the record of client A is cleared. If the Ok message sent by client C is received, and no record is found in Table 1, no operation is performed.

The above steps are convenient for the data server to manage its own upgrade status, and the logic is simple.

Sub-step B13, judge whether the non-upgradeable list is empty; if the non-upgradeable list is empty, then proceed to sub-step B14. If the non-upgradeable list is not empty, keep its own state as the non-upgradeable state.

Sub-step B14, mark its own state as an upgradeable state.

If the non-upgradeable list is not empty, keep its own non-upgradeable state.

As in the foregoing example, for the data server U1, it first receives the OK message sent by the client A, clears the record of the client A, and determines that the table 1 is still not empty, and continues to maintain the HOLD state. If an OK message from client B is received again, the record of client B in Table 1 is deleted in sub-step 221. At this time, it is judged that Table 1 is empty, and the data server U1 modifies its HODL status to OK. , Which means that the data server U1 can be upgraded.

Then, the upgrade control server can select and notify the data server to perform the upgrade operation in a round-robin manner according to the status of each data server. Of course, for the specific process of the upgrade control server controlling the upgrade of each data service, reference may be made to the description in the third embodiment.

Of course, in this embodiment of the application, the data server may also upgrade the exit notification sent by the control server to exit the upgrade preparation state, and exit the upgrade preparation state according to the exit notification, and at the same time send an exit notification to the client according to the exit notification, Make the client exit the upgrade preparation state. Then the data server processes the write request of the client according to the normal processing logic.

In the embodiment of this application, a distributed storage system upgrade method of this application is introduced from the data server side. For the data server, it determines itself according to the received first feedback information or second feedback information sent by the client Whether it is in an upgradeable state or a non-upgradable state. The first feedback information or the second feedback information is obtained based on the comparison result of the number of successful writes and the specified number after the client sends write requests to multiple data servers for the same piece of data to be written; The status includes an upgradeable status and an unupgradable status; the status is for the upgrade control server to select and notify the data server to perform the upgrade operation in a round-robin manner. Therefore, the client controls the status of the data server to ensure that the data to be written by any client is written to a specified number of data servers for backup. The upgrade control server controls each data server in a round-robin manner. During the upgrade, without stopping the upper-level services, the distributed storage system can not only ensure that the response time of the distributed storage system to the client is low, but also improve the reliability of the data, and greatly reduce the risk of user data loss.

Example three

3, there is shown a step flow chart of an embodiment of a method for upgrading a distributed storage system of the present application, which may specifically include the following steps: Step 310: Obtain the status of each data server; the status includes an upgradeable status and an unavailable status. Upgrade status, each data server has a status; the status of the data server is determined according to the first feedback information or the second feedback information; the first feedback information or the second feedback information is based on the same copy of the client After the data to be written sends write requests to multiple data servers, the comparison result of the number of successfully written data and the specified number is obtained.

Combining the description of the first and second embodiments, each data server in the distributed storage system can determine its status according to the first feedback message and/or the second feedback message fed back by the client. The status includes an upgradeable status and an unavailable status. Upgrade status.

Because each data server can only have one state at a time. For example, if a data server is in an upgradeable state, it cannot have other states. Other situations are similar, so I won't repeat them here.

Then, the upgrade control server of the embodiment of the present application can obtain the status of each data service.

There may be many specific ways for the upgrade control server to obtain the status of the data service, which is not limited in the embodiment of the present application.

Step 320: Notify at least one data server in an upgradeable state to perform the upgrade operation in a round-robin manner; the data server performs the upgrade operation according to the notification.

In the embodiment of the present application, since each data server sets its own state according to the feedback information of the client, the upgrade control server can control the data server in the upgradeable state to perform the upgrade operation in a round-robin manner.

For example, each time a batch of data servers that can be upgraded is selected, these data servers are notified to perform an upgrade operation. After the batch of data servers receive the upgrade notification, they can restart the upgrade.

In another preferred embodiment of the present application, step 320 includes sub-steps 321-322: sub-step 321, each time at least one data server in an upgradeable state is selected, and the at least one data server in an upgradeable state is notified For example, the data servers in the upgradeable state are U1, U2, U3, U10, U11, ... U20, etc., and the upgrade control server can select K data servers from them each time. Notify it to perform the upgrade operation, for example, select three. Where K is an integer greater than zero. Of course, K can be set according to actual needs, and this application does not limit it.

Sub-step 322, monitor whether the at least one data server in the upgradeable state has all performed the upgrade operation; if the at least one data server in the upgradeable state has all performed the upgrade operation, enter the sub-step 321; for example, The upgrade control server notifies the aforementioned data servers U1, U2, U3 to perform the upgrade operation, and then the data servers U1, U2, U3 restart the upgrade. After the data server is successfully upgraded, you can modify its own status to an upgraded status, such as DONE.

Then the upgrade control server can monitor whether the status of these data servers is DONE, and if it is not DONE, the upgrade is successful.

Of course, in actual applications, it is possible that during the upgrade of the batch of data servers, one or several of them may fail to upgrade, and the upgrade control server monitors the success or failure of the data server upgrade. In practical applications, the upgrade failure can be determined by the failure of the machine to restart, and the system version has not changed after the restart.

It should be noted that the above-mentioned completed upgrade operations include upgrade success and upgrade failure. After all the upgrade operations are completed, all the data servers are successfully upgraded; if part of the data servers are successfully upgraded, the remaining data servers will fail to upgrade.

If the states of U1, U2, and U3 are all DONE, the next step is to select at least one data server in an upgradeable state, and notify the at least one data server in an upgradeable state to perform the upgrade operation.

When the upgrade control server monitors that all the data servers notified of the upgrade have completed the upgrade operation, it can select the next batch of upgraded data servers from among the data servers that are in an upgradeable state.

In this way, the frequency of the round robin can be avoided too fast, and the risk of data loss can be reduced.

In another preferred embodiment of the present application, after the data server performs the upgrade operation, the method further includes: sub-step 323, if the upgrade operation of any data server is monitored as a result of the upgrade failure, then the data server The server is added to the upgrade blacklist, and the upgrade of the data server is suspended.

If the upgrade of a certain data server fails, the upgrade control server adds the data server to the upgrade blacklist and suspends the upgrade of the data server. Make these data servers wait to be offline or manually repaired.

Preferably, the multiple data servers of the embodiment of the present application can be distributed on at least two racks. Of course, in practical applications, each data server of the distributed storage system is distributed on multiple racks, one rack for one rack. Data server sub-cluster.

Further, step 320 includes sub-step C11: sub-step C11, each time the rack with the most data servers in the upgradeable state is selected, the data servers in the rack are notified to perform the upgrade operation; the rack Each data server in the data server checks its own state according to the notification, and if it is in the upgradeable state, restarts the upgrade, and if it is in the unupgradable state or the upgrade completed state, refuses to restart the upgrade.

In the embodiment of the present application, since the data servers are placed in the form of racks, a batch of data servers are placed in one rack. For a large number of clients, they may access different data servers in different racks, so that each rack may have data servers in an upgradeable state, or there may be data servers in an unupgradable state. There may also be data servers that have been upgraded.

In order to facilitate the upgrade control server to send the upgrade notification, the embodiment of the application sends the upgrade notification in units of racks. For example, the IP segment of a rack is 200.200.200.***, and the upgrade control server in this embodiment of the application A notification for 200.200.200.*** needs to be generated and broadcast to the rack, and each data server of the rack can receive the notification.

Then for the data server in the rack, after receiving the above upgrade notification, it will first determine whether its status is OK, if it is OK, restart the upgrade; if it is not OK, but HOLD or DONE, refuse to restart upgrade.

In the embodiment of the present application, in order to complete the upgrade faster, the rack with the most data servers with the status OK is selected, and the upgrade notification is sent to the rack.

For example, the data servers in the upgradeable state are U1, U2, U3, U10, U11, ... U20, etc., each time you can select K data servers from them, and notify them to perform the upgrade operation, for example, select 3. indivual.

Preferably, after sub-step C11, it further includes sub-step C12: sub-step C12, monitoring whether all the data servers in the rack have completed the upgrade operation; if all the data servers in the rack have completed the upgrade Operation, then enter the sub-step C11.

Then, when it is monitored that all the upgrade operations of the data servers in the upgradeable state in the rack are completed, the next rack can be selected and the data server of the next rack can be notified of the upgrade. This cycle continues until all data servers have been upgraded.

Through the above process, when the upgrade control server monitors that all data servers except the data servers that have failed to upgrade have been upgraded, it can notify each data server to exit the upgrade preparation state, and return the normal processing logic for each data. The server notifies the client accessing the data server to exit the upgrade preparation state, and the client sends back normal processing logic, and steps 310 and 320 are no longer executed.

In the embodiment of this application, a distributed storage system upgrade method of this application is introduced from the side of the upgrade control server. When the upgrade control server controls the upgradeable data server in a round-robin manner to upgrade, because The client controls the status of the data server, and ensures that the data to be written by any client is written to the specified number of data servers for backup at least. Therefore, the distributed storage system can be guaranteed without stopping the upper-layer services. The response time to the client is low, which also improves the reliability of the data and greatly reduces the risk of user data loss; and can tolerate unexpected machine abnormalities during the upgrade process to ensure that the service is not affected. Furthermore, the upgrade time is fast, avoiding a large amount of data migration.

Example four

4, there is shown a step flow chart of an embodiment of a method for upgrading a distributed storage system of the present application, which may specifically include the following steps: Step 410, the client sends the same piece of data to be written to multiple data servers Write request.

Step 412, the client receives the response from each data server, and judges whether the number of successful writes is greater than the specified number according to the response; if the number of successful writes is greater than the specified number, go to step 414 ; If the number of successful writes is not greater than the specified number, go to step 416.

Step 414: The client sends the first feedback information to each data server that has been successfully written.

Step 416, the client sends second feedback information to each data server that has successfully written; in another preferred embodiment of the present application, the first feedback information and the second feedback information include a client identifier. After the step of sending the second feedback information to each successfully written data server, the method further includes: step 417, when new data to be written appears, take multiple data servers including the previously written data server as The subject enters the step of sending write requests to multiple data servers for the same piece of data to be written.

In step 418, the data server receives the first feedback information or the second feedback information sent by the client; in step 420, the data server uses the first feedback information or the second feedback information when it is not upgraded. , Determine the status of the data server itself; the status includes the upgradeable status and the non-upgradable status; preferably on the basis of step 417, the step 420 includes sub-steps D11-D16: sub-step D11, when the second Feedback information, write the client identifier in the second feedback information into the non-upgradeable list, and mark its own status as non-upgradeable; sub-step D12, for the client identifier in the non-upgradeable list, determine whether it is in the designated The second feedback message of the corresponding client is not received within a number of time periods; if the second feedback message of the corresponding client is not received within the specified number of time periods, then the sub-step D13 is entered. If the second feedback message of the corresponding client is not received within a specified number of time periods, the upgradeable state is maintained.

In sub-step D13, the client identifier is deleted from the unupgradeable list. Proceed to sub-step D15.

Sub-step D14, judging whether the non-upgradeable list has the client identifier in the first feedback information; if the non-upgradeable list has the client identifier in the first feedback information, proceed to sub-step D13. If the non-upgradeable list does not have the client identifier in the first feedback information, enter sub-step D15 and sub-step D15 to determine whether the non-upgradeable list is empty; if the non-upgradeable list is empty, then enter the sub-step D16.

Sub-step D16, mark its own state as an upgradeable state.

Step 422, the upgrade control server obtains the status of each data server; step 424, the upgrade control server informs at least one data server in an upgradeable state to perform the upgrade operation in a round-robin manner; step 426, the data server The device performs the upgrade operation according to the notification.

Of course, the principle of the steps on the client side in the embodiment of the present invention can refer to the description of the first embodiment, the principle of the steps on the data server side can refer to the description of the second embodiment, and the principle of the steps on the upgrade control server side can refer to the embodiment. Three descriptions. I won't repeat it here.

In the embodiment of this application, a method for upgrading a distributed storage system of this application is introduced from three aspects: the client, the data server, and the upgrade control server. For clients accessing the distributed storage system, let each client send the write request to multiple data servers at the same time for the same write request of the data to be written, and then analyze how many data servers are written If the input is successful, it is determined whether the number of successful writes is greater than the specified number, and the first feedback information or the second feedback information is sent to each successfully written data server according to the determination result. For the data server, according to the received first feedback information or second feedback information, it is determined whether it is in an upgradeable state or a non-upgradable state. For the upgrade control server, the data server in the upgradeable state can be controlled in a round-robin state to perform the upgrade operation. Through the above process, when the upgrade control server controls the data server in an upgradeable state in a round-robin manner to upgrade, the client controls the state of the data server and ensures that the data to be written by any client is at least written A designated number of data servers are used for backup. Therefore, without stopping the upper-level services, the distributed storage system can not only ensure that the response time of the distributed storage system to the client is low, but also improve the reliability of the data and greatly reduce the user data. The risk of loss; and can tolerate unexpected machine abnormalities during the upgrade process to ensure that the service is not affected.

Example five

5, there is shown a structural block diagram of an embodiment of a distributed storage system upgrade device of the present application, which may specifically include the following modules: a request sending module 510, which is used to send multiple data to the same piece of data to be written The server sends a write request.

In another preferred embodiment of the present application, before step 510, the method further includes: a second upgrade notification receiving module, configured to receive a second upgrade notification sent by the data server when accessing a data server.

The second upgrade preparation module is used to enter the upgrade preparation state according to the second upgrade notification.

The judging module 520 is configured to receive the response returned by each data server, and determine whether the number of successful writes is greater than the specified number according to the response.

The first feedback module 530 is configured to send the first feedback information to each data server that successfully writes if the number of successful writes is greater than the specified number.

The second feedback module 540 is configured to send second feedback information to each data server that has been successfully written if the number of successful writes is not greater than the specified number; wherein, the first feedback information or the second feedback Information for the data server to determine its own status; the status includes an upgradeable status and an unupgradable status; the status of the data server is for the upgrade control server to notify the data server to perform upgrade operations in a round-robin manner .

In another preferred embodiment of the present application, the second feedback module 540 includes: a second feedback information sending sub-module, configured to write to each of the The successful data server sends the second feedback message.

The write request sending submodule is configured to send a write request to at least one data server other than the plurality of data servers for the data to be written if the number of successful writes is less than a specified number.

The judging sub-module is used to receive the response returned by the at least one data server, and to determine whether the current successfully written number is equal to the specified number based on the number of successfully written data; if the current successfully written number is equal to the specified number If the number is equal to the specified number, enter the second feedback information sending sub-module.

In another preferred embodiment of the present application, after the second feedback information sending sub-module, it further includes: a new to-be-written data sending sub-module, which is used when new data to be written appears, including the previous successful writing Enter the request sending module for multiple data servers of the data server.

In another preferred embodiment of the present application, the first feedback information and the second feedback information include a client identifier, and the first feedback information or the second feedback information for the data server to determine its own state includes : The second feedback information is for the data server to write the client identifier in the second feedback information into the non-upgradeable list and mark its own status as the non-upgradeable state after receiving the second feedback information; A feedback message for the data server to delete the client identifier in the first feedback message from the unupgradeable list after receiving the first feedback message, and mark its status as upgradeable after determining that the unupgradable list is empty state.

Example Six

Referring to FIG. 6, there is shown a structural block diagram of an embodiment of a distributed storage system upgrade apparatus of the present application, which may specifically include the following modules: A feedback information receiving module 610 is used to receive the first feedback information sent by the client or The second feedback information; the first feedback information or the second feedback information is based on the comparison result of the number of successful writes and the specified number after the client sends write requests to multiple data servers for the same piece of data to be written get.

In another preferred embodiment of the present application, before the feedback information receiving module 610, it further includes: a first upgrade notification receiving module, configured to receive the first upgrade notification sent by the upgrade control server.

The first upgrade preparation module is configured to enter the upgrade preparation state according to the first upgrade notification, and send a second upgrade notification to the client after receiving the client's access request, so that the client enters the upgrade Ready state.

The status determination module 620 is configured to determine the status of the data server itself based on the first feedback information or the second feedback information when the status determination module 620 is not upgraded; the status includes an upgradeable status and an unupgradable status; The status allows the upgrade control server to select and notify the data server to perform the upgrade operation in a round-robin manner.

In another preferred embodiment of the present application, the status determination module 620 includes: an upgradeable status determination sub-module for deleting the customer in the first feedback information from the unupgradable list when the first feedback information is received Terminal identification, and after determining that the non-upgradable list is empty, mark its own state as an upgradable state.

Preferably, the upgradeable status determining sub-module includes: a client identification judging sub-module for determining whether the unupgradable list has the client identification in the first feedback information; if the unupgradable list has The client identifier in the first feedback information enters the first deletion sub-module.

The first deletion submodule is used to delete the client identifier from the unupgradeable list.

The non-upgradeable list judging sub-module is used to determine whether the non-upgradeable list is empty; if the non-upgradeable list is empty, then enter the upgradeable state to mark the sub-module.

Upgradable status marking sub-modules are used to mark their own status as upgradable.

The non-upgradeable status determination sub-module is used for when receiving the second feedback information, write the client identifier in the second feedback information into the non-upgradeable list, and mark its own status as the non-upgradeable state.

In another preferred embodiment of the present application, after the non-upgradeable status determination sub-module, it further includes: a time judging sub-module, which is used to determine whether it is within a specified number of times for the client identifiers in the non-upgradeable list If the second feedback message of the corresponding client is not received within the period, if the second feedback message of the corresponding client is not received within the specified number of time periods, enter the second delete submodule; the second delete submodule, It is used to delete the client identifier from the non-upgradeable list.

Example Seven

Referring to FIG. 7, there is shown a structural block diagram of an embodiment of a distributed storage system upgrade apparatus of the present application, which may specifically include the following modules: a status acquisition module 710 for acquiring the status of each data server; the status Including upgradeable status and non-upgradable status, each data server has a status; the status of the data server is determined according to the first feedback information or the second feedback information; the first feedback information or the second feedback information After the client sends a write request to multiple data servers for the same piece of data to be written, the comparison result of the number of successful writes and the specified number is obtained.

In another preferred embodiment of the present application, before the status acquisition module 710, it further includes: an upgrade notification sending module, configured to send a first upgrade notification to each data server, so that each data server enters the upgrade preparation state , And make each data server send a second upgrade notification to the client after receiving the access request of the client, so that the client enters the upgrade preparation state.

The upgrade notification module 720 is configured to notify at least one data server in an upgradeable state to perform the upgrade operation in a round-robin manner; the data server performs the upgrade operation according to the notification.

In another preferred embodiment of the present application, the upgrade notification module includes: a first selection sub-module for selecting at least one data server in an upgradeable state each time, and notifying the at least one data server in an upgradeable state each time The data server in the state performs the upgrade operation; the first monitoring sub-module is used to monitor whether the at least one data server in the upgradeable state has all performed the upgrade operation; if the at least one data server in the upgradeable state has completed the upgrade operation; After completing the upgrade operation, enter the first selection sub-module.

In another preferred embodiment of the present application, after monitoring the sub-module, it further includes: suspending the sub-module, which is adapted to update the data if the result of the monitoring of any data server's upgrade operation is that the upgrade fails. The server is added to the upgrade blacklist, and the upgrade of the data server is suspended.

In another preferred embodiment of the present application, each data server is distributed on at least two racks; then the upgrade notification module 720 includes: an upgrade notification sub-module for selecting data in an upgradeable state each time The rack with the most servers informs the data servers in the rack to perform the upgrade operation; each data server in the rack checks its own status according to the notification, and if it is in the upgradeable state, restarts the upgrade. If it is in a non-upgradeable state or the upgrade is completed, refuse to restart the upgrade.

In another preferred embodiment of the present application, after the upgrade notification sub-module, it further includes: a monitoring sub-module for monitoring whether all the data servers in the rack have completed the upgrade operation; if the machine After all the data servers in the rack have completed the upgrade operation, enter the upgrade notification sub-module.

Example eight

8 and 8A, which shows a structural block diagram of an embodiment of a distributed storage system upgrade system of the present application, which may specifically include the following modules: multiple clients 810, multiple data servers 820, and upgrade control Server 830.

Among them, FIG. 8A shows a schematic structural diagram of a distributed storage system according to an embodiment of the present invention. In the embodiment of the present application, each client can send write requests to R in the distributed storage system, and the upgrade control server controls the upgrade process of all data servers.

FIG. 8 shows the connection relationship among the client 810, the data server 820, and the upgrade control server 830.

The client 810 includes: a request sending module 811 for sending a write request to multiple data servers for the same piece of data to be written; a judgment module 812 for receiving a response from each data server, And according to the response, it is judged whether the number of successful writing is greater than the specified number; the first feedback module 813 is used to send to each successfully written data server if the number of successful writing is greater than the specified number The first feedback information; the second feedback module 814, if the number of successful writes is not greater than the specified number, send second feedback information to each successfully written data server; the data server 820 includes : The data storage module 821 is used to receive the write request from the client and send back the response to the client.

The feedback information receiving module 822 is used to receive the first feedback information or the second feedback information sent by the client; the status determining module 823 is used to determine the status according to the first feedback information or the second feedback information when it is not upgraded. 2. Feedback information to determine the status of the data server itself.

The upgrade module 824 is used to perform the upgrade operation according to the notification from the upgrade control server.

The upgrade control server 830 includes: a status acquisition module 831 for acquiring the status of each data server; an upgrade notification module 832 for notifying at least one data server that is in an upgradeable state in a round-robin manner. Upgrade operation.

For the client module of the embodiment of the present application, please refer to the description of the fifth embodiment, the module of the data server can refer to the description of the sixth embodiment, and the module of the upgrade controller can refer to the description of the seventh embodiment. The principle is basically similar, so I won't repeat it here.

In the embodiment of the present application, a method for upgrading a distributed storage system of the present application is introduced from three aspects of a client, a data server, and an upgrade control server. For clients accessing the distributed storage system, let each client send the write request to multiple data servers at the same time for the same write request of the data to be written, and then analyze how many data servers are written If the input is successful, it is determined whether the number of successful writes is greater than the specified number, and the first feedback information or the second feedback information is sent to each successfully written data server according to the determination result. For the data server, according to the received first feedback information or second feedback information, it is determined whether it is in an upgradeable state or a non-upgradable state. For the upgrade control server, the data server in the upgradeable state can be controlled in a round-robin state to perform the upgrade operation. Through the above process, when the upgrade control server controls the data server in an upgradeable state in a round-robin manner to upgrade, the client controls the state of the data server and ensures that the data to be written by any client is at least written A designated number of data servers are used for backup. Therefore, without stopping the upper-level services, the distributed storage system can not only ensure that the response time of the distributed storage system to the client is low, but also improve the reliability of the data and greatly reduce the user data. The risk of loss; and can tolerate unexpected machine abnormalities during the upgrade process to ensure that the service is not affected.

As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to the part of the description of the method embodiment.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other.

Those skilled in the art should understand that the embodiments of the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present application may adopt computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. form.

In a typical configuration, the computer device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. Memory memory may include non-permanent memory in computer-readable media, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory. Body (FLASH RAM). Memory memory is an example of computer-readable media. Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access (RAM), Read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) ) Or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include non-persistent computer-readable media (TRANSITORY MEDIA), such as modulated data signals and carrier waves.

The embodiments of the present application are described with reference to the flowcharts and/or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing terminal equipment to generate a machine, so that the instructions to be executed by the processor of the computer or other programmable data processing terminal equipment A device for realizing the functions specified in a flow or multiple flows in the flowchart and/or a block or multiple blocks in the block diagram is generated.

These computer program instructions can also be stored in a computer-readable storage that can guide a computer or other programmable data processing terminal equipment to work in a specific manner, so that the instructions stored in the computer-readable storage produce a manufactured product including the instruction device , The instruction device realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing terminal equipment, so that a series of operation steps are executed on the computer or other programmable terminal equipment to produce computer-implemented processing, so that the computer or other programmable terminal equipment The instructions executed above provide steps for implementing functions specified in a flow or multiple flows in the flowchart and/or a block or multiple blocks in the block diagram.

Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

Finally, it should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these entities. Or there is any such actual relationship or sequence between operations. Moreover, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements not only includes those elements, but also includes those elements that are not explicitly listed. The other elements listed, or also include elements inherent to this process, method, article, or terminal device. If there are no more restrictions, the elements defined by the sentence "including a..." do not exclude the existence of other identical elements in the process, method, article, or terminal device that includes the elements.

The above provides a detailed introduction to a distributed storage system upgrade method, a distributed storage upgrade device, and a distributed storage upgrade system provided by this application, and specific examples are used in this article to illustrate the principle and implementation of this application. The description of the above embodiments is only used to help understand the methods and core ideas of the application; at the same time, for those of ordinary skill in the art, according to the ideas of the application, there will be changes in the specific implementation and the scope of application In summary, the content of this specification should not be construed as a limitation to this application.

Claims (32)

A method for upgrading a distributed storage system, applied to a client, is characterized in that it includes: sending a write request to multiple data servers for the same piece of data to be written; receiving a response from each data server, and according to the In response, determine whether the number of successful writes is greater than the specified number; if the number of successful writes is greater than the specified number, the first feedback information will be sent to each data server that successfully writes; if the number of successful writes is not If the number is greater than the specified number, the second feedback information is sent to each successfully written data server; wherein the first feedback information or the second feedback information is used for the data server to determine its own state; the state includes an upgradeable state and Unupgradeable status; the status of the data server is for the upgrade control server to notify the data server of the upgrade operation in a round-robin manner, where the specified number is the preset minimum number of successful writes. According to the method described in item 1 of the scope of patent application, wherein, if the number of successful writes is not greater than the specified number, the second feedback information is sent to each successfully written data server for the data server to follow The second feedback information determines that its own status is not upgradeable. The step includes: if the number of successful writes is equal to a specified number, sending the second feedback information to each data server that successfully writes; if the write is successful If the number of successes is less than the specified number, the data to be written will be sent to at least one data server other than the multiple data servers. Request; receive the response returned by the at least one data server, and combine the number of successful writes to determine whether the current number of successful writes is equal to the specified number; if the current number of successful writes is equal to the specified number, Then, the second feedback information is sent to each data server that has been successfully written. According to the method described in item 1 or 2 of the scope of patent application, after the step of sending the second feedback information to each successfully written data server, the method further includes: when new data to be written appears, it includes the previously written data. The multiple data servers of the successfully entered data server are targeted, and the step of sending write requests to multiple data servers for the same piece of data to be written is entered. The method according to item 3 of the scope of patent application, wherein the first feedback information and the second feedback information include a client identifier, and the first feedback information or the second feedback information for the data server to determine its own state includes : The second feedback information is for the data server to write the client identification in the second feedback information into the non-upgradeable list and mark its own status as the non-upgradeable state after receiving the second feedback information; the first feedback information After receiving the first feedback information, the data server deletes the client identifier in the first feedback information from the unupgradeable list, and marks its own status as an upgradable state after determining that the unupgradable list is empty. According to the method described in item 1 of the scope of patent application, Before the step of sending a write request to multiple data servers for the same data to be written, the method further includes: when accessing a data server, receiving a second upgrade notification sent by the data server; according to the second upgrade notification Enter the upgrade preparation state. A method for upgrading a distributed storage system, applied to a data server, is characterized in that it comprises: receiving first feedback information or second feedback information sent by a client; the first feedback information or second feedback information is based on the same After the data to be written is sent to multiple data servers, the comparison result of the number of successfully written data and the specified number is obtained; in the case of not being upgraded, according to the first feedback information or the second feedback Information to determine the status of the data server itself; the status includes an upgradeable status and an unupgradable status; this status is for the upgrade control server to select and notify the data server to perform the upgrade operation in a round-robin manner, where the specified number is The preset minimum number of successful writes. According to the method described in item 6 of the scope of patent application, wherein the first feedback information and the second feedback information include a client identifier, and the user is not upgraded according to the first feedback information or the second feedback Information, the step of determining the status of the data server itself includes: when receiving the second feedback information, writing the client identifier in the second feedback information into the non-upgradeable list, and marking its own status as the non-upgradeable state; When the first feedback information is received, delete the first one from the unupgradeable list The client ID in the feedback information, and after determining that the non-upgradeable list is empty, mark its status as an upgradeable state. The method according to item 7 of the scope of patent application, wherein the step of deleting the client identifier in the first feedback information from the non-upgradeable list, and marking its status as an upgradeable state after determining that the non-upgradable list is empty , Including: determining whether the unupgradable list has the client identifier in the first feedback information; if the unupgradable list has the client identifier in the first feedback information, deleting the client identifier from the unupgradable list ; Determine whether the non-upgradeable list is empty; if the non-upgradeable list is empty, mark its own status as an upgradeable state. The method according to item 7 of the scope of patent application, wherein when the second feedback information is received, the client identifier in the second feedback information is written into the non-upgradeable list, and its status is marked as the non-upgradeable state After the steps, it also includes: for the client identifiers in the non-upgradeable list, judging whether the second feedback message of the corresponding client has not been received within a specified number of time periods, and if it has not been received within a specified number of time periods When the second feedback message to the corresponding client is received, the client identifier is deleted from the unupgradeable list. The method according to item 6 of the scope of patent application, wherein, before the step of receiving the first feedback information or the second feedback information sent by the client terminal, the method further includes: Receive the first upgrade notification sent by the upgrade control server; enter the upgrade preparation state according to the first upgrade notification, and send a second upgrade notification to the client after receiving the client's access request, so that the client enters Upgrade preparation status. A distributed storage system upgrade method, applied to upgrade control servers, is characterized in that it includes: obtaining the status of each data server; the status includes an upgradeable status and an unupgradable status, and each data server has a status; the The state of the data server is determined based on the first feedback information or the second feedback information; the first feedback information or the second feedback information is written after the client sends a write request to multiple data servers for the same piece of data to be written The result of comparing the number of successes with the specified number is obtained, where the specified number is the preset minimum number of successful writes. ; Notify at least one data server in an upgradeable state to perform the upgrade operation in a round-robin manner; the data server performs the upgrade operation according to the notification. The method according to item 11 of the scope of patent application, wherein the step of notifying at least one data server in an upgradeable state to perform an upgrade operation in a round-robin manner includes: selecting at least one data in an upgradeable state each time The server notifies the at least one data server in the upgradeable state to perform the upgrade operation; monitors whether the at least one data server in the upgradeable state has all performed the upgrade operation; If the at least one data server in the upgradeable state has all been upgraded, it will proceed to the next time to select at least one data server in the upgradeable state, and notify the at least one data server in the upgradeable state to perform the upgrade operation. step. According to the method described in item 12 of the scope of patent application, it further includes: if the result of monitoring any data server's upgrade operation is an upgrade failure, adding the data server to the upgrade blacklist and suspending the data server The upgrade of the device. According to the method described in one of items 11-13 in the scope of patent application, wherein each data server is distributed on at least two racks; then at least one data server in an upgradeable state is notified in a round-robin manner The upgrade operation steps include: selecting the rack with the most data servers in the upgradeable state each time, and informing the data servers in the rack to perform the upgrade operation; each data server in the rack checks according to the notification If it is in the upgradeable state, restart the upgrade if it is in the upgradeable state, and refuse to restart the upgrade if it is in the unupgradable state or the upgrade completed state. According to the method described in item 14 of the scope of patent application, after selecting the rack with the most data servers in the upgradeable state each time and informing the data servers in the rack to perform the upgrade operation, the method further includes: Monitor whether all the data servers in the rack have completed the upgrade operation; If all the data servers in the rack have completed the upgrade operation, the next step is to select the rack with the most data servers in the upgradeable state, and notify the data servers in the rack to perform the upgrade operation. According to the method described in item 11 of the scope of patent application, before acquiring the status of each data server, it further includes: sending a first upgrade notification to each data server, making each data server enter the upgrade preparation state, and making After receiving the access request of the client, each data server sends a second upgrade notification to the client, so that the client enters the upgrade preparation state. A distributed storage system upgrade device, applied to a client, is characterized in that it comprises: a request sending module for sending write requests to multiple data servers for the same piece of data to be written; a judgment module for Receive the response returned by each data server, and determine whether the number of successful writes is greater than the specified number based on the response; the first feedback module is used to send the The successfully written data server sends the first feedback information; the second feedback module is used to send the second feedback information to each successfully written data server if the number of successful writes is not greater than the specified number; Wherein, the first feedback information or the second feedback information is used for the data server to determine its own state; the state includes an upgradeable state and an unupgradable state; the state of the data server is notified by the upgrade control server in a round-robin manner The data server performs an upgrade operation, where the specified number is the preset minimum number of successful writes. The device according to item 17 of the scope of patent application, wherein, the second feedback module includes: a second feedback information sending sub-module for sending the data to each The successfully written data server sends the second feedback information; the write request sending sub-module is used to send the data to be written to a data server other than the plurality of data servers if the successfully written data is less than the specified number At least one data server sends a write request; the judging sub-module is used to receive the response returned by the at least one data server, and combine the number of successful writes to determine whether the current number of successful writes is equal to the specified number Number; if the number of currently written successfully is equal to the specified number, enter the second feedback information sending sub-module. According to the device according to item 17 or 18 of the scope of patent application, after the second feedback module, it further includes: a new data to be written sending module, which is used to include the new data to be written when there is new data to be written. Multiple data servers of the successful data server are targeted, enter the request sending module. The device according to item 19 of the scope of patent application, wherein the first feedback information and the second feedback information include a client identifier, and the first feedback information or the second feedback information for the data server to determine its own state includes : The second feedback information is for the data server to write the client identifier in the second feedback information into the non-upgradeable list and mark its own status as the non-upgradeable state after receiving the second feedback information; The first feedback information is used by the data server to delete the client identifier in the first feedback information from the unupgradable list after receiving the first feedback information, and mark its own status as available after determining that the unupgradable list is empty. Upgrade status. The device according to item 17 of the scope of patent application, wherein, before the step of sending a write request to multiple data servers for the same piece of data to be written, it further includes: a second upgrade notification receiving module for When a data server is accessed, a second upgrade notification sent by the data server is received; the second upgrade preparation module is used to enter an upgrade preparation state according to the second upgrade notification. A distributed storage system upgrade device applied to a data server is characterized by comprising: a feedback information receiving module for receiving first feedback information or second feedback information sent by a client; the first feedback information or the second feedback information 2. The feedback information is obtained based on the comparison result of the number of successful writes and the specified number after the client sends a write request to multiple data servers for the same piece of data to be written, where the specified number is the default write The minimum number of successful entries. ; The status determination module is used to determine the status of the data server itself based on the first feedback information or the second feedback information when it is not upgraded; the status includes an upgradeable status and an unupgradable status; the status For the upgrade control server to choose to notify the data server to perform the upgrade operation in a round-robin manner. The device according to item 22 of the scope of patent application, wherein the The status determination module includes: a non-upgradeable status determination sub-module, which is used to write the client identifier in the second feedback information into the non-upgradeable list when receiving the second feedback information, and mark its own status as non-upgradeable Upgrade status; the upgradeable status determination sub-module is used to delete the client ID in the first feedback information from the non-upgradeable list when the first feedback information is received, and mark its own status after determining that the non-upgradeable list is empty It is in an upgradeable state. The device according to item 23 of the scope of patent application, wherein the upgradeable status determination sub-module includes: a client identification determination sub-module for determining whether the non-upgradable list has the client in the first feedback information Terminal identification; if the unupgradable list has the client identification in the first feedback information, enter the first deletion submodule; the first deletion submodule is used to delete the client identification from the unupgradable list; The unupgradable list judging submodule is used to determine whether the unupgradable list is empty; if the unupgradable list is empty, it enters the upgradeable state to mark the submodule; the upgradeable state marks the submodule to mark its own The status is upgradeable. The device according to item 23 of the scope of patent application, wherein, after the non-upgradeable state determination sub-module, it further includes: a time judgment sub-module, which is used for the client in the non-upgradeable list Identifies whether the second feedback message of the corresponding client is not received within a specified number of time periods. If the second feedback message of the corresponding client is not received within the specified number of time periods, enter the second delete sub Module; the second deletion sub-module is used to delete the client ID from the unupgradeable list. The device according to item 22 of the scope of patent application, wherein, before the feedback information receiving module, it further includes: a first upgrade notification receiving module, configured to receive the first upgrade notification sent by the upgrade control server; The upgrade preparation module is configured to enter the upgrade preparation state according to the first upgrade notification, and send a second upgrade notification to the client after receiving the client's access request, so that the client enters the upgrade preparation state. A distributed storage system upgrade device, applied to upgrade control servers, is characterized by comprising: a status acquisition module for acquiring the status of each data server; the status includes an upgradeable status and an unupgradable status, and each data The server has a state; the state of the data server is determined based on the first feedback information or the second feedback information; the first feedback information or the second feedback information is sent to multiple data servers based on the same piece of data to be written by the client After the write request, the result of comparing the number of successful writes with the specified number is obtained, where the specified number is the preset minimum number of successful writes; the upgrade notification module is used in a round-robin manner Notify at least one data server in an upgradeable state to perform the upgrade operation; the data server root Perform the upgrade operation according to the notification. The device according to item 27 of the scope of patent application, wherein the upgrade notification module includes: a first selection sub-module for selecting at least one data server in an upgradeable state each time, and notifying the at least one data server in an upgradeable state each time The data server in the upgrade state performs the upgrade operation; the first monitoring sub-module is used to monitor whether the at least one data server in the upgradeable state has completed the upgrade operation; if the at least one data server in the upgradeable state has been completed After performing all upgrade operations, enter the first selection sub-module. The device according to item 28 of the scope of patent application, which further includes: a suspension sub-module, adapted to add the data server to the upgrade black if the result of the upgrade operation of any data server is monitored as an upgrade failure List, the upgrade of the data server is suspended. According to the device according to one of items 27-29 in the scope of patent application, each data server is distributed on at least two racks; then the upgrade notification module includes: an upgrade notification sub-module for each Select the rack with the most data servers in the upgradeable state, and notify the data servers in the rack to perform the upgrade operation; each data server in the rack checks its own status according to the notification, and if it is in the upgradeable state , Restart the upgrade, if it is in the unupgradeable state or the upgrade is completed state, refuse to restart the upgrade. The device according to item 30 of the scope of patent application, wherein After the upgrade notification sub-module, it also includes: a second monitoring sub-module, which is used to monitor whether all the data servers in the rack have completed the upgrade operation; if all the data servers in the rack have completed the upgrade operation, Then enter the upgrade notification sub-module. The device according to item 27 of the scope of patent application, before the status acquisition module, further includes: an upgrade notification sending module, which is used to send a first upgrade notification to each data server so that each data server enters the upgrade And make each data server send a second upgrade notification to the client after receiving the access request of the client, so that the client enters the upgrade preparation state.

Images