CN110990195A - Data recovery method, equipment and storage medium - Google Patents
Data recovery method, equipment and storage medium Download PDFInfo
- Publication number
- CN110990195A CN110990195A CN201911161216.5A CN201911161216A CN110990195A CN 110990195 A CN110990195 A CN 110990195A CN 201911161216 A CN201911161216 A CN 201911161216A CN 110990195 A CN110990195 A CN 110990195A
- Authority
- CN
- China
- Prior art keywords
- data
- recovery
- rate
- object storage
- recovered
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000011084 recovery Methods 0.000 title claims abstract description 233
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012544 monitoring process Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000002159 abnormal effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1471—Saving, restoring, recovering or retrying involving logging of persistent data for recovery
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a data recovery method, a device and a storage medium, wherein the method comprises the following steps: acquiring a rated recovery rate of the object storage equipment cluster and the total amount of data to be recovered of the object storage equipment cluster; counting the local data volume of the local data to be recovered, and calculating to obtain the ratio of the local data volume in the total data to be recovered; and calculating to obtain the estimated rate according to the ratio and the rated recovery rate, and executing data recovery operation on the local data to be recovered based on the estimated rate. The method can relatively reduce the occupation of system resources by the object storage device cluster in the data recovery process, thereby ensuring the stability of the front-end service when the object storage device cluster recovers the data. In addition, the invention also provides object storage equipment and a storage medium, and the beneficial effects are as above.
Description
Technical Field
The present invention relates to the field of data storage, and in particular, to a data recovery method, device, and storage medium.
Background
With the rise and popularization of cloud computing technology, various kinds of distributed shared storage systems formed based on Object-based storage device (OSD) clusters are getting more and more attention from the industry.
The object storage device cluster has a function of recovering abnormal data when data is damaged or lost, but front-end services supported by the object storage device cluster often require the object storage device cluster to have continuous and stable performance, and when the current object storage device cluster recovers the data, the object storage device cluster occupies larger system resources in the whole data recovery process because the recovery rate among the object storage devices in the cluster is not correspondingly limited, so that the front-end services may be abnormal due to insufficient available system resources.
Therefore, it is a problem to be solved by those skilled in the art to provide a data recovery method to relatively ensure the stability of front-end services when performing data recovery on a cluster of object storage devices.
Disclosure of Invention
The invention aims to provide a data recovery method, equipment and a storage medium, which are used for relatively ensuring the stability of front-end business when an object storage equipment cluster performs data recovery.
In order to solve the above technical problem, the present invention provides a data recovery method, which is applied to an object storage device in an object storage device cluster, and includes:
acquiring a rated recovery rate of the object storage equipment cluster and the total amount of data to be recovered of the object storage equipment cluster;
counting the local data volume of the local data to be recovered, and calculating to obtain the ratio of the local data volume in the total data to be recovered;
and calculating to obtain the estimated rate according to the ratio and the rated recovery rate, and executing data recovery operation on the local data to be recovered based on the estimated rate.
Preferably, the performing the data recovery operation on the local data to be recovered based on the estimated rate includes:
performing data recovery operation on local data to be recovered based on the estimated rate in the current recovery period;
before performing a data recovery operation on the local data to be recovered based on the estimated rate in the current recovery period, the method further includes:
judging whether a history recovery period adjacent to the current recovery period exists or not;
if a history recovery period adjacent to the current recovery period exists, acquiring a history estimated rate and a history actual rate corresponding to the history recovery period;
adjusting the estimated rate according to the historical estimated rate and the historical actual rate, and executing data recovery operation on the local data to be recovered based on the estimated rate in the current recovery period by using the adjusted estimated rate;
and if the historical recovery period adjacent to the current recovery period does not exist, executing the step of executing the data recovery operation on the local to-be-recovered data based on the estimated speed in the current recovery period.
Preferably, the adjusting the estimated rate according to the historical estimated rate and the historical actual rate includes:
calculating a speed difference value between the historical estimated speed and the historical actual speed;
when the historical estimated speed is greater than the historical actual speed, increasing the difference of the estimated speed by an acceleration rate;
and when the historical estimated speed is smaller than the historical actual speed, reducing the speed difference of the estimated speed.
Preferably, the method further comprises:
if the current recovery period is finished, judging whether local data to be recovered still exist;
if the local data to be restored still exist, recording the estimated speed and the actual speed of the current restoration cycle to generate a new estimated speed of the next restoration cycle, and performing data restoration operation on the local data to be restored at the new estimated speed in the next restoration cycle;
and if the local data to be recovered does not exist, stopping the data recovery process.
Preferably, the obtaining of the rated recovery rate of the object storage device cluster and the total amount of data to be recovered of the object storage device cluster includes:
and acquiring the rated recovery rate of the object storage equipment cluster set by a user and the total amount of data to be recovered of the object storage equipment cluster monitored and transmitted by the monitoring node.
In addition, the present invention also provides an object storage device, which is applied to an object storage device cluster, and comprises:
the acquisition module is used for acquiring the rated recovery rate of the object storage equipment cluster and the total amount of data to be recovered of the object storage equipment cluster;
the statistical module is used for counting the local data volume of the local data to be recovered and calculating to obtain the ratio of the local data volume in the total data to be recovered;
and the data recovery module is used for calculating the estimated rate according to the ratio value and the rated recovery rate and executing data recovery operation on the local data to be recovered based on the estimated rate.
Preferably, the data recovery module includes:
the period recovery module is used for performing data recovery operation on the local data to be recovered based on the estimated rate in the current recovery period;
the object storage device further includes:
the period judging module is used for judging whether a history recovery period adjacent to the current recovery period exists or not, if so, the history obtaining module and the adjusting module are sequentially called, and otherwise, the period recovery module is called;
the history acquisition module is used for acquiring a history estimated speed and a history actual speed corresponding to a history recovery period;
and the adjusting module is used for adjusting the estimated rate according to the historical estimated rate and the historical actual rate and calling the period recovery module by using the adjusted estimated rate.
Preferably, the adjusting module includes:
the difference value calculating module is used for calculating the speed difference value between the historical estimated speed and the historical actual speed;
the increasing module is used for increasing the difference value of the estimated speed when the historical estimated speed is greater than the historical actual speed;
and the reduction module is used for reducing the difference of the estimated speed when the historical estimated speed is less than the historical actual speed.
Preferably, the object storage device further includes:
the cycle ending module is used for judging whether local data to be recovered still exists or not if the current recovery cycle is ended, if so, calling the recording module, and otherwise, calling the recovery stopping module;
the recording module is used for recording the estimated speed and the actual speed of the current recovery period so as to generate a new estimated speed of the next recovery period, and performing data recovery operation on the local data to be recovered at the new estimated speed in the next recovery period;
and the stopping and recovering module is used for stopping the data recovering process.
Furthermore, the present invention also provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, realizes the steps of the data recovery method as described above.
The data recovery method provided by the invention is applied to the object storage equipment in the object storage equipment cluster, firstly, the rated recovery rate and the total amount of data to be recovered of the object storage equipment cluster are obtained, then, the local data volume of the local data to be recovered in the object storage equipment is counted, the ratio of the local data volume in the total amount of the data to be recovered is calculated, finally, the estimated rate corresponding to the object storage equipment is calculated through the ratio and the rated recovery rate, and then, the data recovery operation is carried out on the local data to be recovered based on the estimated rate. Because each object storage device in the cluster divides the estimated rate of the corresponding ratio in the rated recovery rate of the cluster according to the proportion of the data volume required to be recovered to the overall data volume to be recovered of the cluster, and is used for performing data recovery operation on the local data to be recovered, when the object storage device cluster performs data recovery, the recovery rate of each object storage device in the cluster is limited by the rated rate of the cluster and the ratio of the local data to be recovered to the overall data volume to be recovered of the cluster, so that the occupation of system resources by the object storage device cluster in the data recovery process can be relatively reduced, and the stability of front-end service of the object storage device cluster in the data recovery process is further ensured. In addition, the invention also provides object storage equipment and a storage medium, and the beneficial effects are as above.
Drawings
In order to illustrate the embodiments of the present invention more clearly, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained by those skilled in the art without inventive effort.
FIG. 1 is a flow chart of a data recovery method applied to an object storage device in an object storage device cluster according to the present invention;
FIG. 2 is a flowchart of a data recovery method applied to an object storage device in an object storage device cluster according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of an object storage device according to the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without any creative work belong to the protection scope of the present invention.
The object storage device cluster has a function of recovering abnormal data when data is damaged or lost, but front-end services supported by the object storage device cluster often require the object storage device cluster to have continuous and stable performance, and when the current object storage device cluster recovers the data, the object storage device cluster occupies larger system resources in the whole data recovery process because the recovery rate among the object storage devices in the cluster is not correspondingly limited, so that the front-end services may be abnormal due to insufficient available system resources.
Therefore, the core of the invention is to provide a data recovery method to relatively ensure the stability of the front-end service when the object storage device cluster performs data recovery.
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
Referring to fig. 1, an embodiment of the present invention discloses a data recovery method, which is applied to an object storage device in an object storage device cluster, and includes:
step S10: and acquiring the rated recovery rate of the object storage device cluster and the total amount of data to be recovered of the object storage device cluster.
It should be noted that, the Object-based Storage Device is an Object-based Storage Device (OSD), and the core is to separate a data path (data reading or writing) from a control path (metadata), and construct a Storage system based on the Object-based Storage Device, where each Object-based Storage Device has a certain function and can automatically manage data distribution thereon. The object storage structure is composed of an object, an object storage device, a metadata server and a client of the object storage system. The object storage device cluster is a cluster composed of object storage devices, and the object storage device cluster generally includes 1 client and 3 or more object storage devices.
The rated recovery rate of the object storage device cluster obtained in this step may be understood as an upper limit of an overall recovery rate for performing data recovery operations on all object storage devices in the object storage device cluster. In addition, the total amount of data to be restored of the object storage device cluster refers to the sum of the amounts of data to be restored of the respective object storage devices in the object storage device cluster.
Step S11: and counting the local data volume of the local data to be recovered, and calculating to obtain the ratio of the local data volume to the total data to be recovered.
After the rated recovery rate of the object storage device cluster and the total amount of data to be recovered of the object storage device cluster are obtained, the object storage device further counts the local data volume of the local data to be recovered, and calculates the ratio of the local data volume in the total amount of data to be recovered, wherein the ratio represents the quantity relationship between the current data volume to be recovered of the object storage device and the data volume to be recovered of other object storage devices in the cluster.
Step S12: and calculating to obtain the estimated rate according to the ratio and the rated recovery rate, and executing data recovery operation on the local data to be recovered based on the estimated rate.
And then after the proportion value of the local data volume in the total data volume to be recovered is obtained, the current object storage device further divides the estimated rate of the corresponding proportion value in the rated recovery rate as the data recovery rate of the current object storage device, and then performs data recovery operation on the local data to be recovered based on the preset rate.
The important point of this embodiment is that the rated recovery rate of the object storage device cluster and the data amount of the object storage device in the object storage device cluster, which needs to perform data recovery, are used as the constraint condition for generating the estimated rate of the object storage device.
The data recovery method provided by the invention is applied to the object storage equipment in the object storage equipment cluster, firstly, the rated recovery rate and the total amount of data to be recovered of the object storage equipment cluster are obtained, then, the local data volume of the local data to be recovered in the object storage equipment is counted, the ratio of the local data volume in the total amount of the data to be recovered is calculated, finally, the estimated rate corresponding to the object storage equipment is calculated through the ratio and the rated recovery rate, and then, the data recovery operation is carried out on the local data to be recovered based on the estimated rate. Because each object storage device in the cluster divides the estimated rate of the corresponding ratio in the rated recovery rate of the cluster according to the proportion of the data volume required to be recovered to the overall data volume to be recovered of the cluster, and is used for performing data recovery operation on the local data to be recovered, when the object storage device cluster performs data recovery, the recovery rate of each object storage device in the cluster is limited by the rated rate of the cluster and the ratio of the local data to be recovered to the overall data volume to be recovered of the cluster, so that the occupation of system resources by the object storage device cluster in the data recovery process can be relatively reduced, and the stability of front-end service of the object storage device cluster in the data recovery process is further ensured.
On the basis of the above examples, the present invention also provides a series of preferred embodiments as follows.
Referring to fig. 2, an embodiment of the present invention discloses a data recovery method, which is applied to an object storage device in an object storage device cluster, and includes:
step S20: and acquiring the rated recovery rate of the object storage device cluster and the total amount of data to be recovered of the object storage device cluster.
Step S21: and counting the local data volume of the local data to be recovered, and calculating to obtain the ratio of the local data volume to the total data to be recovered.
Step S22: and calculating to obtain the estimated rate according to the ratio and the rated recovery rate.
Step S23: and judging whether a history recovery period adjacent to the current recovery period exists, if so, sequentially executing the steps S24 to S25, and otherwise, executing the step S26.
Step S24: and acquiring a historical estimated speed and a historical actual speed corresponding to the historical recovery period.
Step S25: and adjusting the estimated rate according to the historical estimated rate and the historical actual rate, and executing data recovery operation on the local data to be recovered based on the adjusted estimated rate in the current recovery period.
Step S26: and performing data recovery operation on the local data to be recovered based on the estimated rate in the current recovery period.
It should be noted that the important point of this embodiment is that the recovery operation of the object storage device on the data is performed in units of recovery cycles, and the estimated rates generated in each recovery cycle are respectively used in each recovery cycle, that is, the estimated rates used in each recovery cycle of the object storage device during the data recovery process may be different. In addition, another important point of this embodiment is that after the estimated rate is generated, it is further determined whether a history recovery period adjacent to the current recovery period exists, that is, it is determined whether a previous recovery period of the current recovery period exists, if so, the estimated rate obtained in the current recovery period is further adjusted according to a history estimated speed and a history actual speed corresponding to the history recovery period, and then a data recovery operation is performed on the local data to be recovered at the adjusted estimated rate.
In this embodiment, it is considered that, when performing data recovery, the object storage device is often performed in units of data objects, and an integer number of data objects must be recovered in one recovery period, so that there may be a difference between an actual rate and an estimated rate, which is an amount of data actually recovered in one recovery period, and therefore, when performing data recovery in a current recovery period, this embodiment determines whether there is a history recovery period adjacent to the current recovery period, and then adjusts the estimated rate of the current recovery period according to the history estimated rate and the history actual rate of the history recovery period, so as to ensure balance of recovery rates of the respective recovery periods, further reduce occupation degree of system resources, and ensure stability of front-end services of the object storage device cluster during data recovery.
On the basis of the above embodiment, as a preferred implementation, adjusting the estimated rate according to the historical estimated rate and the historical actual rate includes:
calculating a speed difference value between the historical estimated speed and the historical actual speed;
when the historical estimated speed is greater than the historical actual speed, increasing the difference of the estimated speed by an acceleration rate;
and when the historical estimated speed is smaller than the historical actual speed, reducing the speed difference of the estimated speed.
It should be noted that, in this embodiment, the estimated rate of the current recovery period is adjusted according to a rate difference between the historical estimated rate and the historical actual rate, and since the rate difference between the historical estimated rate and the historical actual rate reflects a difference between the actual rate and the theoretical rate in the last recovery period of the current recovery period, the estimated rate is increased or decreased by the rate difference in the current recovery period according to a size relationship between the historical estimated rate and the historical actual rate in the historical recovery period, so as to further ensure the balance of the recovery rates of the object storage device in each recovery period.
On the basis of the above embodiment, as a preferred implementation, the method further includes:
if the current recovery period is finished, judging whether local data to be recovered still exist;
if the local data to be restored still exist, recording the estimated speed and the actual speed of the current restoration cycle to generate a new estimated speed of the next restoration cycle, and performing data restoration operation on the local data to be restored at the new estimated speed in the next restoration cycle;
and if the local data to be recovered does not exist, stopping the data recovery process.
It should be noted that the key point of this embodiment is to further determine whether there is still local data to be restored that has not been restored after the current restoration cycle is ended, and if there is still local data to be restored that has not been restored, then record the estimated rate and the actual rate obtained in the current restoration cycle, so that in the next restoration cycle, the estimated rate of the next restoration cycle is correspondingly adjusted based on the estimated rate and the actual rate of the current restoration cycle, so as to generate a new estimated rate of the next restoration cycle, and then perform a restoration operation on the local data to be restored in the next restoration cycle at the new estimated rate, thereby further ensuring the balance of the restoration rates of the respective restoration cycles of the object storage device.
In addition, on the basis of the foregoing embodiment, as a preferred implementation manner, the obtaining a rated recovery rate of the target storage device cluster and a total amount of data to be recovered of the target storage device cluster includes:
and acquiring the rated recovery rate of the object storage equipment cluster set by a user and the total amount of data to be recovered of the object storage equipment cluster monitored and transmitted by the monitoring node.
It should be noted that the important point of the present embodiment is that the rated recovery rate of the object storage device cluster is preset by a user, a monitoring node is preset in the object storage device cluster, and the total amount of data to be recovered of the object storage device cluster is transferred to the object storage devices in the object storage device cluster through the monitoring node. The monitoring node monitors the data volume to be recovered of each object storage device in the object storage device cluster, summarizes the data volume to be recovered of each object storage device, generates the total data volume to be recovered of the object storage device cluster, and then sends the total data volume to be recovered to each object storage device in the object storage device cluster. The overall controllability of the data recovery process can be relatively ensured by presetting the rated recovery rate of the object storage device cluster by a user, and the overall accuracy of the data recovery process can be relatively ensured by counting and transmitting the total amount of data to be recovered through the preset monitoring nodes.
Referring to fig. 3, an embodiment of the present invention discloses an object storage device, which is applied to an object storage device cluster, and includes:
an obtaining module 10, configured to obtain a rated recovery rate of the object storage device cluster and a total amount of data to be recovered of the object storage device cluster;
the statistical module 11 is configured to count a local data amount of local data to be recovered, and calculate a ratio of the local data amount to a total amount of the data to be recovered;
and the data recovery module 12 is configured to calculate an estimated rate according to the percentage value and the rated recovery rate, and perform data recovery operation on the local data to be recovered based on the estimated rate.
The object storage device provided by the invention is applied to an object storage device cluster, firstly, the rated recovery rate and the total amount of data to be recovered of the object storage device cluster are obtained, then, the local data volume of the local data to be recovered in the object storage device is counted, the ratio of the local data volume in the total amount of the data to be recovered is calculated, finally, the estimated rate corresponding to the object storage device is calculated through the ratio and the rated recovery rate, and then, the data recovery operation is carried out on the local data to be recovered based on the estimated rate. Because each object storage device in the cluster divides the estimated rate of the corresponding ratio in the rated recovery rate of the cluster according to the proportion of the data volume required to be recovered to the overall data volume to be recovered of the cluster, and is used for performing data recovery operation on the local data to be recovered, when the object storage device cluster performs data recovery, the recovery rate of each object storage device in the cluster is limited by the rated rate of the cluster and the ratio of the local data to be recovered to the overall data volume to be recovered of the cluster, so that the occupation of system resources by the object storage device cluster in the data recovery process can be relatively reduced, and the stability of front-end service of the object storage device cluster in the data recovery process is further ensured.
On the basis of the object storage device, as a preferred embodiment, the data recovery module includes:
the period recovery module is used for performing data recovery operation on the local data to be recovered based on the estimated rate in the current recovery period;
the object storage device further includes:
the period judging module is used for judging whether a history recovery period adjacent to the current recovery period exists or not, if so, the history obtaining module and the adjusting module are sequentially called, and otherwise, the period recovery module is called;
the history acquisition module is used for acquiring a history estimated speed and a history actual speed corresponding to a history recovery period;
and the adjusting module is used for adjusting the estimated rate according to the historical estimated rate and the historical actual rate and calling the period recovery module by using the adjusted estimated rate.
As a preferred embodiment, the adjusting module includes:
the difference value calculating module is used for calculating the speed difference value between the historical estimated speed and the historical actual speed;
the increasing module is used for increasing the difference value of the estimated speed when the historical estimated speed is greater than the historical actual speed;
and the reduction module is used for reducing the difference of the estimated speed when the historical estimated speed is less than the historical actual speed.
As a preferred embodiment, the object storage device further includes:
the cycle ending module is used for judging whether local data to be recovered still exists or not if the current recovery cycle is ended, if so, calling the recording module, and otherwise, calling the recovery stopping module;
the recording module is used for recording the estimated speed and the actual speed of the current recovery period so as to generate a new estimated speed of the next recovery period, and performing data recovery operation on the local data to be recovered at the new estimated speed in the next recovery period;
and the stopping and recovering module is used for stopping the data recovering process.
Further, the present invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the data recovery method as described above.
The computer-readable storage medium provided by the invention is applied to the object storage equipment in the object storage equipment cluster, firstly, the rated recovery rate and the total amount of data to be recovered of the object storage equipment cluster are obtained, then, the local data volume of the local data to be recovered in the object storage equipment is counted, the ratio of the local data volume in the total amount of the data to be recovered is calculated, finally, the estimated rate corresponding to the object storage equipment is calculated through the ratio and the rated recovery rate, and then, the data recovery operation is carried out on the local data to be recovered based on the estimated rate. Because each object storage device in the cluster divides the estimated rate of the corresponding ratio in the rated recovery rate of the cluster according to the proportion of the data volume required to be recovered to the overall data volume to be recovered of the cluster, and is used for performing data recovery operation on the local data to be recovered, when the object storage device cluster performs data recovery, the recovery rate of each object storage device in the cluster is limited by the rated rate of the cluster and the ratio of the local data to be recovered to the overall data volume to be recovered of the cluster, so that the occupation of system resources by the object storage device cluster in the data recovery process can be relatively reduced, and the stability of front-end service of the object storage device cluster in the data recovery process is further ensured.
The data recovery method, the data recovery device and the storage medium provided by the invention are described in detail above. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A data recovery method is applied to an object storage device in an object storage device cluster, and comprises the following steps:
acquiring a rated recovery rate of the object storage equipment cluster and the total amount of data to be recovered of the object storage equipment cluster;
counting the local data volume of local data to be recovered, and calculating to obtain the ratio of the local data volume in the total data volume to be recovered;
and calculating to obtain an estimated rate according to the ratio and the rated recovery rate, and executing data recovery operation on the local data to be recovered based on the estimated rate.
2. The data recovery method of claim 1, wherein the performing the data recovery operation on the local data to be recovered based on the pre-estimated rate comprises:
performing the data recovery operation on the local data to be recovered based on the estimated rate in the current recovery period;
before the performing the data recovery operation on the local data to be recovered based on the estimated rate in the current recovery period, the method further includes:
judging whether a history recovery period adjacent to the current recovery period exists or not;
if a history recovery period adjacent to the current recovery period exists, acquiring a history estimated speed and a history actual speed corresponding to the history recovery period;
adjusting the estimated rate according to the historical estimated rate and the historical actual rate, and executing the data recovery operation on the local data to be recovered based on the estimated rate in the current recovery period by using the adjusted estimated rate;
and if the historical recovery period adjacent to the current recovery period does not exist, executing the data recovery operation on the local data to be recovered based on the estimated speed in the current recovery period.
3. The data recovery method of claim 2, wherein said adjusting the estimated rate based on the historical estimated rate and the historical actual rate comprises:
calculating a speed difference value between the historical estimated speed and the historical actual speed;
when the historical estimated speed is larger than the historical actual speed, increasing the speed difference value for the estimated speed;
and when the historical estimated speed is smaller than the historical actual speed, reducing the speed difference for the estimated speed.
4. The data recovery method of claim 2, wherein the method further comprises:
if the current recovery period is finished, judging whether the local data to be recovered still exists;
if the local data to be restored still exists, recording the estimated speed and the actual speed of the current restoration cycle to generate a new estimated speed of the next restoration cycle, and performing the data restoration operation on the local data to be restored at the new estimated speed in the next restoration cycle;
and if the local data to be recovered does not exist, stopping the data recovery process.
5. The method for recovering data according to claims 1 to 4, wherein the obtaining of the rated recovery rate of the object storage device cluster and the total amount of data to be recovered of the object storage device cluster comprises:
and acquiring the rated recovery rate of the object storage equipment cluster set by a user and the total amount of the data to be recovered of the object storage equipment cluster monitored and transmitted by the monitoring node.
6. An object storage device applied to an object storage device cluster, comprising:
the acquisition module is used for acquiring the rated recovery rate of the object storage equipment cluster and the total amount of data to be recovered of the object storage equipment cluster;
the statistical module is used for counting the local data volume of the local data to be recovered and calculating to obtain the ratio of the local data volume in the total data volume to be recovered;
and the data recovery module is used for calculating an estimated rate according to the ratio value and the rated recovery rate and executing data recovery operation on the local data to be recovered based on the estimated rate.
7. The object storage device of claim 6, wherein the data recovery module comprises:
the period recovery module is used for executing the data recovery operation on the local data to be recovered based on the estimated rate in the current recovery period;
the object storage device further includes:
the period judging module is used for judging whether a history recovery period adjacent to the current recovery period exists or not, if so, the history obtaining module and the adjusting module are sequentially called, and if not, the period recovering module is called;
the history acquisition module is used for acquiring a history estimated rate and a history actual rate corresponding to the history recovery period;
the adjusting module is used for adjusting the estimated rate according to the historical estimated rate and the historical actual rate and calling the period recovery module by using the adjusted estimated rate.
8. The object storage device of claim 7, wherein the adjustment module comprises:
the difference value calculating module is used for calculating the speed difference value between the historical estimated speed and the historical actual speed;
an increasing module, configured to increase the rate difference for the estimated rate when the historical estimated rate is greater than the historical actual rate;
and the reducing module is used for reducing the speed difference value of the estimated speed when the historical estimated speed is smaller than the historical actual speed.
9. The object storage device of claim 7, further comprising:
the cycle ending module is used for judging whether the local data to be recovered still exists or not if the current recovery cycle is ended, if so, calling the recording module, and otherwise, calling the recovery stopping module;
the recording module is configured to record the estimated rate and the actual rate of the current recovery period, so as to generate a new estimated rate of a next recovery period, and perform the data recovery operation on the local data to be recovered at the new estimated rate in the next recovery period;
and the stopping and recovering module is used for stopping the data recovering process.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the data recovery method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911161216.5A CN110990195A (en) | 2019-11-24 | 2019-11-24 | Data recovery method, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911161216.5A CN110990195A (en) | 2019-11-24 | 2019-11-24 | Data recovery method, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110990195A true CN110990195A (en) | 2020-04-10 |
Family
ID=70086117
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911161216.5A Withdrawn CN110990195A (en) | 2019-11-24 | 2019-11-24 | Data recovery method, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110990195A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008310591A (en) * | 2007-06-14 | 2008-12-25 | Nomura Research Institute Ltd | Cluster system, computer, and failure recovery method |
| CN103106152A (en) * | 2012-12-13 | 2013-05-15 | 深圳先进技术研究院 | Data scheduling method based on gradation storage medium |
| CN105335251A (en) * | 2015-09-23 | 2016-02-17 | 浪潮(北京)电子信息产业有限公司 | Fault recovery method and system |
| CN108196983A (en) * | 2017-12-29 | 2018-06-22 | 合肥赛猊腾龙信息技术有限公司 | A kind of method and system for improving Backup and Restore speed |
| CN108958975A (en) * | 2018-06-29 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of method, device and equipment controlling data resume speed |
| CN109117306A (en) * | 2018-07-24 | 2019-01-01 | 广东浪潮大数据研究有限公司 | A kind of method and device based on object read-write time delay adjustment data resume speed |
| CN109144782A (en) * | 2018-08-22 | 2019-01-04 | 郑州云海信息技术有限公司 | A kind of data reconstruction method and device |
-
2019
- 2019-11-24 CN CN201911161216.5A patent/CN110990195A/en not_active Withdrawn
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008310591A (en) * | 2007-06-14 | 2008-12-25 | Nomura Research Institute Ltd | Cluster system, computer, and failure recovery method |
| CN103106152A (en) * | 2012-12-13 | 2013-05-15 | 深圳先进技术研究院 | Data scheduling method based on gradation storage medium |
| CN105335251A (en) * | 2015-09-23 | 2016-02-17 | 浪潮(北京)电子信息产业有限公司 | Fault recovery method and system |
| CN108196983A (en) * | 2017-12-29 | 2018-06-22 | 合肥赛猊腾龙信息技术有限公司 | A kind of method and system for improving Backup and Restore speed |
| CN108958975A (en) * | 2018-06-29 | 2018-12-07 | 郑州云海信息技术有限公司 | A kind of method, device and equipment controlling data resume speed |
| CN109117306A (en) * | 2018-07-24 | 2019-01-01 | 广东浪潮大数据研究有限公司 | A kind of method and device based on object read-write time delay adjustment data resume speed |
| CN109144782A (en) * | 2018-08-22 | 2019-01-04 | 郑州云海信息技术有限公司 | A kind of data reconstruction method and device |
Non-Patent Citations (1)
| Title |
|---|
| 胡至洵: "面向分布式文件存储系统的数据恢复策略", 《能源与环保》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107577441B (en) | OSD slow disc processing method, system, device and computer storage medium | |
| CN111142942B (en) | Window data processing method and device, server and storage medium | |
| CN107273182B (en) | Method and system for dynamically expanding virtual machine resources | |
| CN113032157B (en) | Automatic intelligent server capacity expansion and reduction method and system | |
| CN112165436B (en) | Flow control method, device and system | |
| US10783005B2 (en) | Component logical threads quantity adjustment method and device | |
| CN110795284B (en) | Data recovery method, device and equipment and readable storage medium | |
| CN111277640B (en) | User request processing method, device, system, computer equipment and storage medium | |
| CN106789141A (en) | A kind of gateway device failure processing method and processing device | |
| CN114866563A (en) | Capacity expansion method, device, system and storage medium | |
| CN114500128B (en) | Flow control charging method, device, system, electronic equipment, medium and product | |
| CN113079062B (en) | Resource adjusting method and device, computer equipment and storage medium | |
| CN107426012B (en) | Fault recovery method and device based on super-fusion architecture | |
| CN110855484B (en) | Method, system, electronic device and storage medium for automatically detecting traffic change | |
| CN117472282A (en) | Sequence number generation method and device based on distributed storage system and electronic equipment | |
| CN112615795A (en) | Flow control method and device, electronic equipment, storage medium and product | |
| CN110990195A (en) | Data recovery method, equipment and storage medium | |
| KR20150049850A (en) | Apparatus and Method for Changing Resource Using Pattern Information, and Recording Medium Using the Same | |
| CN114490091B (en) | Method and device for monitoring rule engine performance in industrial data acquisition management system | |
| WO2016101564A1 (en) | Call bill processing method and device | |
| CN103312529B (en) | The method for supervising of systemic-function, device and supervisory control system | |
| WO2018001114A1 (en) | Method and device for processing data | |
| WO2016165471A1 (en) | Method and apparatus for collecting index of hadoop trunking component | |
| TWI735520B (en) | Method and device for adjusting the number of component logic threads | |
| CN111061585A (en) | Data recovery method, device and equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200410 |
|
| WW01 | Invention patent application withdrawn after publication |