CN107317864B - Data equalization method and device of storage equipment - Google Patents
Data equalization method and device of storage equipment Download PDFInfo
- Publication number
- CN107317864B CN107317864B CN201710517682.7A CN201710517682A CN107317864B CN 107317864 B CN107317864 B CN 107317864B CN 201710517682 A CN201710517682 A CN 201710517682A CN 107317864 B CN107317864 B CN 107317864B
- Authority
- CN
- China
- Prior art keywords
- preset
- minimum
- object storage
- weight
- storage units
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000008569 process Effects 0.000 claims description 11
- 230000008859 change Effects 0.000 claims description 9
- 239000013585 weight reducing agent Substances 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 230000001960 triggered effect Effects 0.000 claims description 4
- 230000007547 defect Effects 0.000 abstract description 3
- 230000004048 modification Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 238000011084 recovery Methods 0.000 description 2
- 230000010076 replication Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000005034 decoration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Abstract
The invention discloses a data balancing method of a storage system, which comprises the steps of counting the number of object storage devices in the storage system and the number of minimum storage units in each object storage device; calculating the sum of the minimum storage unit number in each object storage device; calculating the distribution average value of the minimum storage unit; judging whether the minimum storage units are uniformly distributed according to a preset balance principle, if not, judging whether all preset weight adjustment is completed, if not, performing weight adjustment on the object storage equipment according to the preset adjustment principle, and after adjustment, redistributing the minimum storage units; and re-counting the number of the minimum storage units in each object storage device, and returning to the balance judgment operation. The invention also provides a data balancing device of the storage system. The invention improves the defect of unbalanced data distribution by adjusting the OSD weight, ensures that the OSD capacities in the storage pool are basically the same, and improves the overall capacity utilization rate of the system.
Description
Technical Field
The invention relates to the technical field of computer clusters, in particular to a data balancing method of a storage device and a data balancing device of the storage device.
Background
With the increase of internet traffic, the rapid increase of access amount and data traffic, the processing intensity of each core part of the storage system is relatively increased, so that the workload of the system is increased. In the storage system, a consistent Hash algorithm and a CRUSH (Controlled Replication and Untensible Scalable Replication algorithm) algorithm are used for calculating data distribution, in the process, the CRUSH algorithm maps PG (minimum storage unit) to a group of OSD (object storage device), and each OSD is distributed with a plurality of equal PG. However, in an actual application environment, the PGs are only pseudo-randomly distributed on the OSDs according to the CRUSH algorithm, so that the actual PG distribution cannot achieve perfect balance, and some OSDs have a larger amount of PG distributed thereon and some OSDs have a smaller amount of PG. When data is written into the distributed file system, the intuitive phenomenon is that the data on the OSD is not distributed uniformly, which causes that the OSD capacity in the storage pool has larger difference, the whole capacity utilization rate of the system is lower, and the balance is poor.
Therefore, how to provide a data balancing method and apparatus for a storage device to improve the imbalance of data distribution in a target storage device is a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The invention aims to provide a data balancing method and a data balancing device of storage equipment, which overcome the defect of unbalanced data distribution by adjusting OSD weight, ensure that OSD capacities in a storage pool are basically the same and improve the overall capacity utilization rate of a system.
In order to solve the above technical problem, the present invention provides a data equalization method for a storage device, including:
s 1: counting the number of object storage devices in the storage system and the number of minimum storage units in each object storage device;
s 2: calculating the sum of the minimum storage unit quantity in each object storage device; calculating the ratio of the sum of the minimum storage unit number to the object storage equipment number to obtain the distribution average value of the minimum storage unit;
s 3: judging whether the minimum storage units are uniformly distributed according to a preset balance principle, the number of the minimum storage units in each object storage device and the distribution average value of the minimum storage units, if not, entering a step s4, otherwise, ending the operation;
s 4: judging whether all preset weight adjustment is finished, if not, carrying out weight adjustment on the object storage equipment according to a preset adjustment principle, and redistributing the minimum storage units after adjustment; and re-counting the number of the minimum storage units in each object storage device, and returning to the step s 3.
Preferably, the preset adjustment principle includes:
and sorting the number of the minimum storage units in each object storage device from large to small, selecting a plurality of object storage devices according to a preset selection principle and the sorting of the number of the minimum storage units, and reducing the weight of the selected object storage devices according to the preset weight reduction principle.
Preferably, the preset equalization principle is specifically as follows:
judging whether the MAX _ PG distribution average value is met or not and a preset threshold value, if not, judging that the distribution is not uniform, and entering a step s4, otherwise, ending the operation;
wherein the MAX _ PG is the number of the smallest storage units on the object storage device where the smallest storage units are most distributed.
Preferably, the preset threshold is any value between 102% and 110%.
Preferably, the step s4 of determining whether all preset weight adjustments are completed further includes:
when all preset weight adjustments corresponding to the current preset threshold are completed, adding a preset change value to the current preset threshold, judging whether the current preset threshold is greater than a preset extreme value, if not, returning to the step s 3; otherwise, the operation is ended.
Preferably, after the step s3 judges that the minimum storage unit distribution is not uniform according to the preset equalization rule, the method further includes:
the weights of the object storage devices are adjusted to initial values, and the process then proceeds to step s 4.
Preferably, the method comprises the following steps:
the counting module is used for counting the number of the object storage devices in the storage system and the number of the minimum storage units in each object storage device;
a calculation module, configured to calculate a sum of the minimum number of storage units in each of the object storage devices; calculating the ratio of the sum of the minimum storage unit number to the object storage equipment number to obtain the distribution average value of the minimum storage unit;
the balance judging module is used for judging whether the minimum storage units are uniformly distributed or not according to a preset balance principle, the number of the minimum storage units in each object storage device and the distribution average value of the minimum storage units, if so, the weight adjusting module is triggered, and otherwise, the operation is ended;
the weight adjusting module is used for judging whether all preset weight adjustments are finished, if not, carrying out weight adjustment on the object storage equipment according to a preset adjusting principle, and redistributing the adjusted minimum storage units; and re-counting the number of the minimum storage units in each object storage device, and returning to the judgment module.
Preferably, the weight adjusting module includes:
the sorting unit is used for sorting the number of the minimum storage units in the object storage equipment from large to small;
the selecting unit is used for selecting a plurality of object storage devices according to a preset selecting principle and the sorting of the minimum storage unit quantity;
and the weight adjusting unit is used for reducing the weight of the selected object storage equipment according to the preset weight reduction principle.
Preferably, the equalization determining module includes:
the calculation unit is used for calculating the product of the distribution average value of the minimum storage unit and a preset threshold value;
a judging unit, configured to judge whether the number of the smallest storage units on the object storage device where the smallest storage units are most distributed is smaller than or equal to the product, and if so, judge that the distribution is not uniform, and trigger the weight adjusting module; otherwise, the operation is ended.
Preferably, the weight adjusting module further comprises:
the threshold adjusting unit is used for increasing a preset change value on the current preset threshold when all preset weight adjustments corresponding to the current preset threshold are finished, judging whether the current preset threshold is greater than a preset extreme value or not, and returning to the balance judging module if the current preset threshold is not greater than the preset extreme value; otherwise, the operation is ended.
The invention provides a data equalization method and a data equalization device of storage equipment, which comprise the steps of counting the distribution condition of PG on OSD and calculating the distribution average value of PG as a reference, judging whether PG is uniformly distributed according to the distribution average value of PG and a preset equalization principle, and adjusting the weight of OSD for OSD groups with non-uniform PG distribution according to the preset adjustment principle to redistribute part of PG on other OSD, so that the imbalance caused by pseudo-random distribution of PG on OSD according to CRUSH algorithm is weakened, the capacity rates of OSD in a storage pool are basically the same, the integral capacity utilization rate of the storage system is further improved, and the cost performance of the storage system is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed in the prior art and the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a flow chart of a process of a data balancing method of a storage device according to the present invention;
FIG. 2 is a process flow diagram of an embodiment of a method for data balancing of a storage device according to the present invention;
fig. 3 is a schematic structural diagram of a data equalization apparatus of a storage device according to the present invention.
Detailed Description
The core of the invention is to provide a data balancing method and device of a storage device, which overcomes the defect of unbalanced data distribution by adjusting the weight of OSD, ensures that the OSD capacities in a storage pool are basically the same, and improves the capacity utilization rate of the whole system.
The invention provides a data equalization method of a storage device, and referring to fig. 1, fig. 1 is a flowchart of a process of the data equalization method of the storage device provided by the invention. The method comprises the following steps: s 1: counting the number of object storage devices in the storage system and the number of minimum storage units in each object storage device;
the data storage process of the storage device is necessarily based on the establishment of the storage pool of the storage system, and is not described herein again.
s 2: calculating the sum of the minimum storage unit number in each object storage device; calculating the ratio of the sum of the minimum storage unit number to the object storage equipment number to obtain the distribution average value of the minimum storage unit; the sum of the minimum storage unit numbers may be calculated according to the storage pool data redundancy policy and the number of the object storage devices, or may be accumulated by calculating the minimum storage unit numbers in the object storage devices according to the actual distribution. The method of the present invention for summing the minimum number of memory cells is not limited.
Obviously, the distribution average value of the minimum storage unit is not changed under the normal condition of the number of the group of object storage devices, so that the distribution average value of the minimum storage unit can be called after the minimum storage unit is redistributed, and the calculation is not needed again.
s 3: judging whether the distribution of the minimum storage units is uniform or not according to a preset balance principle, the number of the minimum storage units in each object storage device and the distribution average value of the minimum storage units, if not, entering a step s4, otherwise, ending the operation; s 4: judging whether all preset weight adjustment is finished, if not, carrying out weight adjustment on the object storage equipment according to a preset adjustment principle, and redistributing the minimum storage units after adjustment; and re-counting the number of the minimum storage units in each object storage device, and returning to the step s 3.
The condition for completing all preset weight adjustments includes that the weight adjustment times reach preset times, for example, the preset times is 25 times, then a group of operations performs weight adjustment for at most 25 times, and even if the storage device does not reach a balanced state at this time, the weight adjustment operation is not continued. The condition for completing the adjustment of all the preset weights may also be that a preset lower limit is reached, for example, when the weights of a certain proportion of the minimum memory cells reach the preset lower limit, the continuous adjustment operation may also be terminated. Of course, the adjustment of all the preset weights in the present invention is not limited to the constraints of the preset times and the preset lower limit, and the present invention does not limit the specific form and the specific requirements of the preset weights.
In a preferred embodiment of the present invention, the preset adjustment principle comprises:
the number of the minimum storage units in each object storage device is sorted according to the sequence from large to small, a plurality of object storage devices are selected according to a preset selection principle and the sorting of the number of the minimum storage units, and the weight of the selected object storage devices is reduced according to a preset weight reduction principle.
The preset selection principle may specifically be to adjust the weight of all the object storage devices whose minimum number of storage units exceeds 110% of the distribution average, or may select the object storage devices according to a proportion, which are the first 5 sorted object storage devices, or select the object storage device containing the largest number of minimum storage units to adjust. Of course, the preset adjustment principle is not limited to the above.
In addition, the preset weight reduction principle may specifically be to reduce the weight of the object storage device according to a preset interval, for example, reduce the weight of the object storage device by 0.1 each time, or randomly reduce the weight of the object storage device, and of course, the present invention does not limit the specific value of the preset interval nor the specific implementation manner of the preset weight reduction principle.
As such, the present invention is not limited to the preset adjustment principle, and any modifications and decorations known to those skilled in the art can be made only by the present invention
The above embodiment improves the imbalance of the distribution of the minimum storage units by adjusting the weight of the target storage device in the target storage device information table to promote the redistribution of part of the minimum storage units, thereby improving the imbalance of the data distribution. It is further added that, after the step s3 judges that the minimum memory cell distribution is not uniform according to the preset equalization principle, the method further includes:
the weights of the object storage devices are adjusted to initial values, and the process proceeds to step s 4.
The above-mentioned process of weight recovery can avoid that the excessive adjustment or modification of the OSD (object storage device) weight in the OSDMap (object storage device information table) may have a bad influence on the distribution of the minimum storage unit or the system function of the storage system.
Another preferred embodiment of the present invention defines the preset equalization principle, and the specific embodiment of step s3 is as follows:
judging whether the MAX _ PG distribution average value is equal to or less than a preset threshold value, if not, judging that the distribution is not uniform, and entering a step s41, otherwise, ending the operation;
the MAX _ PG is the number of the minimum memory cells on the object storage device with the most distributed minimum memory cells.
The preset equalization principle in this embodiment is to determine whether the MAX _ PG distribution average value is satisfied with a preset threshold, but this is only one preferable scheme in the embodiment of the present invention, and the present invention does not limit the specific requirements of the preset equalization principle. It should be noted that simple modifications and inferences made in the principles of the present invention's predictive balancing are within the scope of the invention.
It should be noted that the preset threshold may be understood as a set of balance targets for the balance operation of the storage device, and may be set by a user according to a requirement of the capacity utilization rate of the storage system. Preferably, the preset threshold is any value between 102% and 110%, for example, the preset threshold may be 102%. The present invention does not limit the specific value of the preset threshold.
Based on the above embodiments, the present invention further provides a preferred solution, and referring to fig. 2, fig. 2 is a flowchart of a process of a specific embodiment of a data balancing method for a storage device provided by the present invention. In the preferred embodiment shown in fig. 2, the step of determining whether all preset weight adjustments are completed in step s4 further includes:
when all preset weight adjustments corresponding to the current preset threshold are completed, increasing a preset change value on the current preset threshold, judging whether the current preset threshold is greater than a preset extreme value, and if not, returning to the step s 3; otherwise, the operation is ended.
The specific embodiment is as follows:
s 1: counting the number of object storage devices in the storage system and the number of minimum storage units in each object storage device;
s 21: calculating the sum of the minimum storage unit number in each object storage device;
s 22: calculating the distribution average value of the minimum storage unit;
s 3: judging whether the MAX _ PG distribution average value is met or not and a preset threshold value, if not, judging that the distribution is not uniform, and entering a step s 41;
the MAX _ PG is the number of the minimum memory cells on the object storage device with the most distributed minimum memory cells.
s 41: judging whether all preset weight adjustments corresponding to the current preset threshold are finished, if not, entering step s 42; if so, proceed to s 44;
s 42: carrying out weight adjustment on the object storage equipment according to a preset adjustment principle, and redistributing the minimum storage units;
s 43: counting the number of the minimum storage units in each object storage device again, and returning to the step s 3;
s 44: increasing a preset change value by a preset threshold value;
the preset variation value may be a preset percentage, for example, the preset variation value is 1%. The invention does not limit the numerical value, the form, the setting standard and the like of the preset change value.
s 45: and judging whether the current preset threshold is larger than a preset extreme value or not, if not, entering the step s3, and if not, finishing the operation.
The preset threshold may be set by a user, and may be understood as a maximum acceptable equalization target, for example, the preset threshold may be 110%, and the invention is not limited to the preset threshold.
Assuming that the initial value of the preset threshold may be 102%, the preset variation value is 1%, and the preset extremum is 110%, the variation process of the preset threshold may be understood as trying 103% of the equalization targets when the equalization targets do not reach 102%, and sequentially increasing until 110% of the equalization targets still cannot be equalized, and determining that equalization fails. It is obvious that the preset threshold of the present invention can be used as long as the equalization target satisfying the basic requirements of the system is satisfied.
In another embodiment, the step s4 of determining whether all preset weight adjustments are completed further includes:
when all preset weight adjustments corresponding to the current preset threshold are judged to be completed, judging whether the current preset threshold reaches a preset extreme value, if not, adding a preset change value to the current preset threshold, and returning to the step s 3; otherwise, the operation is ended.
The invention provides a data balancing method of storage equipment, which comprises the steps of counting the distribution condition of PG on OSD, calculating the distribution average value of PG as a reference, judging whether PG is uniformly distributed according to the distribution average value of PG and a preset balancing principle, adjusting the weight of OSD for the OSD group with non-uniform PG distribution according to the preset adjusting principle, so that part of PG is redistributed to the rest of OSD, reducing unbalance caused by pseudo-random distribution of PG on OSD according to CRUSH algorithm, ensuring that the capacity rates of OSD in a storage pool are basically the same, further improving the integral capacity utilization rate of the storage system, and simultaneously improving the cost performance of the storage system.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a data equalization apparatus of a storage device provided in the present invention, where the apparatus includes:
the statistical module is used for counting the number of the object storage devices in the storage system and the number of the minimum storage units in each object storage device;
the calculation module is used for calculating the sum of the minimum storage unit number in each object storage device; calculating the ratio of the sum of the minimum storage unit number to the object storage equipment number to obtain the distribution average value of the minimum storage unit;
the balance judging module is used for judging whether the minimum storage units are uniformly distributed or not according to a preset balance principle, the number of the minimum storage units in each object storage device and the distribution average value of the minimum storage units, if not, the weight adjusting module is triggered, and if not, the operation is ended;
the weight adjusting module is used for judging whether all preset weight adjustments are finished, if not, the weight adjustment is carried out on the object storage equipment according to a preset adjusting principle, and the adjusted minimum storage units are redistributed; and re-counting the number of the minimum storage units in each object storage device, and returning to the judgment module.
As a preferable solution, the weight adjusting module includes:
the sorting unit is used for sorting the number of the minimum storage units in each object storage device in a descending order;
the selecting unit is used for selecting a plurality of object storage devices according to a preset selecting principle and the sequencing of the minimum storage unit number;
and the weight adjusting unit is used for reducing the weight of the selected object storage equipment according to a preset weight reduction principle.
In an embodiment of the present invention, the equalization determining module includes:
the calculation unit is used for calculating the product of the distribution average value of the minimum storage unit and a preset threshold value;
the judging unit is used for judging whether the number of the minimum storage units on the object storage equipment with the most distributed minimum storage units is smaller than or equal to the product, if so, the distribution is judged to be non-uniform, and the weight adjusting module is triggered; otherwise, the operation is ended.
In a preferred embodiment of the present invention, the weight adjusting module further comprises:
the threshold adjusting unit is used for increasing a preset change value on the current preset threshold when all preset weight adjustments corresponding to the current preset threshold are finished, judging whether the current preset threshold is greater than a preset extreme value or not, and returning to the balance judging module if the current preset threshold is not greater than the preset extreme value; otherwise, the operation is ended.
As a preferable mode, the apparatus further comprises:
and the weight recovery module is used for adjusting the weight of each object storage device to an initial value after the judging module judges that the minimum storage units are unevenly distributed according to the preset equalization principle, and then entering the weight adjusting module.
The invention provides a data balancing device of a storage device, which comprises a step of counting the distribution condition of PG on OSD and calculating the distribution average value of PG as a reference, a step of judging whether PG is uniformly distributed according to the distribution average value of PG and a preset balancing principle, and a step of adjusting the weight of OSD for OSD groups with nonuniform PG distribution according to the preset adjusting principle so that part of PG is redistributed to other OSD, thereby weakening unbalance caused by pseudo-random distribution of PG on OSD according to CRUSH algorithm, ensuring that the capacity rates of OSD in a storage pool are basically the same, further improving the integral capacity utilization rate of the storage system, and simultaneously improving the cost performance of the storage system.
The data equalization method and apparatus for a storage device provided by the present invention are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. A method for data equalization in a storage system, comprising:
s 1: counting the number of object storage devices in the storage system and the number of minimum storage units in each object storage device;
s 2: calculating the sum of the minimum storage unit quantity in each object storage device; calculating the ratio of the sum of the minimum storage unit number to the object storage equipment number to obtain the distribution average value of the minimum storage unit;
s 3: judging whether the minimum storage units are uniformly distributed according to a preset balance principle, the number of the minimum storage units in each object storage device and the distribution average value of the minimum storage units, if not, entering a step s4, otherwise, ending the operation;
s 4: judging whether all preset weight adjustment is finished, if not, carrying out weight adjustment on the object storage equipment according to a preset adjustment principle, and redistributing the minimum storage units after adjustment; counting the number of the minimum storage units in each object storage device again, and returning to the step s 3; and the condition for finishing all the preset weight adjustment comprises that the weight adjustment times reach preset times.
2. The method of claim 1, wherein the preset adjustment criteria comprises:
and sorting the number of the minimum storage units in each object storage device from large to small, selecting a plurality of object storage devices according to a preset selection principle and the sorting of the number of the minimum storage units, and reducing the weight of the selected object storage devices according to the preset weight reduction principle.
3. The method according to claim 1, wherein the preset equalization principle is specifically:
judging whether the MAX _ PG is not more than the distribution average value and a preset threshold value is met, if not, judging that the distribution is not uniform, and entering a step s4, otherwise, ending the operation;
wherein the MAX _ PG is the number of the smallest storage units on the object storage device where the smallest storage units are most distributed.
4. The method according to claim 3, wherein the predetermined threshold value is any value between 102% and 110%.
5. The method according to claim 3, wherein the step s4 of determining whether all preset weight adjustments are completed further comprises:
when all preset weight adjustments corresponding to the current preset threshold are completed, adding a preset change value to the current preset threshold, judging whether the current preset threshold is greater than a preset extreme value, if not, returning to the step s 3; otherwise, the operation is ended.
6. The method according to any of claims 1 to 5, wherein after the step s3 determining that the minimum storage unit distribution is not uniform according to the preset equalization rule, the method further comprises:
the weights of the object storage devices are adjusted to initial values, and the process then proceeds to step s 4.
7. A data equalization apparatus of a storage system, comprising:
the counting module is used for counting the number of the object storage devices in the storage system and the number of the minimum storage units in each object storage device;
a calculation module, configured to calculate a sum of the minimum number of storage units in each of the object storage devices; calculating the ratio of the sum of the minimum storage unit number to the object storage equipment number to obtain the distribution average value of the minimum storage unit;
the balance judging module is used for judging whether the minimum storage units are uniformly distributed or not according to a preset balance principle, the number of the minimum storage units in each object storage device and the distribution average value of the minimum storage units, if so, the weight adjusting module is triggered, and otherwise, the operation is ended;
the weight adjusting module is used for judging whether all preset weight adjustments are finished, if not, carrying out weight adjustment on the object storage equipment according to a preset adjusting principle, and redistributing the adjusted minimum storage units; counting the number of the minimum storage units in each object storage device again, and returning to the judgment module; and the condition for finishing all the preset weight adjustment comprises that the weight adjustment times reach preset times.
8. The apparatus of claim 7, wherein the weight adjustment module comprises:
the sorting unit is used for sorting the number of the minimum storage units in the object storage equipment from large to small;
the selecting unit is used for selecting a plurality of object storage devices according to a preset selecting principle and the sorting of the minimum storage unit quantity;
and the weight adjusting unit is used for reducing the weight of the selected object storage equipment according to the preset weight reduction principle.
9. The apparatus of claim 7, wherein the equalization determination module comprises:
the calculation unit is used for calculating the product of the distribution average value of the minimum storage unit and a preset threshold value;
a judging unit, configured to judge whether the number of the smallest storage units on the object storage device where the smallest storage units are most distributed is smaller than or equal to the product, and if so, judge that the distribution is not uniform, and trigger the weight adjusting module; otherwise, the operation is ended.
10. The apparatus of claim 9, wherein the weight adjustment module further comprises:
the threshold adjusting unit is used for increasing a preset change value on the current preset threshold when all preset weight adjustments corresponding to the current preset threshold are finished, judging whether the current preset threshold is greater than a preset extreme value or not, and returning to the balance judging module if the current preset threshold is not greater than the preset extreme value; otherwise, the operation is ended.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710517682.7A CN107317864B (en) | 2017-06-29 | 2017-06-29 | Data equalization method and device of storage equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710517682.7A CN107317864B (en) | 2017-06-29 | 2017-06-29 | Data equalization method and device of storage equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107317864A CN107317864A (en) | 2017-11-03 |
| CN107317864B true CN107317864B (en) | 2020-08-21 |
Family
ID=60179721
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710517682.7A Active CN107317864B (en) | 2017-06-29 | 2017-06-29 | Data equalization method and device of storage equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107317864B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109343798A (en) * | 2018-09-25 | 2019-02-15 | 郑州云海信息技术有限公司 | To the method, apparatus and medium of the main PG equilibrium adjustment of distributed memory system |
| CN109558437B (en) * | 2018-11-16 | 2021-01-01 | 新华三技术有限公司成都分公司 | Main OSD (on-screen display) adjusting method and device |
| CN109933285A (en) * | 2019-02-26 | 2019-06-25 | 新华三技术有限公司成都分公司 | The data balancing method and device of distributed storage |
| CN110018799B (en) * | 2019-04-12 | 2020-07-03 | 苏州浪潮智能科技有限公司 | Storage pool PG (packet data) master determination method, device, equipment and readable storage medium |
| CN111090390B (en) * | 2019-10-31 | 2022-06-10 | 平安科技（深圳）有限公司 | Storage allocation method and device of distributed system and computer equipment |
| CN111124309B (en) * | 2019-12-22 | 2022-02-18 | 浪潮电子信息产业股份有限公司 | Method, device and equipment for determining fragmentation mapping relation and storage medium |
| CN116737380B (en) * | 2023-06-16 | 2024-02-09 | 深圳市青葡萄科技有限公司 | Balanced storage method, device and equipment for distributed memory and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101697526A (en) * | 2009-10-10 | 2010-04-21 | 中国科学技术大学 | Method and system for load balancing of metadata management in distributed file system |
| CN103345430A (en) * | 2013-07-03 | 2013-10-09 | 中国科学院高能物理研究所 | Distributed type storage pool fuzzy balancing method and system |
| CN106055277A (en) * | 2016-05-31 | 2016-10-26 | 重庆大学 | Decentralized distributed heterogeneous storage system data distribution method |
| CN106790273A (en) * | 2017-02-17 | 2017-05-31 | 深圳市中博睿存信息技术有限公司 | The encryption storage method and device of stream medium data in distributed file system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10003649B2 (en) * | 2015-05-07 | 2018-06-19 | Dell Products Lp | Systems and methods to improve read/write performance in object storage applications |
-
2017
- 2017-06-29 CN CN201710517682.7A patent/CN107317864B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101697526A (en) * | 2009-10-10 | 2010-04-21 | 中国科学技术大学 | Method and system for load balancing of metadata management in distributed file system |
| CN103345430A (en) * | 2013-07-03 | 2013-10-09 | 中国科学院高能物理研究所 | Distributed type storage pool fuzzy balancing method and system |
| CN106055277A (en) * | 2016-05-31 | 2016-10-26 | 重庆大学 | Decentralized distributed heterogeneous storage system data distribution method |
| CN106790273A (en) * | 2017-02-17 | 2017-05-31 | 深圳市中博睿存信息技术有限公司 | The encryption storage method and device of stream medium data in distributed file system |
Non-Patent Citations (1)
| Title |
|---|
| 负载均衡的大数据分布存储方法研究与实现;贺昱洁;《中国硕士期刊学术库》;20161231;第四章 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107317864A (en) | 2017-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107317864B (en) | Data equalization method and device of storage equipment | |
| CN110018799B (en) | Storage pool PG (packet data) master determination method, device, equipment and readable storage medium | |
| CN104102543B (en) | The method and apparatus of adjustment of load in a kind of cloud computing environment | |
| CN107124472A (en) | Load-balancing method and device, computer-readable recording medium | |
| US10613898B2 (en) | Apparatuses and methods for flexible scheduling in a database system | |
| CN110389813B (en) | Virtual machine dynamic migration method for network target range | |
| CN106385468A (en) | Method for balancing predictable dynamic load of Web clusters | |
| CN103778071A (en) | Cache space distribution method and device | |
| CN105760230B (en) | A kind of method and device of adjust automatically cloud host operation | |
| CN111045808B (en) | Distributed network task scheduling method and device | |
| CN104484233B (en) | A kind of resource allocation methods | |
| CN111245732B (en) | Flow control method, device and equipment | |
| US10684942B2 (en) | Selective application testing | |
| CN112148427A (en) | Cloud platform resource allocation method and device and computer readable storage medium | |
| CN106775470B (en) | Data storage method and system | |
| CN111562884B (en) | Data storage method and device and electronic equipment | |
| CN104199724A (en) | Virtual resource dispatching and optimizing method based on cost performance | |
| CN115167780A (en) | Data reconstruction method and device, electronic equipment and storage medium | |
| CN107346350B (en) | Distribution method, device and cluster system for integrated circuit layout data processing tasks | |
| CN111428885A (en) | User indexing method in federated learning and federated learning device | |
| CN114185767A (en) | AB test method, device and computer readable storage medium | |
| CN108920282A (en) | A kind of copy of content generation, placement and the update method of holding load equilibrium | |
| CN105139020A (en) | User clustering method and device | |
| CN115842828A (en) | Gateway load balancing control method, device, equipment and readable storage medium | |
| US11223675B2 (en) | Hash data structure biasing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200723 Address after: 215100 No. 1 Guanpu Road, Guoxiang Street, Wuzhong Economic Development Zone, Suzhou City, Jiangsu Province Applicant after: SUZHOU LANGCHAO INTELLIGENT TECHNOLOGY Co.,Ltd. Address before: 450018 Henan province Zheng Dong New District of Zhengzhou City Xinyi Road No. 278 16 floor room 1601 Applicant before: ZHENGZHOU YUNHAI INFORMATION TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |