CN109788006B - Data equalization method and device and computer equipment - Google Patents
Data equalization method and device and computer equipment Download PDFInfo
- Publication number
- CN109788006B CN109788006B CN201711106068.8A CN201711106068A CN109788006B CN 109788006 B CN109788006 B CN 109788006B CN 201711106068 A CN201711106068 A CN 201711106068A CN 109788006 B CN109788006 B CN 109788006B
- Authority
- CN
- China
- Prior art keywords
- data
- data node
- storage device
- migration
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 208000036829 Device dislocation Diseases 0.000 claims abstract description 111
- 238000013508 migration Methods 0.000 claims description 149
- 230000005012 migration Effects 0.000 claims description 149
- 238000012545 processing Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 11
- 230000003287 optical effect Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a data equalization method, a data equalization device and computer equipment, wherein the method comprises the following steps: determining a first data node and a second data node to be balanced based on the utilization rate of the storage device of each data node; determining storage device migration information of the first data node and the second data node; and updating the data distribution information corresponding to the first data node and the second data node respectively according to the storage device migration information so as to complete the storage device migration between the first data node and the second data node. The technical scheme provided by the embodiment of the application improves the data balancing efficiency.
Description
Technical Field
The embodiment of the application relates to the technical field of data processing, in particular to a data equalization method, a data equalization device and computer equipment.
Background
The distributed file system utilizes a plurality of computers to cooperatively solve the problems of calculation, storage and the like which cannot be solved by a single computer. The distributed file system is mainly used for data storage, and data resources are stored on storage equipment of a physical machine in a file form.
In order to facilitate data storage, reading, and management, an existing distributed system generally includes a master node and each data node connected to the master node, where the data node is responsible for storing data, and the master node is responsible for storing data distribution information of each data node. Each data node is allocated storage space, typically having one or more storage devices to store data.
The distributed file system has a phenomenon of unbalanced utilization rate of the storage device in the using process, so that data balancing needs to be performed on the distributed file system to ensure that the utilization rate of the storage device of each data node meets the balancing requirement.
In the prior art, generally, any two data nodes to be balanced are determined according to the storage device utilization rate of each data node, data is migrated from the data node with the higher storage device utilization rate to the data node with the lower storage device utilization rate, and then a master node is notified to update data distribution information of the two data nodes.
Disclosure of Invention
The embodiment of the application provides a data equalization method, a data equalization device and computer equipment, which are used for solving the technical problem of low data equalization efficiency in the prior art.
In a first aspect, an embodiment of the present application provides a data equalization method, including:
determining a first data node and a second data node to be balanced based on the utilization rate of the storage device of each data node;
determining storage device migration information of the first data node and the second data node;
and updating the data distribution information corresponding to the first data node and the second data node respectively according to the storage device migration information so as to complete the storage device migration between the first data node and the second data node.
In a second aspect, an embodiment of the present application provides a data equalization apparatus, including:
the first node determining module is used for determining a first data node and a second data node to be balanced based on the utilization rate of the storage device of each data node;
a migration determining module, configured to determine storage device migration information of the first data node and the second data node;
and the first balancing module is used for updating the data distribution information corresponding to the first data node and the second data node respectively according to the storage device migration information so as to complete the storage device migration between the first data node and the second data node.
In a third aspect, embodiments of the present application provide a computer device, comprising one or more storage components and one or more processing components;
the one or more storage components store one or more computer program instructions;
the one or more processing components invoke and execute the one or more computer program instructions to perform operations comprising:
determining a first data node and a second data node to be balanced based on the utilization rate of the storage device of each data node;
determining storage device migration information of the first data node and the second data node;
and updating the data distribution information corresponding to the first data node and the second data node respectively according to the storage device migration information so as to complete the storage device migration between the first data node and the second data node.
In the embodiment of the application, based on the utilization rate of the storage device of each data node, a first data node and a second data node to be balanced are determined, and the storage device migration information of the first data node and the second data node is determined; and updating the data distribution information corresponding to the first data node and the second data node respectively according to the storage device migration information so as to complete the storage device migration between the first data node and the second data node. The data redistribution of the distributed file system is quickly realized by adopting a device migration mode, the aim of data equalization is fulfilled, and the data equalization efficiency is improved.
These and other aspects of the present application will be more readily apparent from the following description of the embodiments.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1a is a schematic diagram illustrating an architecture of a distributed file system provided herein;
FIG. 1b is a schematic diagram illustrating another structure of a distributed file system provided herein;
FIG. 2 is a flow chart illustrating one embodiment of a data equalization method provided herein;
FIG. 3 is a flow chart illustrating a further embodiment of a data equalization method provided herein;
FIG. 4 is a flow chart illustrating a further embodiment of a data equalization method provided herein;
FIG. 5 is a flow chart illustrating a further embodiment of a data equalization method provided herein;
FIG. 6 is a schematic structural diagram illustrating an embodiment of a data equalization apparatus provided in the present application;
fig. 7 is a schematic structural diagram illustrating a data equalization apparatus according to another embodiment of the present application;
fig. 8 is a schematic structural diagram illustrating a further embodiment of a data equalization apparatus provided in the present application;
fig. 9 is a schematic structural diagram illustrating a further embodiment of a data equalization apparatus provided in the present application;
FIG. 10 is a schematic diagram illustrating an embodiment of a computer device provided by the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
In some of the flows described in the specification and claims of this application and in the above-described figures, a number of operations are included that occur in a particular order, but it should be clearly understood that these operations may be performed out of order or in parallel as they occur herein, the number of operations, e.g., 101, 102, etc., merely being used to distinguish between various operations, and the number itself does not represent any order of performance. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.
The technical scheme of the application is mainly applied to a Distributed File System, such as a Hadoop Distributed File System (HDFS), a Network File System (NFS), and the like. In a cloud computer scenario, a storage device for storing data in a distributed file system is located at a cloud end, and a storage space is provided by a network, which is also referred to as a cloud disk or a network disk.
Fig. 1a shows a general structural diagram of a distributed file system, and it is understood that the distributed file system may include a main node 10 and various data nodes 20 connected to the main node 10. Each data node 20 may be implemented by one or more physical machines, may independently launch one or more operating system processes, and may be allocated storage space, typically on-board one or more storage devices, for storing data. The master node 10 may be implemented by one or more physical machines for storing data distribution information, also referred to as "metadata" or "directory", of the respective data nodes to indicate correspondence of data to storage addresses. A large data is usually divided into a plurality of file blocks, and stored in each data node in the form of file blocks, and the data referred to herein can be generally understood as file blocks.
In a cloud computing scenario, as shown in fig. 1b, a schematic structural diagram of a distributed file system in the cloud computing scenario is shown, where the distributed file system is a cloud storage system.
In this embodiment, the storage device may be a device capable of reading and writing, and may be a magnetic memory, a flash memory, a magnetic disk or an optical disk, and at present, a magnetic disk is usually used for storing data, and the magnetic disk may be, for example, an SSD (Solid State Drive), an HDD (hard disk Drive), an SHHD (Solid State Drive, Hybrid Drive), and the like.
Actual data can be exchanged among the data nodes, and data distribution information is mainly updated between the data nodes and the main node.
Since a data node may have one or more storage devices, the storage device utilization of the data node may refer to an average utilization of each storage device that the data node has, for example, a certain data node includes two storage devices, i.e., storage device a and storage device B, where the utilization of storage device a is 80%, and the utilization of storage device B is 90%, then the storage device utilization of the data node may be (80% + 90%)/2, i.e., 85%.
In the using process of the distributed system file system, the utilization rate of storage equipment among data nodes is unbalanced due to various reasons, for example, the data nodes are newly added, newly generated data can be preferentially written into new data nodes, and the utilization rate difference of the storage equipment of new and old data nodes is large; as another example, client behavior, client application write data has a certain skew behavior, resulting in unbalanced storage device utilization among data nodes. The imbalance of the utilization rate of the storage equipment can cause that some data nodes can not continuously write data or can not successfully write data, so that the normal use of the distributed file system is influenced, and therefore, the data balance is needed.
In order to solve the technical problem of low data equalization efficiency, the inventor provides the technical scheme of the application through a series of researches, and in the embodiment of the application, a first data node and a second data node to be equalized are determined firstly based on the utilization rate of storage equipment of each data node; then determining storage device migration information of the first data node and the second data node; and updating the data distribution information corresponding to the first data node and the second data node respectively according to the storage device migration information so as to complete the storage device migration of the first data node and the second data node. According to the method and the device, the device migration mode is adopted, data migration among the data nodes is not needed, data redistribution among the data nodes can be completed only by updating the data distribution information according to the migration information of the storage device, network bandwidth does not need to be occupied, the purpose of data balance of the distributed file system can be achieved rapidly, and the data balance efficiency is improved.
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 2 is a flow chart of an embodiment of a data equalization method provided in the present application, where the method may include the following steps:
201: and determining a first data node and a second data node to be balanced based on the utilization rate of the storage device of each data node.
The technical solution of this embodiment may be executed by the master node, that is, the master node performs data balancing operations, and may of course be executed by any data node, and the storage device utilization rate of each data node may be obtained by calculation by the master node; of course, the method can also be executed by a third-party device independent from the distributed file system, and the storage device utilization rate of each data node is also obtained through the master node.
The first data node and the second data node may be any two data nodes meeting the condition to be balanced, and are named as the first data node and the second data node for convenience of description.
The condition to be balanced may refer to, for example, that a difference between utilization rates of the storage devices is greater than a first preset value, so that it can be determined that any two data nodes with a large difference between the utilization rates of the storage devices need to be balanced. The first preset value can be actually set in combination with the first data equalization requirement and the equalization granularity requirement. For example, the first preset value may be 60%, and the storage utilization of the data node a is assumed to be 15%. The storage device utilization rate of the data node B is 85%, the difference between the two is 70% and is greater than the first preset value by 60%, and the data node a and the data node B can be respectively used as a first data node and a second data node to be balanced if the conditions to be balanced are met.
202: and determining storage device migration information corresponding to the first data node and the second data node.
As an alternative, the storage device migration information of the first data node and the second data node may be determined based on a first data balancing requirement.
The first data balancing requirement may refer to that the storage device utilization rates of any two data nodes are equal, and may also be set according to different actual scenarios, for example, the storage device utilization rate of each data node may be within a preset range, or a difference value between the storage device utilization rates of any two data nodes is smaller than a first balancing threshold, and the like, which is not specifically limited in the present application.
If the first equalization threshold is equal to the first preset value, that is, the difference between the storage device utilization rates of any two data nodes is greater than the first preset value, the first data equalization requirement is that the difference between the storage device utilization rates of any two data nodes is less than the first preset value, and the condition to be equalized may be an opposite condition of the first data equalization requirement. It should be noted that, if the difference between the storage device utilization rates of any two data nodes is equal to the first preset value, it may be considered that the any two data nodes satisfy the condition to be balanced or satisfy the first data balancing requirement.
The storage device migration information may include storage device information of the first data node migrated to the second data node, and/or storage device information of the second data node migrated to the first data node.
Therefore, it can be understood that in the embodiment of the present application, each data node may have at least two storage devices.
The storage device migration information may be determined based on the first data balancing requirement, so that the respective storage device utilization rates of the first data node and the second data node after the storage device migration meet the first data balancing requirement.
In order to determine the storage device migration information, optionally, respective storage devices corresponding to the first data node and the second data node may be determined first;
and reestablishing the corresponding relation between the first data node and the second data node and each storage device respectively based on a first data balance requirement so as to obtain the migration information of the storage devices.
For example, the first data node includes storage device a and storage device B, and the utilization rate of storage device a is 60%. The utilization rate of the storage device B is 90%, and the storage device utilization rate of the first data node is the average utilization rate of the storage device A and the storage device B, namely 75%;
the second data node comprises a storage device C and a storage device D, the utilization rate of the storage device C is 10%, the utilization rate of the storage device D is 50%, and the utilization rate of the storage device of the second data node is 30%.
Assuming that the to-be-balanced condition is that the difference between the utilization rates of the storage devices of any two data nodes is greater than 30%, and the difference between the utilization rates of the storage devices of the first data node and the second data node is 75% -30%, that is, 45%, it can be known that the to-be-balanced condition is satisfied.
Assuming that the first data balancing requirement is that the difference between the utilization rates of the storage devices of any two data nodes is less than 20%, based on the first data balancing requirement, two migration modes can be determined, and the storage device B of the first data node and the storage device D of the second data node can be exchanged by the first migration mode; the second migration method can exchange the storage device a of the first data node with the storage device C of the second data node, so that the storage device utilization rate of the first data node becomes 55%, the storage device utilization rate of the second data node becomes 50%, the storage device utilization rate difference between the first data node and the second data node becomes 5%, and the first data balance requirement is met. Optionally, a migration mode is selected, i.e. the storage device migration information can be obtained.
Of course, as yet another alternative, the storage device migration information of the first data node and the second data node may be determined according to a device migration rule;
the device migration rule may be exchanged between a high-utilization storage device of a data node with a high storage device utilization rate and a low-utilization storage device of a data node with a low storage device utilization rate.
As described in the above example, according to the device migration rule, the storage device utilization rate of the first data node is high, and the utilization rate of the storage device B is highest; if the utilization rate of the storage device C in the second data node is the lowest, a migration manner of exchanging the storage device B of the first data node with the storage device C of the second data node may be adopted, so that it may be determined that the storage device migration information is that the storage device migrated from the first data node to the second data node is the storage device B, and that the storage device migrated from the second data node to the first data node is the storage device C.
203: and updating the data distribution information corresponding to the first data node and the second data node respectively according to the storage device migration information so as to complete the storage device migration of the first data node and the second data node.
If the first data node and the second data node perform storage device migration, the host node needs to be notified to report respective data information so as to update data distribution information corresponding to the first data node and the second data node in the host node, and thus the storage device migration of the first data node and the second data node is finished.
If the technical solution of this embodiment can be executed by the master node, the master node can automatically update the data distribution information corresponding to the first data node and the second data node, respectively, according to the storage device migration information.
If the technical scheme of this embodiment is executed by another device other than the master node, specifically, the master node may be notified to update the data distribution information corresponding to the first data node and the second data node according to the storage device migration information.
In an actual application, the technical solution of the embodiment of the present application may be applied to a cloud computing scenario, as shown in fig. 1b, in the cloud computing scenario, hardware resources of a data node, such as a processing device and a storage device, may be virtual resources leased from a cloud computing platform, but not actual physical resources, and if the storage device is a leased virtual resource, the data node maintains a corresponding relationship with the corresponding storage device, and based on the corresponding relationship, may perform data processing operation on the leased storage device. Therefore, if the storage device is a leased virtual resource, and the storage devices of the first data node and the second data node are migrated, it is only necessary to re-establish the correspondence between the first data node and the second data node and each storage device, and update the data distribution information corresponding to the first data node and the second data node, respectively, accordingly.
If the hardware resources of the data node are actual physical resources, not virtual resources, as another embodiment, after determining the storage device migration information of the first data node and the second data node, the method may further include:
and outputting storage device migration prompt information to prompt a user to exchange the storage devices of the first data node and the second data node according to the storage device migration information. That is, the user can plug and unplug the storage device in the physical machine to perform the storage device migration.
In this embodiment, an equipment migration manner is adopted, data relocation between data nodes is not required, data redistribution between data nodes can be completed only by updating data distribution information according to storage equipment migration information, network bandwidth is not required to be occupied, the purpose of data equalization can be quickly achieved, and data equalization efficiency is improved.
Fig. 3 is a flow chart of another embodiment of a data equalization method provided in the present application, where the method may include the following steps:
301: and determining a first data node and a second data node to be balanced based on the utilization rate of the storage device of each data node.
302: determining storage device migration information of the first data node and the second data node.
303: and updating the data distribution information corresponding to the first data node and the second data node respectively according to the storage device migration information so as to complete the storage device migration between the first data node and the second data node.
Step 301 to step 303 can refer to step 201 to step 203 in the above embodiments, and are not described herein again.
304: determining each first storage device corresponding to the first data node and each second storage device corresponding to the second data node after the storage device is migrated according to the storage device migration information.
And if the data is migrated through the storage device, the corresponding relation between the first data node and the storage device and the corresponding relation between the second data node and the storage device are changed.
Therefore, after the storage device migration is performed, the respective first storage devices corresponding to the first data nodes and the respective second storage devices corresponding to the second data nodes can be determined.
The first storage device and the second storage device are only used for distinguishing the storage devices corresponding to the first data node and the second data node from each other in terms of names, and do not represent the association relationship such as order, progression, number and the like.
305: and performing data migration on each first storage device based on the utilization rate of each first storage device to obtain first data migration information.
306: and performing data migration on each second storage device based on the utilization rate of each second storage device to obtain second data migration information.
Since the storage device utilization of the data node is the average utilization of all the storage devices that the data node has, after the storage device utilization of the data node is balanced, the utilization between the storage devices inside the data node may need to be further balanced.
For example, the data node includes a storage device a and a storage device B, the utilization rate of the storage device a is 20%, and the utilization rate of the storage device B is 100%, and it is known that the internal data distribution of the data node is not balanced.
Therefore, the internal balancing of the first data node and the second data node, namely the data migration between the internal storage devices, can be respectively performed.
Optionally, data migration may be performed on each first storage device based on the utilization rate and the internal balancing requirement of each first storage device, so as to obtain first data migration information of the first data node.
The data migration of each second storage device may be performed based on the utilization rate of each second storage device and the internal balancing requirement, so as to obtain second data migration information of the second data node.
The internal balancing requirement may be, for example, that the utilization of each storage device is equal, or that the utilization of each storage device is within an internal balancing range.
As in the above example, if the utilization rate of storage device a of the data node is 20%, the utilization rate of storage device B is 100%, and the internal balancing requirement is that the utilization rates of each storage device are equal, then migration of data of storage device B to storage device a may be performed, so that the utilization rates of storage device a and storage device B reach 60%.
307: and updating the data distribution information corresponding to the first data node and the second data node respectively according to the first data migration information and the second data migration information.
After the storage device migration is performed on the first data node and the second data node, if internal balancing is performed, the respective data distribution information may be continuously updated according to the first data migration information of the first data node and the second data migration information of the second data node.
In this embodiment, storage device balancing between the data nodes may be quickly achieved through a device migration manner, and on the basis of device migration, data distribution of the storage devices inside each data node is readjusted to perform internal balancing, thereby further ensuring data balancing of each data node.
Fig. 4 is a flow chart of another embodiment of a data equalization method provided in the present application, where the method may include the following steps:
401: and judging whether a first data node and a second data node meeting the condition to be balanced exist or not based on the storage device utilization rate of each data node, if so, executing a step 402, and if not, executing a step 404.
The condition to be equalized may be, for example, that the difference between the utilization rates of the storage devices is greater than a first preset value.
402: and determining storage device migration information of the first data node and the second data node.
Optionally, storage device migration information may be determined based on the first data leveling requirement. Of course, the storage device migration information may also be determined based on the device migration rule, which may specifically refer to the description in the foregoing embodiment.
If the condition to be balanced and the first data balancing requirement are opposite conditions, whether a first data node and a second data node meeting the condition to be balanced exist or not is judged, and whether a first data node and a second data node which do not meet the first data balancing requirement exist or not is also specifically judged. That is, if there is no data node satisfying the condition to be balanced, it can be shown that each data node satisfies the first data balancing requirement.
403: and updating the data distribution information corresponding to the first data node and the second data node respectively according to the storage device migration information so as to complete the storage device migration of the first data node and the second data node.
Optionally, after the storage device is migrated according to the storage device migration information, determining each first storage device corresponding to the first data node and each second storage device corresponding to the second data node;
performing data migration on each first storage device based on the utilization rate of each first storage device to obtain first data migration information;
performing data migration on each second storage device based on the utilization rate of each second storage device to obtain second data migration information;
therefore, the data distribution information corresponding to the first data node and the second data node can be continuously updated according to the first data migration information and the second data migration information.
404: and determining a third data node and a fourth data node which meet the conditions to be migrated based on the utilization rate of the storage device of each data node.
405: and performing data migration between the third data node and the fourth data node based on a second data balance requirement to obtain third data migration information.
The condition to be migrated may be, for example, that a difference between utilization rates of the storage devices is greater than a second preset value, where the second preset value may be smaller than the first preset value.
406: updating data distribution information of the third data node and the fourth data node based on the third data migration information.
If any two data nodes meeting the conditions to be migrated do not exist, the process can be ended.
The second data leveling requirement may be, for example, that the difference in storage device utilization is less than a second leveling threshold;
wherein the second equalization threshold may be less than or equal to the first equalization threshold.
In addition, the second equalization threshold may be equal to the second preset value or smaller than the second preset value, and if the second equalization threshold is equal to the second preset value, that is, the condition to be migrated and the second data equalization requirement may be two conditions opposite to each other, it is determined whether a third data node and a fourth data node that satisfy the condition to be migrated exist, or specifically, whether a third data node and a fourth data node that do not satisfy the second data equalization requirement exist.
Based on the second data balance requirement, data migration can be performed and third data migration information can be obtained, wherein the third data migration information comprises migration data identification, and therefore data distribution information can be updated based on the third data migration information.
The data migration method is the same as that in the prior art, and is not described herein again.
In this embodiment, data equalization between data nodes is quickly achieved by adopting an equipment migration mode, when data nodes which all meet a first data equalization requirement or data nodes which do not store data to be equalized are all satisfied, data equalization can be further performed on the data nodes by adopting the data migration mode, and the data equalization process is divided into two stages, so that the data equalization of coarse granularity can be performed by using the equipment migration mode at first, and the data equalization of fine granularity can be further performed by using the data migration mode, because the data equalization of coarse granularity is performed by using the data migration mode, the data equalization of fine granularity only needs to be performed on part of data in the data nodes, and therefore, excessive network bandwidth does not need to be occupied, too much time is not occupied, and the data equalization efficiency is guaranteed while the data equalization fineness of the data equalization is guaranteed.
Fig. 5 is a flow chart of another embodiment of a data equalization method provided in the present application, where the method may include the following steps:
501: and determining a first data node and a second data node to be balanced based on the utilization rate of the storage device of each data node.
502: determining storage device migration information of the first data node and the second data node.
Optionally, the storage device migration information of the first data node and the second data node may be determined based on a first data balancing requirement or based on a device migration rule.
503: and updating the data distribution information corresponding to the first data node and the second data node respectively according to the storage device migration information so as to complete the storage device migration of the first data node and the second data node.
Optionally, after the storage device is migrated according to the storage device migration information, determining each first storage device corresponding to the first data node and each second storage device corresponding to the second data node;
performing data migration on each first storage device based on the utilization rate of each first storage device to obtain first data migration information;
performing data migration on each second storage device based on the utilization rate of each second storage device to obtain second data migration information;
therefore, the data distribution information corresponding to the first data node and the second data node may be continuously updated according to the first data migration information and the second data migration information.
504: judging whether the first data node and the second data node after the storage device migration is performed according to the storage device migration information meet a second data balance requirement, if not, executing step 505; if so, the method returns to the step 501 to continue the execution.
505: and carrying out data migration between the first data node and the second data node to obtain fourth data migration information.
506: updating data distribution information of the first data node and the second data node based on the fourth data migration information.
In this embodiment, if the first data node and the second data node after the migration of the storage device do not satisfy the first data balancing requirement, data balancing may be further performed in a data migration manner, and since the first data node and the second data node have performed the migration of the storage device, migrated data required for the data migration may be smaller, and may not occupy more network resources and time, thereby not only ensuring the efficiency of data balancing, but also ensuring the fineness of data balancing.
In practical application, a current distributed file system may store data by using a disk, if a storage device is a disk, the storage device utilization rate is a disk utilization rate, and the storage device migration is also a disk migration, so that the data redistribution through the disk migration can achieve the purpose of data equalization, and the data equalization efficiency can be ensured.
Fig. 6 is a schematic structural diagram of an embodiment of a data balancing apparatus provided in the present application, where the data balancing apparatus may be configured in a master node, a data node, or a third-party device of an independent and distributed file system;
the apparatus may include:
the first node determining module 601 is configured to determine a first data node and a second data node to be balanced based on storage device utilization rates of the data nodes.
Optionally, the first node determining module 601 may be specifically configured to:
determining respective storage devices corresponding to the first data node and the second data node;
based on a first data balance requirement, reestablishing the corresponding relation between the first data node and the storage device and the corresponding relation between the second data node and the storage devices respectively, so as to obtain the migration information of the storage devices.
Optionally, the first node determining module 601 may be specifically configured to:
and determining a first data node and a second data node which meet the condition to be balanced based on the utilization rate of the storage device of each data node.
A migration determining module 602, configured to determine storage device migration information of the first data node and the second data node.
A first balancing module 603, configured to update data distribution information corresponding to the first data node and the second data node, respectively, according to the storage device migration information, so as to complete storage device migration of the first data node and the second data node.
In the embodiment, an equipment migration mode is adopted, data migration between data nodes is not needed, data redistribution between the data nodes can be completed only by updating data distribution information according to the storage equipment migration information, network bandwidth does not need to be occupied, the purpose of data equalization can be quickly achieved, and the data equalization efficiency is improved.
Fig. 7 is a schematic structural diagram of another embodiment of a data equalization apparatus provided in the present application, and unlike the apparatus shown in fig. 6, the apparatus may further include:
a device determining module 701, configured to determine, after the storage device is migrated according to the storage device migration information, each first storage device corresponding to the first data node and each second storage device corresponding to the second data node;
a first information determining module 702, configured to perform data migration on each first storage device based on a utilization rate of each first storage device to obtain first data migration information;
a second information determining module 703, configured to perform data migration on each second storage device based on the utilization rate of each second storage device, so as to obtain second data migration information;
a second balancing module 704, configured to update data distribution information corresponding to the first data node and the second data node according to the first data migration information and the second data migration information.
The first node determining module 601 may specifically determine, based on the storage device utilization of each data node, a first data node and a second data node that satisfy a condition to be balanced.
If there is no first data node and no second data node that satisfy the condition to be balanced, the data migration method may be continuously used to perform data balancing on the distributed file system, so as to provide a further embodiment, as shown in fig. 8, a difference from the apparatus shown in fig. 6 is that the apparatus may further include:
a second node determining module 801, configured to determine, if there are no first data node and no second data node that meet the condition to be balanced, a third data node and a fourth data node that meet the condition to be migrated based on a storage device utilization rate of each data node;
a first migration module 802, configured to perform data migration between the third data node and the fourth data node based on a second data balancing requirement, so as to obtain third data migration information;
a third balancing module 803, configured to update the data distribution information of the third data node and the fourth data node based on the third data migration information.
The first node determining module 601 may be specifically configured to determine, based on the storage device utilization of each data node, a first data node and a second data node, where a storage device utilization difference is greater than a first preset value.
The second node determining module 801 may be specifically configured to determine, if there are no two data nodes whose storage device utilization difference is greater than the first preset value, a third data node and a fourth data node whose storage device utilization difference is greater than a second preset value based on the storage device utilization of each data node; wherein the second preset value is less than the first preset value.
Therefore, the data balancing process of the embodiment is divided into two stages, so that coarse-grained data balancing can be performed by using an equipment migration mode at first, fine-grained data balancing can be performed by using data migration, the coarse-grained data balancing is performed by using the equipment migration mode, the fine-grained data balancing only needs to be performed on partial data in partial data nodes, and therefore the data balancing method does not need to occupy too much network bandwidth and time, and the data balancing efficiency is guaranteed while the data balancing fineness of the data balancing is guaranteed.
In addition, if the first data node and the second data node after the migration of the storage device does not satisfy the first data balancing requirement, a data migration manner may be further adopted to perform data balancing, so as shown in fig. 9, as another embodiment, the difference from the apparatus shown in fig. 5 is that the apparatus may further include:
the determining module 901 determines whether the first data node and the second data node after the storage device migration meet a second data balancing requirement;
a second migration module 902, configured to, if the first data node and the second data node do not meet the second data balancing requirement, perform data migration between the first data node and the second data node to obtain fourth data migration information;
a fourth balancing module 903, configured to update the data distribution information of the first data node and the second data node based on the fourth data migration information.
In this embodiment, if the first data node and the second data node after the migration of the storage device do not satisfy the first data balancing requirement, data balancing is further performed in a data migration manner, and since the first data node and the second data node have performed device migration, migrated data required for data migration is relatively small, and does not occupy more network resources and time, so that the data balancing efficiency is ensured, and the fineness of the data balancing can also be ensured.
In an actual application, the technical solution of the embodiment of the present application may be applied to a cloud computing scenario, as shown in fig. 1b, hardware resources of a data node, such as processing devices and storage devices, are leased virtual resources, and storage device migration of a first data node and a second data node only needs to re-establish a correspondence between the first data node and the second data node and each storage device, and accordingly update data distribution information corresponding to the first data node and the second data node.
And if the hardware resource of the data node is an actual physical resource, in some embodiments, the apparatus may further include:
the output module is used for outputting the migration prompt information of the storage equipment; and the storage device migration prompting information is used for prompting a user to exchange the storage devices in the first data node and the second data node according to the storage device migration information. That is, the storage device can be plugged in and unplugged from the physical machine by the user to realize the migration of the storage device.
In a possible design, any of the data equalization apparatuses shown in fig. 6 to 9 may be implemented as a computer device, where the computer device may be a master node or a data node in the distributed file system, or may be a third-party device independent of the distributed file system;
as shown in fig. 10, the computer device may include one or more storage components 1001 and one or more processing components 1002;
the one or more storage components 1001 store one or more pieces of computer program instructions;
the one or more processing components 1002 invoke and execute the one or more computer program instructions to:
determining a first data node and a second data node to be balanced based on the utilization rate of the storage device of each data node;
determining storage device migration information of the first data node and the second data node;
and updating the data distribution information corresponding to the first data node and the second data node respectively according to the storage device migration information so as to complete the storage device migration of the first data node and the second data node.
Optionally, the determining, by the one or more processing components 1002, the storage device migration information of the first data node and the second data node may specifically be:
determining respective storage devices corresponding to the first data node and the second data node;
based on a first data balance requirement, reestablishing the corresponding relation between the first data node and the storage device and the corresponding relation between the second data node and the storage devices respectively, so as to obtain the migration information of the storage devices.
Optionally, the one or more processing components 1002 are further configured to: :
determining each first storage device corresponding to the first data node and each second storage device corresponding to the second data node after storage device migration is performed according to the storage device migration information;
performing data migration on each first storage device based on the utilization rate of each first storage device to obtain first data migration information;
performing data migration on each second storage device based on the utilization rate of each second storage device to obtain second data migration information;
and updating the data distribution information corresponding to the first data node and the second data node respectively according to the first data migration information and the second data migration information.
Optionally, the one or more processing components 1002 may determine, based on the storage device utilization of each data node, that the first data node and the second data node to be balanced are:
and determining a first data node and a second data node which meet the condition to be balanced based on the utilization rate of the storage device of each data node.
Optionally, the one or more processing components 1002 are further configured to: if any two data nodes meeting the conditions to be balanced do not exist, determining a third data node and a fourth data node meeting the conditions to be migrated based on the utilization rate of the storage device of each data node;
performing data migration between the third data node and the fourth data node based on a second data balance requirement to obtain third data migration information;
updating data distribution information of the third data node and the fourth data node based on the third data migration information.
Optionally, the one or more processing components 1002 are further configured to: judging whether the first data node and the second data node after the migration of the storage equipment meet a second data balance requirement or not;
if the first data node and the second data node do not meet the second data balance requirement, performing data migration between the first data node and the second data node to obtain fourth data migration information;
updating data distribution information of the first data node and the second data node based on the fourth data migration information.
The condition to be equalized may mean that a difference between utilization rates of the storage devices is greater than a first preset value, and the condition to be migrated may mean that a difference between utilization rates of the storage devices is greater than a second preset value.
Optionally, the one or more processing components 1002 are further configured to: outputting migration prompt information of the storage equipment; and the storage device migration prompting information is used for prompting a user to migrate the storage devices in the first data node and the second data node according to the storage device migration information.
The memory components described above may be implemented, among other things, by any type of volatile or non-volatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
When the computer device is a data node, the one or more storage components 1001 include the storage device for storing data.
The processing component may be a CPU, or may also be an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components.
It should be noted that the computer device may also comprise other necessary components, such as a bus, an input or output interface, a power supply component, an output component, etc.
In addition, an embodiment of the present application further provides a computer storage medium, where the computer storage medium stores one or more computer instructions, and the one or more computer instructions may enable a computer to implement the data equalization method according to any of the above embodiments when executed.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (19)
1. A method of data equalization, comprising:
determining a first data node and a second data node to be balanced based on the utilization rate of the storage device of each data node; each data node is realized by one or more physical machines, and each data node is hung with one or more storage devices;
determining storage device migration information of the first data node and the second data node; the storage device migration information includes: information migrated from the first data node to the storage device of the second data node, and/or information migrated from the second data node to the storage device of the first data node;
updating data distribution information respectively corresponding to the first data node and the second data node according to the storage device migration information so as to complete storage device migration between the first data node and the second data node;
after the migration of the storage device is completed, the method further comprises the following steps: and performing data migration on each storage device based on the utilization rate of each storage device in the data node after the storage device migration is completed to obtain data migration information, and updating the data distribution information according to the data migration information.
2. The method of claim 1, wherein determining storage device migration information for a first data node and a second data node comprises:
determining respective storage devices corresponding to the first data node and the second data node;
based on a first data balance requirement, reestablishing the corresponding relation between the first data node and the storage device and the corresponding relation between the second data node and the storage devices respectively, so as to obtain the migration information of the storage devices.
3. The method according to claim 1, wherein after the updating of the data distribution information corresponding to the first data node and the second data node, respectively, according to the storage device migration information to complete the storage device migration between the first data node and the second data node, the method further comprises:
determining each first storage device corresponding to the first data node and each second storage device corresponding to the second data node;
performing data migration on each first storage device based on the utilization rate of each first storage device to obtain first data migration information;
performing data migration on each second storage device based on the utilization rate of each second storage device to obtain second data migration information;
and updating the data distribution information corresponding to the first data node and the second data node respectively according to the first data migration information and the second data migration information.
4. The method of claim 1, wherein determining the first data node and the second data node to be balanced based on storage device utilization of each data node comprises:
and determining a first data node and a second data node which meet the condition to be balanced based on the utilization rate of the storage device of each data node.
5. The method of claim 4, further comprising:
if the first data node and the second data node which meet the conditions to be balanced do not exist, determining a third data node and a fourth data node which meet the conditions to be migrated based on the utilization rate of the storage device of each data node;
performing data migration between the third data node and the fourth data node based on a second data balance requirement to obtain third data migration information;
updating data distribution information of the third data node and the fourth data node based on the third data migration information.
6. The method according to claim 1, wherein after the updating of the data distribution information corresponding to the first data node and the second data node, respectively, according to the storage device migration information to complete the storage device migration between the first data node and the second data node, the method further comprises:
judging whether the first data node and the second data node after the storage device migration is carried out according to the storage device migration information meet a second data balance requirement or not;
if the first data node and the second data node do not meet the second data balance requirement, performing data migration between the first data node and the second data node to obtain fourth data migration information;
updating data distribution information of the first data node and the second data node based on the fourth data migration information.
7. The method of claim 5, wherein determining the first data node and the second data node that satisfy the condition to be balanced based on the storage device utilization of each data node comprises:
and determining a first data node and a second data node of which the difference value of the utilization rates of the storage devices is greater than a first preset value based on the utilization rates of the storage devices of the data nodes.
8. The method of claim 7, wherein if there are no first data node and no second data node that satisfy the condition to be balanced, determining a third data node and a fourth data node that satisfy the condition to be migrated based on storage device utilization of the respective data nodes comprises:
if any two data nodes with the difference value of the utilization rates of the storage devices larger than the first preset value do not exist, determining a third data node and a fourth data node with the difference value of the utilization rates of the storage devices larger than a second preset value based on the utilization rate of the storage devices of the data nodes; wherein the second preset value is less than the first preset value.
9. The method of claim 1, wherein after the storage device migration information determining step, the method further comprises:
outputting migration prompt information of the storage equipment; and the storage device migration prompting information is used for prompting a user to exchange the storage devices in the first data node and the second data node according to the storage device migration information.
10. A data equalization apparatus, comprising:
the first node determining module is used for determining a first data node and a second data node to be balanced based on the utilization rate of the storage device of each data node; each data node is realized by one or more physical machines, and each data node is hung with one or more storage devices;
a migration determining module, configured to determine storage device migration information of the first data node and the second data node; the storage device migration information includes: information migrated from the first data node to the storage device of the second data node, and/or information migrated from the second data node to the storage device of the first data node;
the first balancing module is configured to update data distribution information corresponding to the first data node and the second data node, respectively, according to the storage device migration information, so as to complete storage device migration between the first data node and the second data node;
after the migration of the storage device is completed, the method further comprises the following steps: and performing data migration on each storage device based on the utilization rate of each storage device in the data node after the storage device migration is completed to obtain data migration information, and updating the data distribution information according to the data migration information.
11. The apparatus of claim 10, wherein the first node determining module is specifically configured to:
determining respective storage devices corresponding to the first data node and the second data node;
based on a first data balance requirement, reestablishing the corresponding relation between the first data node and the storage device and the corresponding relation between the second data node and the storage devices respectively, so as to obtain the migration information of the storage devices.
12. The apparatus of claim 10, further comprising:
a device determining module, configured to determine, after the storage device is migrated according to the storage device migration information, each first storage device corresponding to the first data node and each second storage device corresponding to the second data node;
a first information determining module, configured to perform data migration on each first storage device based on a utilization rate of each first storage device to obtain first data migration information;
a second information determining module, configured to perform data migration on each second storage device based on a utilization rate of each second storage device to obtain second data migration information;
and the second balancing module is configured to update the data distribution information corresponding to the first data node and the second data node according to the first data migration information and the second data migration information.
13. The apparatus of claim 10, wherein the first node determining module is specifically configured to: and determining a first data node and a second data node which meet the condition to be balanced based on the utilization rate of the storage device of each data node.
14. The apparatus of claim 13, further comprising:
the second node determining module is used for determining a third data node and a fourth data node which meet the conditions to be migrated based on the utilization rate of the storage device of each data node if the first data node and the second data node which meet the conditions to be balanced do not exist;
the first migration module is used for performing data migration between the third data node and the fourth data node based on a second data balance requirement to obtain third data migration information;
and the third balancing module is used for updating the data distribution information of the third data node and the fourth data node based on the third data migration information.
15. The apparatus of claim 10, further comprising:
the judging module is used for judging whether the first data node and the second data node after the storage equipment is migrated according to the storage equipment migration information meet second data balance requirements or not;
a second migration module, configured to perform data migration between the first data node and the second data node to obtain fourth data migration information if the first data node and the second data node do not meet the second data balancing requirement;
a fourth balancing module, configured to update the data distribution information of the first data node and the second data node based on the fourth data migration information.
16. The apparatus according to claim 14, wherein the first node determining module is specifically configured to determine, based on the storage device utilization of each data node, a first data node and a second data node, which have a storage device utilization difference greater than a first preset value.
17. The apparatus according to claim 16, wherein the second node determining module is specifically configured to determine, if there are no two data nodes having a difference in storage device utilization that is greater than the first preset value, a third data node and a fourth data node having a difference in storage device utilization that is greater than a second preset value based on the storage device utilization of each data node; wherein the second preset value is less than the first preset value.
18. The apparatus of claim 10, further comprising:
the output module is used for outputting the migration prompt information of the storage equipment; and the storage device migration prompting information is used for prompting a user to exchange the storage devices in the first data node and the second data node according to the storage device migration information.
19. A computer device comprising one or more storage components and one or more processing components;
the one or more storage components store one or more computer program instructions;
the one or more processing components invoke and execute the one or more computer program instructions to perform operations comprising:
determining a first data node and a second data node to be balanced based on the utilization rate of the storage device of each data node; each data node is realized by one or more physical machines, and each data node is hung with one or more storage devices;
determining storage device migration information of the first data node and the second data node; the storage device migration information includes: information migrated from the first data node to the storage device of the second data node, and/or information migrated from the second data node to the storage device of the first data node;
updating data distribution information respectively corresponding to the first data node and the second data node according to the storage device migration information so as to complete storage device migration between the first data node and the second data node;
after the migration of the storage device is completed, the method further comprises the following steps: and performing data migration on each storage device based on the utilization rate of each storage device in the data node after the storage device migration is completed to obtain data migration information, and updating the data distribution information according to the data migration information.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711106068.8A CN109788006B (en) | 2017-11-10 | 2017-11-10 | Data equalization method and device and computer equipment |
| PCT/CN2018/113902 WO2019091349A1 (en) | 2017-11-10 | 2018-11-05 | Data balancing method, apparatus and computer device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711106068.8A CN109788006B (en) | 2017-11-10 | 2017-11-10 | Data equalization method and device and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109788006A CN109788006A (en) | 2019-05-21 |
| CN109788006B true CN109788006B (en) | 2021-08-24 |
Family
ID=66437583
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711106068.8A Active CN109788006B (en) | 2017-11-10 | 2017-11-10 | Data equalization method and device and computer equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109788006B (en) |
| WO (1) | WO2019091349A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111143324B (en) * | 2019-12-20 | 2023-05-02 | 浪潮软件股份有限公司 | Kudu database data equalization system based on size and implementation method |
| CN112667160A (en) * | 2020-12-25 | 2021-04-16 | 深圳创新科技术有限公司 | Rapid equalization method and device for mass storage system |
| CN114979153B (en) * | 2022-04-07 | 2023-10-27 | 浙江大华技术股份有限公司 | Load balancing method, computer equipment and storage device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104378447A (en) * | 2014-12-03 | 2015-02-25 | 深圳市鼎元科技开发有限公司 | Non-migration distributed storage method and non-migration distributed storage system on basis of Hash ring |
| CN105657066A (en) * | 2016-03-23 | 2016-06-08 | 天津书生云科技有限公司 | Load rebalance method and device used for storage system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7599941B2 (en) * | 2005-07-25 | 2009-10-06 | Parascale, Inc. | Transparent redirection and load-balancing in a storage network |
| JP5561006B2 (en) * | 2010-08-05 | 2014-07-30 | 富士通株式会社 | Data transfer device, data receiving device, data transfer method, and data transfer program |
-
2017
- 2017-11-10 CN CN201711106068.8A patent/CN109788006B/en active Active
-
2018
- 2018-11-05 WO PCT/CN2018/113902 patent/WO2019091349A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104378447A (en) * | 2014-12-03 | 2015-02-25 | 深圳市鼎元科技开发有限公司 | Non-migration distributed storage method and non-migration distributed storage system on basis of Hash ring |
| CN105657066A (en) * | 2016-03-23 | 2016-06-08 | 天津书生云科技有限公司 | Load rebalance method and device used for storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109788006A (en) | 2019-05-21 |
| WO2019091349A1 (en) | 2019-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107943421B (en) | Partition division method and device based on distributed storage system | |
| EP3067797A1 (en) | Creating new cloud resource instruction set architecture | |
| US20170024285A1 (en) | Automatic serial starting of resource groups on failover | |
| US10324754B2 (en) | Managing virtual machine patterns | |
| US10356150B1 (en) | Automated repartitioning of streaming data | |
| US20240061812A1 (en) | Metadata control in a load-balanced distributed storage system | |
| CN109788006B (en) | Data equalization method and device and computer equipment | |
| US9354826B2 (en) | Capacity expansion method and device | |
| CN107423301B (en) | Data processing method, related equipment and storage system | |
| CN105743808A (en) | Method and device of adapting QoS | |
| CN104283951A (en) | Method and device for migrating instances and system | |
| CN104461698A (en) | Dynamic virtual disk mounting method, virtual disk management device and distributed storage system | |
| EP3432132A1 (en) | Data storage method and device | |
| CN105302536A (en) | Configuration method and apparatus for related parameters of MapReduce application | |
| JP2016510918A (en) | Identifying workloads and sizing buffers for volume replication purposes | |
| CN114667506A (en) | Management of multi-physical function non-volatile memory devices | |
| CN108833592A (en) | Cloud host schedules device optimization method, device, equipment and storage medium | |
| CN114996003A (en) | Cloud service deployment method and device, electronic equipment and storage medium | |
| CN110692043B (en) | System and method for load balancing backup data | |
| US10606478B2 (en) | High performance hadoop with new generation instances | |
| CN104657216A (en) | Resource allocation method and device for resource pool | |
| JP2014206805A (en) | Control device | |
| CN108804018A (en) | A kind of method and device of data partition | |
| US10097431B1 (en) | Routing to tenant services utilizing a service directory | |
| US10860433B1 (en) | Directional consistency in capture and recovery of cloud-native applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |