CN112416888B - Dynamic load balancing method and system for distributed file system - Google Patents
Dynamic load balancing method and system for distributed file system Download PDFInfo
- Publication number
- CN112416888B CN112416888B CN202011107475.2A CN202011107475A CN112416888B CN 112416888 B CN112416888 B CN 112416888B CN 202011107475 A CN202011107475 A CN 202011107475A CN 112416888 B CN112416888 B CN 112416888B
- Authority
- CN
- China
- Prior art keywords
- data
- node
- racks
- nodes
- data node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000013500 data storage Methods 0.000 claims abstract description 7
- 238000004590 computer program Methods 0.000 claims description 9
- 230000004044 response Effects 0.000 claims description 7
- 238000007726 management method Methods 0.000 description 31
- 238000010586 diagram Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
Abstract
The application discloses a dynamic load balancing method for a distributed file system, which comprises the following steps: acquiring a plurality of heartbeat packets provided by a plurality of data nodes, wherein the data nodes are nodes for data storage in the distributed file system; acquiring node load information of each data node at least based on the plurality of heartbeat packets; receiving a writing request of a client; responding to the write request: and selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node so as to butt-joint the writing task of the client through the plurality of target data nodes. In this embodiment, the heartbeat packet provides load related information to select a suitable data node for writing, so as to ensure the writing speed of the client.
Description
Technical Field
The present disclosure relates to the field of cloud data, and in particular, to a dynamic load balancing method, system, computer device, and computer readable storage medium for a distributed file system.
Background
With the development and popularization of computer and information technology, data has been explosively increased. The accumulated data are written into and stored in the distributed file system, so that the load balance of the distributed file system is ensured while the data are effectively utilized and processed, and the data become one of research hotspots on the distributed file system.
The distributed file system includes a plurality of data nodes and a management node. The data nodes are responsible for storing, reading and writing real files, and the management nodes are responsible for distributing tasks to the data nodes for execution. Task allocation is one of the most basic and challenging problems for distributed file systems. When the scale of the distributed system is enlarged to a certain extent, as the data are distributed on different data nodes, the frequency difference of data writing and reading is huge, and meanwhile, the configuration difference of each data node finally causes the load deviation between the data nodes to become large and the load is unbalanced. When the management node allocates the writing task to the client, the writing task may be continuously allocated to the data node with high load, so that the data processing speed is reduced, and the overall writing speed is affected.
Disclosure of Invention
An object of the embodiments of the present application is to provide a dynamic load balancing method, a system, a computer device and a computer readable storage medium for a distributed file system, so as to solve the problems of slow writing speed and unbalanced load of the distributed file system.
An aspect of an embodiment of the present application provides a dynamic load balancing method for a distributed file system, the method including:
an aspect of an embodiment of the present application further provides a dynamic load balancing system for a distributed file system, including: acquiring a plurality of heartbeat packets provided by a plurality of data nodes, wherein the data nodes are nodes for data storage in the distributed file system; acquiring node load information of each data node at least based on the plurality of heartbeat packets; receiving a writing request of a client; responding to the write request: and selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node so as to butt-joint the writing task of the client through the plurality of target data nodes.
Optionally, the distributed file system comprises a plurality of racks, each rack comprising one or more data nodes; selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node, wherein the method comprises the following steps: selecting an ith batch of data nodes from an ith group of racks, and selecting one of the data nodes from the ith batch of data nodes as an ith target data node according to node load information of each data node in the ith batch of data nodes; wherein: when i=1, the first set of racks includes the plurality of racks, the first batch of data nodes includes at least two data nodes of the plurality of racks, and the first target data node is configured to store a first copy of data written by the client; when i is more than or equal to 2, the ith group of racks comprises other racks of the plurality of racks except for the racks where the first to ith target data nodes are located; the ith batch of data nodes comprise at least two data nodes in the other racks, and the ith target data node is used for storing the ith data copy written by the client.
Optionally, the distributed file system comprises a plurality of racks, each rack comprising one or more data nodes; selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node, wherein the method comprises the following steps: distributing a first batch of data nodes from the plurality of racks; selecting one of the first batch of data nodes as a first target data node; the first target data node is used for storing a first data copy written by the client; distributing a second batch of data nodes from other racks of the plurality of racks except the rack where the first data node is located; judging whether the second batch of data nodes are successfully distributed from other racks except the rack where the first data node is located; if the second batch of data nodes fails to be allocated from the other racks except the rack where the first data node is located, then: distributing the second batch of data nodes in a rack where the first data node is located; selecting one data node from the second batch of data nodes as a second target data node; the second target data node is used for storing a second data copy written by the client.
Optionally, selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node, and further including: judging whether the first target data node and the second target data node are positioned in the same rack or not; if the first target data node and the second target data node are located in the same rack, distributing a third batch of data nodes from other racks of the plurality of racks except the rack where the first data node is located; judging whether the third batch of data nodes are successfully distributed from other racks except the rack where the first data node is located in the plurality of racks; if the third batch of data nodes are failed to be distributed from other racks of the plurality of racks except the rack where the first data node is located, distributing the third batch of data nodes in the rack where the first data node is located; if the first target data node and the second target data node are not in the same rack, distributing the third batch of data nodes from the plurality of racks; selecting one data node from the third batch of data nodes as a third target data node; and the third target data node is used for storing a third data copy written by the client.
Optionally, based on at least the plurality of heartbeat packets, acquiring node load information of each data node includes: and calculating node load information of each data node according to the heartbeat packet of each data node in the previous time window and the times of being selected as a target data node in a plurality of continuous time windows.
Optionally, each heartbeat packet includes at least one parameter mean value of the corresponding data node in the previous time window; the at least one parameter mean comprises: the average value of input output quantity per second in the previous time window, the average value of the percentage of the total time of the I/O request, the average value of the system load in the previous time window and the average value of the bandwidth in the previous time window.
Optionally, the node load information of each data node is equal to the weighted sum between each parameter mean value in the heartbeat packet of the corresponding data node and the selection frequency coefficient of the corresponding data node; the selection frequency coefficient is obtained according to the frequency of the corresponding data node selected as a target data node in a plurality of continuous time windows, and the weight value of each parameter mean value and the weight of the selection frequency coefficient are dynamically adjusted.
An aspect of an embodiment of the present application further provides a dynamic load balancing system for a distributed file system, including: the first acquisition module is used for acquiring a plurality of heartbeat packets provided by a plurality of data nodes, wherein the plurality of data nodes are nodes for data storage in the distributed file system; the second acquisition module is used for acquiring node load information of each data node at least based on the plurality of heartbeat packets; the receiving module is used for receiving the writing request of the client; and a response module for responding to the write request: and selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node so as to butt-joint the writing task of the client through the plurality of target data nodes.
An aspect of the embodiments of the present application further provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the dynamic load balancing method for a distributed file system as described above when the computer program is executed.
An aspect of the embodiments of the present application further provides a computer readable storage medium comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the dynamic load balancing method for a distributed file system as described above when the computer program is executed.
According to the dynamic load balancing method, system, equipment and computer readable storage medium for the distributed file system, load related information of each data node is obtained through the heartbeat packet, and appropriate data nodes are selected for writing based on the load related information of each data node, so that the writing speed of a client is guaranteed. In addition, the embodiment adopts the heartbeat packet form to provide the load related information, so that the heartbeat time interval of the data node is smaller, the load of the data node with high load is reduced after the processing of the task is finished, the data node can be continuously distributed with the writing task, and the load deviation of the data node in the whole cluster is ensured to be kept in a smaller range so as to ensure the load balance.
Drawings
FIG. 1 schematically illustrates an application environment diagram of a dynamic load balancing method for a distributed file system according to an embodiment of the present application;
FIG. 2 schematically illustrates a flow chart of a dynamic load balancing method for a distributed file system according to an embodiment one of the present application;
FIG. 3 schematically illustrates another flow chart of a dynamic load balancing method for a distributed file system according to one embodiment of the present application;
FIG. 4 schematically illustrates another flow chart of a dynamic load balancing method for a distributed file system according to one embodiment of the present application;
FIG. 5 is a sub-step diagram of step S206 in FIG. 2;
FIG. 6 is a sub-step diagram of step S206 in FIG. 2;
FIG. 7 schematically illustrates an example reference diagram for selecting a target data node;
FIG. 8 schematically illustrates a block diagram of a dynamic load balancing system for a distributed file system according to a second embodiment of the present application; and
Fig. 9 schematically illustrates a hardware architecture diagram of a computer device adapted to implement a dynamic load balancing method for a distributed file system according to a third embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
It should be noted that the descriptions of "first," "second," etc. in the embodiments of the present application are for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be regarded as not exist and not within the protection scope of the present application.
In the description of the present application, it should be understood that the numerical references before the steps do not indicate the order of performing the steps, but are only used for convenience in describing the present application and distinguishing each step, and thus should not be construed as limiting the present application.
FIG. 1 schematically illustrates an environmental application schematic of a dynamic load balancing method for a distributed file system according to an embodiment of the present application. The environment schematic includes a distributed file system and clients 400. The distributed file system includes a management node 200 and a plurality of data nodes (e.g., datanodes 300-308).
The distributed file system (Distributed File System) is an extensible master-slave distributed file system.
A management node (NameNode) 200, responsible for storing the distributed file system metadata information and selecting data nodes to write for clients 400, etc.
The data nodes (e.g., datanodes 300-308) are responsible for storing, reading and writing of real files.
A Client (e.g., client 400) may access the distributed file system and write and store data to the distributed file system.
Services may be provided between management node 200, datanodes 300-308, and client 400 via one or more networks. The one or more networks include various network devices such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and/or the like. The one or more networks may include physical links, such as coaxial cable links, twisted pair cable links, fiber optic links, combinations thereof, and the like. The one or more networks may include wireless links, such as cellular links, satellite links, wi-Fi links, and the like.
The inventor finds that when the cluster size of the distributed file system is enlarged to a certain extent, because the data is distributed on different datanodes, the frequency difference of data writing/reading is huge, meanwhile, the configuration difference of each DataNode causes the load deviation between the datanodes to become large, and when the management node 200 selects the DataNode for writing for the client, the load difference between the datanodes is not balanced, so that the DataNode with higher load continuously writes a large amount of data, and the single DataNode is excessively loaded, thereby causing the writing delay of part of the clients.
The present application aims to enable the management node 200 to select a suitable DataNode for data writing based on the load condition of each DataNode, avoid delay of data processing speed caused by high-load DataNode, and ensure writing speed of the client 400.
Example 1
Fig. 2 schematically illustrates a flow chart of a dynamic load balancing method for a distributed file system according to an embodiment of the present application. The following description is made by taking the management node 200 as an execution body.
As shown in fig. 2, the dynamic load balancing method for the distributed file system may include steps S200 to S206, wherein:
Step S200, a plurality of Heartbeat packets (Heartbeat) provided by a plurality of data nodes are acquired.
The plurality of data nodes are nodes for data storage in the distributed file system, such as datanodes 300-308.
Compared with the method only used for reporting survival information, each heartbeat packet in the embodiment is also used for reporting load related information of the corresponding data node. The load related information includes IOPS, IOUtil, load, bandWidth, and the like. Wherein: (1) IPOS (Input/Output Operations Per Second, input/output per second) is a performance parameter used for performance testing of computer storage devices (e.g., hard disk, solid state disk, or storage area network). (2) IOUtil, the percentage of time taken to issue an I/O request, in total, is used to represent how busy a disk, etc. (3) Load may be a system Load parameter of a hardware device such as a CPU. (4) BandWidth, which refers to the amount of data that can be transmitted in a unit time, is an important indicator for measuring network performance. It should be understood that the above several parameters are not intended to limit the present application. Other parameters, such as disk usage, etc., can be added as needed.
In an exemplary embodiment, to smooth the calculated load values, spikes and glitches are filtered, with each data node providing a corresponding heartbeat packet in a sliding time window. The time window refers to: the time window is divided into smaller time segments, and each time segment is slid to the right by one grid, and each time segment is provided with an independent counter, so that the calculated load value can be smoothed, and peaks and burrs can be filtered. Therefore, each heartbeat packet includes at least one parameter mean value of the corresponding data node in the previous time window; the at least one parameter mean comprises: the average value of the IOPS in the previous time window, the average value of the IOUtil in the previous time window, the average value of the Load in the previous time window and the average value of the BandWidth in the previous time window.
Taking the DataNode 302 as an example, the DataNode 302 may start an independent thread to collect IOPS, IOUtil, load and BandWidth of itself, calculate the average value of each parameter in a sliding manner of a time window, and report the average value to the management node 200 through a heartbeat packet. For example, the DataNode 302 may take 6 minutes as a time window, and each heartbeat packet includes the IOPS mean, the IOUtil mean, the Load mean, and the BandWidth mean in the previous time window.
Step S202, based on at least the plurality of heartbeat packets, node load information of each data node is obtained.
The management node 200 analyzes the plurality of heartbeat packets to obtain IOPS, IOUtil, load, bandWidth and the like uploaded by each data node; node load information of each data node is acquired based on IOPS, IOUtil, load and BandWidth uploaded by each data node, and the like. The node load information may be a fractional value, the higher the fractional value representing the more busy the corresponding data node. As an example, score value=iops×corresponding weight value+ioutil×corresponding weight value+load×corresponding weight value+bandwidth×corresponding weight value. The weight value of each parameter can be dynamically adjusted according to different environments and requirements.
Considering that the data written recently will be read, the load of the data node written will increase, so the management node 200 may take the number of times each data node is written in the last period into consideration when selecting the data node. As shown in fig. 3, the step S202 may also be implemented by step S300: and calculating node load information of each data node according to the heartbeat packet of each data node in the previous time window and the times of being selected as a target data node in a plurality of continuous time windows.
Wherein the node load information of each data node is equal to a weighted sum between each parameter average value in the heartbeat packet of the corresponding data node and the selection frequency coefficient (e.g., the selection frequency average value) of the corresponding data node. The selection frequency coefficient is obtained according to the frequency of the corresponding data node selected as a target data node in a plurality of continuous time windows, and the weight value of each parameter mean value and the weight of the selection frequency coefficient are dynamically adjusted.
The following description is made by taking the DataNode 302 as an example:
(1) The DataNode 302 can provide a heartbeat packet every 6 minutes with 6 minutes as a time window.
(2) The management node 200 receives the heartbeat packet reported by the DataNode 302 to obtain IOPS, IOUtil, load and BandWidth of the DataNode 302.
(3) The management node 200 may select the last and consecutive 10 time windows, calculate the number of times the DataNode 302 is selected to access the write in each time window, and calculate the average of the number of selections based on the 10 time windows.
(4) The management node 200 may calculate the score value of the DataNode 302 as follows: IOPS corresponding weight value + IOUtil corresponding weight value + Load corresponding weight value + BandWidth corresponding weight value + the number of selections mean value corresponding weight value.
In step S204, a write Request (Request) of the client is received.
The write request is used to request the management node 200 to select a data node (i.e., a target data node) for which to write data.
Step S206, responding to the write request: and selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node so as to butt-joint the writing task of the client through the plurality of target data nodes.
After the management node 200 selects the plurality of target data nodes, information of the plurality of target data nodes may be fed back to the client 400, so that the client 400 may write data to the plurality of target data nodes. Wherein each target data node is configured to store a corresponding copy of data written by client 400.
As an example, the management node 200 may randomly allocate data nodes for the client 400 according to a default selection policy. And the default selection strategy is to select a target data node according to the information such as the connection number of the data node.
As an example, the management node 200 may also consider the overall load situation of the data node to implement write balancing of the data node.
The dynamic load balancing method for the distributed file system has the following technical advantages:
(1) In the prior art, when a management node allocates a writing task to a client, the writing task may be continuously allocated to a data node with high load, so that the data processing speed is slow, and the overall writing speed is affected.
In this embodiment, load related information of each data node is obtained through a heartbeat packet, and a data node with a lower load is selected for writing based on the load related information of each data node, so that the data node with a high load is prevented from being used for writing, and the writing speed of a client is ensured. Meanwhile, the DataNode with lower load is selected based on the load related information of each data node, so that the load condition of the whole data node is balanced, and the reading speed is ensured.
In addition, because the embodiment adopts the heartbeat packet to provide the load related information, the heartbeat time interval of the DataNode is smaller, the load of the DataNode with high load is reduced after the task is processed, the management node 200 can be continuously distributed and written into the task, and the load deviation of the DataNode in the whole cluster is ensured to be kept in a smaller range.
(2) In this embodiment, a time window-based heartbeat packet is adopted, and each heartbeat packet carries various parameter averages of a previous time window, so that the heartbeat packet can be used for smoothing calculated load values and filtering peaks and burrs.
(3) In this embodiment, when selecting the data node, the number of times that each data node is selected recently may be taken into consideration, so that the overall data node load condition is further balanced, and the writing speed and the reading speed are ensured.
(4) In this embodiment, the weights of the parameters may be dynamically adjusted in different time periods or in different environments, so that the overall data node load conditions in various situations are further balanced, and the writing speed and the reading speed are ensured.
(5) In this embodiment, the stored file may be further divided into multiple copies in the distributed file system, and different copies are placed on the datanodes of different racks, so that the copies are uniformly distributed in the cluster, and when a part of the datanodes fail, the file can be effectively prevented from being lost, so that the reliability of the distributed file system is improved.
The distributed file system includes a plurality of racks, each rack including one or more data nodes. The management node 200 may obtain rack information of the plurality of racks, and obtain a network topology of the distributed file system through the rack information. The rack information refers to the distribution of data nodes in a machine room (for example, the data nodes 300-304 are in one rack, the data nodes 306-308 are in another rack), and the nodes of the same rack share the same external facilities such as a switch and a power supply, so that the possibility of occurrence of collective faults is high. The rack information can be provided by the machine room operation and maintenance and input into the configuration file. The configuration file is loaded when the management node 200 service is started.
In an exemplary embodiment, when the management node 200 is to select a plurality of data nodes for the client 400 to store a plurality of data copies, as shown in fig. 4, the step S206 may include steps S400 to S404, wherein: step S400, selecting an ith batch of data nodes from an ith group of racks, and selecting one of the ith batch of data nodes as an ith target data node according to node load information of each data node in the ith batch of data nodes. Wherein: when i=1, the first set of racks includes the plurality of racks, the first batch of data nodes includes at least two data nodes of the plurality of racks, and the first target data node is configured to store a first copy of data written by the client; when i is more than or equal to 2, the ith group of racks comprises other racks of the plurality of racks except for the racks where the first to ith target data nodes are located; the ith batch of data nodes comprise at least two data nodes in the other racks, and the ith target data node is used for storing the ith data copy written by the client. i is a positive integer, and the maximum value of i is determined by the number of data copies. In this embodiment, the data copies are distributed among different racks of the distributed file system to improve data security.
In another exemplary embodiment, as shown in fig. 5, when the management node 200 is to select a plurality of data nodes for the client to store two copies of data, the step S206 may include steps S500 to S510, wherein: step S500, distributing a first batch of data nodes from the racks; step S502, selecting one data node from the first batch of data nodes as a first target data node; the first target data node is used for storing a first data copy written by the client; step S504, distributing a second batch of data nodes from other racks of the plurality of racks except the rack where the first data node is located; step S506, judging whether the second batch of data nodes are successfully distributed from other racks except the rack where the first data node is located; step S508, if the second batch of data nodes fails to be allocated from the other racks except the rack where the first data node is located, then: distributing the second batch of data nodes in a rack where the first data node is located; step S510, selecting one data node from the second batch of data nodes as a second target data node; the second target data node is used for storing a second data copy written by the client.
When the management node 200 is to select a plurality of data nodes for the client to store three copies of data, as shown in fig. 6, the step S206 may further include steps S512 to S522, in which: step S512, determining whether the first target data node and the second target data node are located in the same rack; step S514, if the first target data node and the second target data node are located in the same rack, allocating a third batch of data nodes from the other racks of the plurality of racks except for the rack in which the first data node is located; step S516, determining whether the third batch of data nodes are successfully allocated from the other racks except the rack where the first data node is located; step S518, if the third lot of data nodes are allocated to fail from the other racks except the rack where the first data node is located, the third lot of data nodes are allocated to the rack where the first data node is located; step S520, if the first target data node and the second target data node are not in the same rack, allocating the third batch of data nodes from the plurality of racks; step S522, selecting one of the data nodes from the third batch of data nodes as a third target data node; and the third target data node is used for storing a third data copy written by the client.
As shown in fig. 7, in order to make the present embodiment more clear, a reference example is provided below.
S1: a write request is received by client 400.
Selecting a first target data node for writing a first copy of data:
s2: two data nodes are randomly distributed from all data nodes of the distributed file system.
S3: according to the node load information of the two data nodes in step S1, a data node with smaller load is selected as the first target data node (DataNode a). The DataNode a is in rack a of a plurality of racks in the distributed file system.
Selecting a second target data node for writing a second copy of data:
s4: two data nodes are randomly allocated from the other racks than the rack a among the plurality of racks.
S5: and judging whether the two data nodes are successfully distributed in the step S4. If not, go to step S6, otherwise go to step S7.
S6: again two data nodes are randomly allocated from rack a.
S7: and selecting the data node with smaller load as a second target data node (DataNodeB) according to the node load information of the two data nodes in the step S4 or the step S6. The DataNodeB and DataNodeB are not the same data node.
S8: it is determined whether the number of data copies is greater than 2. If so, step S9 is entered, otherwise the data node selection operation is ended.
Selecting a third target data node for writing a third copy of data:
s9: it is determined whether the DataNodeB and the DataNodeB are located in the same rack (i.e., rack a). If yes, go to step S10, otherwise go to step S13.
S10: two data nodes are randomly allocated from the other racks than the rack a among the plurality of racks.
S11: it is determined whether the allocation is successful in step S10. If not, go to step S12; if so, step S13 is entered.
S12: again two data nodes are randomly allocated from rack a.
S13: two data nodes are randomly distributed from all data nodes of the distributed file system.
S14: the two data nodes obtained in step S10, S12 or S13 select the data node with smaller load as the third target data node (datanode) according to the node load information of the two data nodes.
And so on, determining whether the number of data copies is greater than 3. If so, a fourth target data node (DataNodeD) is selected.
In this embodiment, in response to the write request of the client 400, the management node 200 combines the rack to randomly select several data nodes from the entire data node topology for comparison, so as to further efficiently and quickly select a suitable data node.
In addition, for the convenience of user's checking, node load information, heartbeat packet information, etc. of each data node can all be visually displayed through JMX (Java Management Extensions, java management extension) interface.
Example two
Fig. 8 schematically illustrates a block diagram of a dynamic load balancing system for a distributed file system according to a second embodiment of the present application, which may be partitioned into one or more program modules, stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. Program modules in the embodiments of the present application refer to a series of computer program instruction segments capable of implementing specific functions, and the following description specifically describes the functions of each program module in the embodiment.
As shown in fig. 8, the dynamic load balancing system 800 for a distributed file system may include, among others:
the first obtaining module 810 is configured to obtain a plurality of heartbeat packets provided by a plurality of data nodes, where the plurality of data nodes are nodes used for data storage in the distributed file system.
The second obtaining module 820 is configured to obtain node load information of each data node based at least on the plurality of heartbeat packets.
A receiving module 830, configured to receive a write request of a client.
A response module 840, configured to respond to the write request: and selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node so as to butt-joint the writing task of the client through the plurality of target data nodes.
In an exemplary embodiment, the distributed file system includes a plurality of racks, each rack including one or more data nodes. The response module 840 is further configured to: selecting an ith batch of data nodes from an ith group of racks, and selecting one of the data nodes from the ith batch of data nodes as an ith target data node according to node load information of each data node in the ith batch of data nodes; wherein: when i=1, the first set of racks includes the plurality of racks, the first batch of data nodes includes at least two data nodes of the plurality of racks, and the first target data node is configured to store a first copy of data written by the client; when i is more than or equal to 2, the ith group of racks comprises other racks of the plurality of racks except for the racks where the first to ith target data nodes are located; the ith batch of data nodes comprise at least two data nodes in the other racks, and the ith target data node is used for storing the ith data copy written by the client.
In an exemplary embodiment, the distributed file system includes a plurality of racks, each rack including one or more data nodes. The response module 840 is further configured to: : distributing a first batch of data nodes from the plurality of racks; selecting one of the first batch of data nodes as a first target data node; the first target data node is used for storing a first data copy written by the client; distributing a second batch of data nodes from other racks of the plurality of racks except the rack where the first data node is located; judging whether the second batch of data nodes are successfully distributed from other racks except the rack where the first data node is located; if the second batch of data nodes fails to be allocated from the other racks except the rack where the first data node is located, then: distributing the second batch of data nodes in a rack where the first data node is located; selecting one data node from the second batch of data nodes as a second target data node; the second target data node is used for storing a second data copy written by the client.
In an exemplary embodiment, the response module 840 is further configured to: judging whether the first target data node and the second target data node are positioned in the same rack or not; if the first target data node and the second target data node are located in the same rack, distributing a third batch of data nodes from other racks of the plurality of racks except the rack where the first data node is located; judging whether the third batch of data nodes are successfully distributed from other racks except the rack where the first data node is located in the plurality of racks; if the third batch of data nodes are failed to be distributed from other racks of the plurality of racks except the rack where the first data node is located, distributing the third batch of data nodes in the rack where the first data node is located; if the first target data node and the second target data node are not in the same rack, distributing the third batch of data nodes from the plurality of racks; selecting one data node from the third batch of data nodes as a third target data node; and the third target data node is used for storing a third data copy written by the client.
In an exemplary embodiment, the second obtaining module 820 is configured to: and calculating node load information of each data node according to the heartbeat packet of each data node in the previous time window and the times of being selected as a target data node in a plurality of continuous time windows.
In an exemplary embodiment, each heartbeat packet includes at least one parameter mean value for the respective data node in the previous time window; the at least one parameter mean comprises: the average value of input output quantity per second in the previous time window, the average value of the percentage of the total time of the I/O request, the average value of the system load in the previous time window and the average value of the bandwidth in the previous time window.
In an exemplary embodiment, the node load information of each data node is equal to a weighted sum between each parameter mean value in the heartbeat packet of the corresponding data node and the selection count coefficient of the corresponding data node; the selection frequency coefficient is obtained according to the frequency of the corresponding data node selected as a target data node in a plurality of continuous time windows, and the weight value of each parameter mean value and the weight of the selection frequency coefficient are dynamically adjusted.
Example III
Fig. 9 schematically shows a hardware architecture diagram of a computer device 9 adapted to implement a dynamic load balancing method for a distributed file system according to a third embodiment of the present application. The computer device 9 may be provided as a management node 200 or as a component of a management node 200. In the present embodiment, the computer device 9 is a device capable of automatically performing numerical calculation and/or information processing in accordance with instructions set or stored in advance. For example, it may be a workstation, rack server, blade server, tower server, or rack server (including a stand-alone server, or a server cluster made up of multiple servers), etc. As shown in fig. 9, the computer device 9 includes at least, but is not limited to: memory 910, processor 920, and network interface 930 may be communicatively linked to each other by a system bus. Wherein:
the memory 910 includes at least one type of computer-readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 910 may be an internal storage module of the computer device 9, such as a hard disk or memory of the computer device 9. In other embodiments, the memory 910 may also be an external storage device of the computer device 9, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 9. Of course, the memory 910 may also include both internal memory modules of the computer device 9 and external memory devices. In this embodiment, the memory 910 is typically used to store an operating system and various types of application software installed on the computer device 9, such as program code for a dynamic load balancing method for a distributed file system. In addition, the memory 910 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 920 may be a central processing unit (Central Processing Unit, simply CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 920 is typically used to control the overall operation of the computer device 9, such as performing control and processing related to data interaction or communication with the computer device 9, and the like. In this embodiment, the processor 920 is configured to execute program codes or process data stored in the memory 910.
The network interface 930 may include a wireless network interface or a wired network interface, which network interface 930 is typically used to establish communication links between the computer device 9 and other computer devices. For example, the network interface 930 is used to connect the computer device 9 with an external terminal through a network, establish a data transmission channel and a communication link between the computer device 9 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a global system for mobile communications (Global System of Mobile communication, abbreviated as GSM), wideband code division multiple access (Wideband Code Division Multiple Access, abbreviated as WCDMA), a 4G network, a 5G network, bluetooth (Bluetooth), wi-Fi, etc.
It should be noted that FIG. 9 only shows a computer device having components 910-930, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead.
In this embodiment, the dynamic load balancing method for a distributed file system stored in the memory 910 may also be divided into one or more program modules and executed by one or more processors (the processor 920 in this embodiment) to complete the embodiments of the present application.
Example IV
The present application also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the dynamic load balancing method for a distributed file system in embodiment one.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of a computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card), etc. that are provided on the computer device. Of course, the computer-readable storage medium may also include both internal storage units of a computer device and external storage devices. In this embodiment, the computer readable storage medium is typically used to store an operating system and various types of application software installed on a computer device, such as program code for a dynamic load balancing method for a distributed file system in the embodiment, and the like. Furthermore, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
It will be apparent to those skilled in the art that the modules or steps of the embodiments of the application described above may be implemented in a general purpose computing device, they may be centralized on a single computing device, or distributed over a network of computing devices, or they may alternatively be implemented in program code executable by computing devices, such that they may be stored in a memory device and, in some cases, executed in a different order than that shown or described, or they may be implemented as individual integrated circuit modules, or as individual integrated circuit modules. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.
It should be noted that the foregoing is only a preferred embodiment of the present application, and is not intended to limit the scope of the patent protection of the present application, and all equivalent structures or equivalent processes using the descriptions and the contents of the present application or direct or indirect application to other related technical fields are included in the scope of the patent protection of the present application.
Claims (8)
1. A method for dynamic load balancing of a distributed file system, the method comprising:
acquiring a plurality of heartbeat packets provided by a plurality of data nodes, wherein the data nodes are nodes for data storage in the distributed file system;
acquiring node load information of each data node at least based on the plurality of heartbeat packets;
receiving a writing request of a client; and
Responding to the writing request; selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node so as to butt-joint the writing task of the client through the plurality of target data nodes;
wherein the distributed file system comprises a plurality of racks, each rack comprising one or more data nodes; selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node, wherein the method comprises the following steps:
distributing a first batch of data nodes from the plurality of racks;
selecting one of the first batch of data nodes as a first target data node; the first target data node is used for storing a first data copy written by the client;
Distributing a second batch of data nodes from other racks of the plurality of racks except the rack where the first data node is located;
judging whether the second batch of data nodes are successfully distributed from other racks except the rack where the first data node is located;
if the second batch of data nodes fails to be allocated from the other racks except the rack where the first data node is located, then: distributing the second batch of data nodes in a rack where the first data node is located; and
Selecting one of the second plurality of data nodes as a second target data node; the second target data node is used for storing a second data copy written by the client;
judging whether the first target data node and the second target data node are positioned in the same rack or not;
if the first target data node and the second target data node are located in the same rack, distributing a third batch of data nodes from other racks of the plurality of racks except the rack where the first data node is located;
judging whether the third batch of data nodes are successfully distributed from other racks except the rack where the first data node is located in the plurality of racks;
If the third batch of data nodes are failed to be distributed from other racks of the plurality of racks except the rack where the first data node is located, distributing the third batch of data nodes in the rack where the first data node is located;
if the first target data node and the second target data node are not in the same rack, distributing the third batch of data nodes from the plurality of racks; and
Selecting one of the third batch of data nodes as a third target data node; and the third target data node is used for storing a third data copy written by the client.
2. The method of dynamic load balancing for a distributed file system of claim 1, wherein the distributed file system comprises a plurality of racks, each rack comprising one or more data nodes; selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node, wherein the method comprises the following steps:
selecting an ith batch of data nodes from an ith group of racks, and selecting one of the data nodes from the ith batch of data nodes as an ith target data node according to node load information of each data node in the ith batch of data nodes; wherein:
When i=1, the first set of racks includes the plurality of racks, the first batch of data nodes includes at least two data nodes of the plurality of racks, and the first target data node is configured to store a first copy of data written by the client;
when i is more than or equal to 2, the ith group of racks comprises other racks of the plurality of racks except for the racks where the first to ith target data nodes are located; the ith batch of data nodes comprise at least two data nodes in the other racks, and the ith target data node is used for storing the ith data copy written by the client.
3. The dynamic load balancing method for a distributed file system according to any one of claims 1 to 2, wherein obtaining node load information of each data node based at least on the plurality of heartbeat packets comprises:
and calculating node load information of each data node according to the heartbeat packet of each data node in the previous time window and the times of being selected as a target data node in a plurality of continuous time windows.
4. A method for dynamic load balancing of a distributed file system according to claim 3, wherein each heartbeat packet includes at least one parameter mean of the respective data node in the previous time window; the at least one parameter mean comprises: the average value of input output quantity per second in the previous time window, the average value of the percentage of the total time of the I/O request, the average value of the system load in the previous time window and the average value of the bandwidth in the previous time window.
5. The dynamic load balancing method for a distributed file system according to claim 4, wherein the node load information of each data node is equal to a weighted sum between each parameter mean value in a heartbeat packet of the corresponding data node and a selection count coefficient of the corresponding data node; the selection frequency coefficient is obtained according to the frequency of the corresponding data node selected as a target data node in a plurality of continuous time windows, and the weight value of each parameter mean value and the weight of the selection frequency coefficient are dynamically adjusted.
6. A dynamic load balancing system for a distributed file system, comprising:
the first acquisition module is used for acquiring a plurality of heartbeat packets provided by a plurality of data nodes, wherein the plurality of data nodes are nodes for data storage in the distributed file system;
the second acquisition module is used for acquiring node load information of each data node at least based on the plurality of heartbeat packets;
the receiving module is used for receiving the writing request of the client; and
A response module, configured to respond to the write request: selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node so as to butt-joint the writing task of the client through the plurality of target data nodes;
Wherein the distributed file system comprises a plurality of racks, each rack comprising one or more data nodes; selecting a plurality of target data nodes from the plurality of data nodes according to the node load information of each data node, and further comprising:
distributing a first batch of data nodes from the plurality of racks;
selecting one of the first batch of data nodes as a first target data node; the first target data node is used for storing a first data copy written by the client;
distributing a second batch of data nodes from other racks of the plurality of racks except the rack where the first data node is located;
judging whether the second batch of data nodes are successfully distributed from other racks except the rack where the first data node is located;
if the second batch of data nodes fails to be allocated from the other racks except the rack where the first data node is located, then: distributing the second batch of data nodes in a rack where the first data node is located; and
Selecting one of the second plurality of data nodes as a second target data node; the second target data node is used for storing a second data copy written by the client;
Judging whether the first target data node and the second target data node are positioned in the same rack or not;
if the first target data node and the second target data node are located in the same rack, distributing a third batch of data nodes from other racks of the plurality of racks except the rack where the first data node is located;
judging whether the third batch of data nodes are successfully distributed from other racks except the rack where the first data node is located in the plurality of racks;
if the third batch of data nodes are failed to be distributed from other racks of the plurality of racks except the rack where the first data node is located, distributing the third batch of data nodes in the rack where the first data node is located;
if the first target data node and the second target data node are not in the same rack, distributing the third batch of data nodes from the plurality of racks; and
Selecting one of the third batch of data nodes as a third target data node; and the third target data node is used for storing a third data copy written by the client.
7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor is adapted to implement the steps of the dynamic load balancing method for a distributed file system according to any of claims 1 to 5 when the computer program is executed by the processor.
8. A computer readable storage medium, characterized in that it has stored therein a computer program executable by at least one processor to cause the at least one processor to perform the steps of the dynamic load balancing method for a distributed file system according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011107475.2A CN112416888B (en) | 2020-10-16 | 2020-10-16 | Dynamic load balancing method and system for distributed file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011107475.2A CN112416888B (en) | 2020-10-16 | 2020-10-16 | Dynamic load balancing method and system for distributed file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112416888A CN112416888A (en) | 2021-02-26 |
| CN112416888B true CN112416888B (en) | 2024-03-12 |
Family
ID=74854642
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011107475.2A Active CN112416888B (en) | 2020-10-16 | 2020-10-16 | Dynamic load balancing method and system for distributed file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112416888B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115794660B (en) * | 2023-02-06 | 2023-05-16 | 青软创新科技集团股份有限公司 | Control method, device and system based on distributed program evaluation |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011140951A1 (en) * | 2010-08-25 | 2011-11-17 | 华为技术有限公司 | Method, device and system for load balancing |
| CN103152395A (en) * | 2013-02-05 | 2013-06-12 | 北京奇虎科技有限公司 | Storage method and device of distributed file system |
| CN104852857A (en) * | 2014-02-14 | 2015-08-19 | 航天信息股份有限公司 | Distributed data transmission method and system based on load balancing |
| CN105897499A (en) * | 2015-12-07 | 2016-08-24 | 乐视云计算有限公司 | Distributed storage system node state monitoring method, center code and system |
| CN106027647A (en) * | 2016-05-20 | 2016-10-12 | 云南云电同方科技有限公司 | LXPFS (Linux XProgram File System) cluster distributed file storage system |
| CN107172214A (en) * | 2017-07-18 | 2017-09-15 | 广州华多网络科技有限公司 | A kind of service node with load balancing finds method and device |
| CN109740037A (en) * | 2019-01-02 | 2019-05-10 | 山东省科学院情报研究所 | The distributed online real-time processing method of multi-source, isomery fluidised form big data and system |
| CN110035122A (en) * | 2019-04-04 | 2019-07-19 | 科讯嘉联信息技术有限公司 | A kind of load-balancing method based on dynamic probability model, device and system |
| CN110781156A (en) * | 2019-09-20 | 2020-02-11 | 苏州浪潮智能科技有限公司 | Data node distribution method, equipment and medium |
| CN111726415A (en) * | 2020-06-30 | 2020-09-29 | 国电南瑞科技股份有限公司 | TCP long connection load balancing scheduling method and system based on negative feedback mechanism |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9348707B2 (en) * | 2013-12-18 | 2016-05-24 | International Business Machines Corporation | Dynamically adjusting the number of replicas of a file according to the probability that the file will be accessed within a distributed file system |
-
2020
- 2020-10-16 CN CN202011107475.2A patent/CN112416888B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011140951A1 (en) * | 2010-08-25 | 2011-11-17 | 华为技术有限公司 | Method, device and system for load balancing |
| CN103152395A (en) * | 2013-02-05 | 2013-06-12 | 北京奇虎科技有限公司 | Storage method and device of distributed file system |
| CN104852857A (en) * | 2014-02-14 | 2015-08-19 | 航天信息股份有限公司 | Distributed data transmission method and system based on load balancing |
| CN105897499A (en) * | 2015-12-07 | 2016-08-24 | 乐视云计算有限公司 | Distributed storage system node state monitoring method, center code and system |
| CN106027647A (en) * | 2016-05-20 | 2016-10-12 | 云南云电同方科技有限公司 | LXPFS (Linux XProgram File System) cluster distributed file storage system |
| CN107172214A (en) * | 2017-07-18 | 2017-09-15 | 广州华多网络科技有限公司 | A kind of service node with load balancing finds method and device |
| CN109740037A (en) * | 2019-01-02 | 2019-05-10 | 山东省科学院情报研究所 | The distributed online real-time processing method of multi-source, isomery fluidised form big data and system |
| CN110035122A (en) * | 2019-04-04 | 2019-07-19 | 科讯嘉联信息技术有限公司 | A kind of load-balancing method based on dynamic probability model, device and system |
| CN110781156A (en) * | 2019-09-20 | 2020-02-11 | 苏州浪潮智能科技有限公司 | Data node distribution method, equipment and medium |
| CN111726415A (en) * | 2020-06-30 | 2020-09-29 | 国电南瑞科技股份有限公司 | TCP long connection load balancing scheduling method and system based on negative feedback mechanism |
Non-Patent Citations (1)
| Title |
|---|
| 分布式数据传输系统关键技术设计;万翔等;《舰船电子工程》;20150420(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112416888A (en) | 2021-02-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108874640B (en) | Cluster performance evaluation method and device | |
| US10637916B2 (en) | Method and device for storage resource allocation for video cloud storage | |
| CN108683720B (en) | Container cluster service configuration method and device | |
| US7631034B1 (en) | Optimizing node selection when handling client requests for a distributed file system (DFS) based on a dynamically determined performance index | |
| US7467291B1 (en) | System and method for calibrating headroom margin | |
| CN107426274B (en) | Method and system for service application and monitoring, analyzing and scheduling based on time sequence | |
| CN107656807B (en) | Automatic elastic expansion method and device for virtual resources | |
| CN112256438B (en) | Load balancing control method and device, storage medium and electronic equipment | |
| CN111966289B (en) | Partition optimization method and system based on Kafka cluster | |
| CN109981702B (en) | File storage method and system | |
| CN111277640B (en) | User request processing method, device, system, computer equipment and storage medium | |
| CN111522636A (en) | Application container adjusting method, application container adjusting system, computer readable medium and terminal device | |
| CN112269661B (en) | Partition migration method and device based on Kafka cluster | |
| CN109067607B (en) | Method and device for adjusting rated bandwidth | |
| CN112416888B (en) | Dynamic load balancing method and system for distributed file system | |
| CN113688001A (en) | Dynamic balancing method and device for server hard disk power consumption, terminal and storage medium | |
| CN108664321B (en) | System resource allocation adjusting method and device | |
| CN110515728B (en) | Server scheduling method and device, electronic equipment and machine-readable storage medium | |
| CN114490078A (en) | Dynamic capacity reduction and expansion method, device and equipment for micro-service | |
| CN111444074A (en) | Data monitoring method and device, electronic equipment and readable storage medium | |
| CN114173396B (en) | Method and device for determining terminal networking time, electronic equipment and storage medium | |
| CN109257256A (en) | Apparatus monitoring method, device, computer equipment and storage medium | |
| CN112367384B (en) | Kafka cluster-based dynamic speed limiting method and device and computer equipment | |
| CN112948104B (en) | Load balancing data acquisition method and device | |
| CN111367878B (en) | IPFS node monitoring method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |