CN113806082A - Method, device and equipment for collecting node performance data and readable medium - Google Patents
Method, device and equipment for collecting node performance data and readable medium Download PDFInfo
- Publication number
- CN113806082A CN113806082A CN202111035188.XA CN202111035188A CN113806082A CN 113806082 A CN113806082 A CN 113806082A CN 202111035188 A CN202111035188 A CN 202111035188A CN 113806082 A CN113806082 A CN 113806082A
- Authority
- CN
- China
- Prior art keywords
- monitoring node
- monitoring
- nodes
- node
- performance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012544 monitoring process Methods 0.000 claims abstract description 276
- 238000012545 processing Methods 0.000 claims abstract description 88
- 230000007334 memory performance Effects 0.000 claims description 37
- 238000004590 computer program Methods 0.000 claims description 11
- 230000003993 interaction Effects 0.000 abstract description 6
- 238000001514 detection method Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a method, a device, equipment and a readable medium for collecting node performance data, wherein the method comprises the following steps: the method comprises the steps that a main monitoring node in a distributed storage cluster collects configuration information of all monitoring nodes; calculating the processing capacity of each monitoring node based on the collected configuration information; distributing a plurality of common nodes for each monitoring node according to the processing capacity of each monitoring node, and monitoring and acquiring data of the corresponding common nodes by each monitoring node; and each monitoring node sends the collected data to the main monitoring node. By using the scheme of the invention, the work of alarm detection and performance data acquisition of the nodes in the cluster can be not influenced, the pressure of the main monitoring node occupied by resources such as a CPU (central processing unit), network bandwidth and the like due to the fact that the main monitoring node needs to carry out information interaction on other nodes is reduced, the utilization rate of other monitoring node resources is improved, the pressure of the main monitoring node can be divided, and a pressure balance is maintained among all monitoring nodes.
Description
Technical Field
The present invention relates to the field of computers, and more particularly, to a method, an apparatus, a device, and a readable medium for node performance data acquisition.
Background
In a distributed storage cluster environment, nodes may be divided into MON (monitor) nodes and non-MON nodes, where the MON nodes may be further divided into a master MON node and a common MON node. In order to ensure the normal operation and use of the cluster, the state of each node in the cluster needs to be monitored and performance data needs to be collected, and the main MON node is responsible for completing the operations. However, as the cluster size increases, frequent data exchange may cause the load on the main MON node to increase, which may cause the main MON node to go down in severe cases, thereby causing unpredictable influence to the cluster.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method, an apparatus, a device, and a readable medium for collecting node performance data, which may not affect the alarm detection and performance data collection of nodes in a cluster, and simultaneously reduce the pressure of the main monitoring node occupied by resources such as a CPU and a network bandwidth of the main monitoring node due to information interaction required by the main monitoring node on other nodes, improve the utilization rate of resources of other monitoring nodes, and may distribute the pressure of the main monitoring node, and maintain a pressure balance among the monitoring nodes.
In view of the above, one aspect of the present invention provides a method for collecting node performance data, including the following steps:
the method comprises the steps that a main monitoring node in a distributed storage cluster collects configuration information of all monitoring nodes;
calculating the processing capacity of each monitoring node based on the collected configuration information;
distributing a plurality of common nodes for each monitoring node according to the processing capacity of each monitoring node, and monitoring and acquiring data of the corresponding common nodes by each monitoring node;
and each monitoring node sends the collected data to the main monitoring node.
According to an embodiment of the present invention, the collecting, by the master monitoring node in the distributed storage cluster, the configuration information of all the monitoring nodes includes:
the main monitoring node respectively collects the number of CPUs, the number of cores of a single CPU, CPU main frequency, the size of a memory and the size of a CPU cache of each monitoring node.
According to one embodiment of the invention, calculating the processing power of each monitoring node based on the collected configuration information comprises:
setting a first proportional parameter and a second proportional parameter of the influence of a CPU and a memory on the performance of a monitoring node;
calculating the CPU performance of each monitoring node based on the collected configuration information of the monitoring nodes;
calculating the memory performance of each monitoring node based on the collected configuration information of the monitoring nodes;
and calculating the processing capacity of each monitoring node based on the calculated CPU performance, memory performance and proportional parameters.
According to one embodiment of the invention, calculating the CPU performance of each monitoring node based on the collected configuration information of the monitoring nodes comprises:
according to the formula: and calculating the CPU performance of each monitoring node, wherein the CPU performance is the main frequency multiplied by the total number of cores multiplied by the CPU cache size.
According to an embodiment of the present invention, calculating the memory performance of each monitoring node based on the collected configuration information of the monitoring nodes includes:
according to the formula: and calculating the CPU performance of each monitoring node according to the memory performance which is the memory size/1G.
According to an embodiment of the present invention, calculating the processing capacity of each monitoring node based on the calculated CPU performance and memory performance and the proportional parameter includes:
according to the formula: and calculating the processing capacity of each monitoring node, wherein the processing capacity of each monitoring node is the CPU performance multiplied by the first proportional parameter plus the memory performance multiplied by the second proportional parameter.
According to an embodiment of the present invention, allocating a plurality of common nodes to each monitoring node according to the processing capability of each monitoring node, wherein the monitoring and collecting of data of the corresponding common nodes by each monitoring node includes:
allocating a common node for the monitoring node with the maximum processing capacity, and reducing the processing capacity of the monitoring node with the maximum processing capacity by a preset value;
and repeating the steps until all the common nodes are distributed.
According to another aspect of the present invention, there is also provided an apparatus for node performance data acquisition, the apparatus comprising:
the collection module is configured to collect configuration information of all monitoring nodes by the main monitoring nodes in the distributed storage cluster;
a computing module configured to compute a processing power of each monitoring node based on the collected configuration information;
the distribution module is configured to distribute a plurality of common nodes to each monitoring node according to the processing capacity of each monitoring node, and each monitoring node monitors and collects data of the corresponding common nodes;
and the summarizing module is configured to enable each monitoring node to send the acquired data to the main monitoring node.
According to one embodiment of the invention, the collection module is further configured to:
the main monitoring node respectively collects the number of CPUs, the number of cores of a single CPU, CPU main frequency, the size of a memory and the size of a CPU cache of each monitoring node.
According to one embodiment of the invention, the computing module is further configured to:
setting a first proportional parameter and a second proportional parameter of the influence of a CPU and a memory on the performance of a monitoring node;
calculating the CPU performance of each monitoring node based on the collected configuration information of the monitoring nodes;
calculating the memory performance of each monitoring node based on the collected configuration information of the monitoring nodes;
and calculating the processing capacity of each monitoring node based on the calculated CPU performance, memory performance and proportional parameters.
According to one embodiment of the invention, the computing module is further configured to:
according to the formula: and calculating the CPU performance of each monitoring node, wherein the CPU performance is the main frequency multiplied by the total number of cores multiplied by the CPU cache size.
According to one embodiment of the invention, the computing module is further configured to:
according to the formula: and calculating the CPU performance of each monitoring node according to the memory performance which is the memory size/1G.
According to one embodiment of the invention, the computing module is further configured to:
according to the formula: and calculating the processing capacity of each monitoring node, wherein the processing capacity of each monitoring node is the CPU performance multiplied by the first proportional parameter plus the memory performance multiplied by the second proportional parameter.
According to yet another aspect of the invention, the allocation module is further configured to:
allocating a common node for the monitoring node with the maximum processing capacity, and reducing the processing capacity of the monitoring node with the maximum processing capacity by a preset value;
and repeating the steps until all the common nodes are distributed.
In another aspect of an embodiment of the present invention, there is also provided a computer apparatus including:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of any of the methods described above.
In another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium storing a computer program, which when executed by a processor implements the steps of any one of the above-mentioned methods.
The invention has the following beneficial technical effects: the method for collecting the node performance data provided by the embodiment of the invention collects the configuration information of all monitoring nodes through the main monitoring node in the distributed storage cluster; calculating the processing capacity of each monitoring node based on the collected configuration information; distributing a plurality of common nodes for each monitoring node according to the processing capacity of each monitoring node, and monitoring and acquiring data of the corresponding common nodes by each monitoring node; according to the technical scheme, each monitoring node sends the collected data to the main monitoring node, the work of alarm detection and performance data collection of the nodes in the cluster can be unaffected, meanwhile, the pressure occupied by resources such as a CPU (central processing unit) and network bandwidth of the main monitoring node due to the fact that the main monitoring node needs to carry out information interaction on other nodes is reduced, the utilization rate of other monitoring node resources is improved, the pressure of the main monitoring node can be distributed, and the balance of pressure can be maintained among the monitoring nodes.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
FIG. 1 is a schematic flow chart diagram of a method of node performance data acquisition in accordance with one embodiment of the present invention;
FIG. 2 is a schematic diagram of an apparatus for node performance data acquisition according to one embodiment of the present invention;
FIG. 3 is a schematic diagram of a computer device according to one embodiment of the present invention;
fig. 4 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
In view of the above, a first aspect of embodiments of the present invention proposes an embodiment of a method for node performance data acquisition. Fig. 1 shows a schematic flow diagram of the method.
As shown in fig. 1, the method may include the steps of:
s1 the master monitoring node in the distributed storage cluster collects the configuration information of all monitoring nodes.
When the distributed storage cluster is built, the main MON node actively collects configuration information of all the MON nodes, wherein the configuration information comprises the number of CPUs (central processing units) of each monitoring node, the number of cores of a single CPU, the CPU main frequency, the memory size and the CPU cache size.
S2 calculates the processing power of each monitoring node based on the collected configuration information.
The processing capacity of each MON node can be calculated through configuration information of each MON node, factors which have large influence on the performance of the MON node mainly comprise a CPU and a memory, the influence ratio of the CPU and the memory on the performance of the MON node is set to be m: n, and for the CPU, main factors which influence the performance of the CPU have a main frequency, the total number of cores and the cache size of the CPU, so that the processing capacity of the MON node can be calculated according to a formula: the CPU performance of each MON node is calculated by multiplying the CPU performance by the master frequency × the total number of cores × the cache size, and the memory performance is mainly affected by the memory size, so that the CPU performance can be calculated according to the formula: the memory performance of each MON node is calculated by memory size/1G, and the processing capacity of a single MON node is calculated by the following formula: the node processing capability is m × CPU performance + n × memory performance.
S3, distributing a plurality of common nodes for each monitoring node according to the processing capacity of each monitoring node, and each monitoring node monitors and collects the data of the corresponding common nodes.
Firstly, distributing a common node for the monitoring node with the maximum processing capacity, enabling the monitoring node to monitor the information of the common node, then reducing the processing capacity of the monitoring node with the maximum processing capacity by a preset value, then selecting the monitoring node with the maximum processing capacity from the monitoring nodes, distributing a common node for the monitoring node, enabling the monitoring node to monitor the information of the common node, then reducing the processing capacity of the monitoring node with the maximum processing capacity by the preset value, and repeating the steps until all the common nodes are completely distributed.
S4, each monitoring node sends the collected data to the main monitoring node.
And finally, the data collected by each monitoring node is collected and sent to the main monitoring node, so that the range of data interaction with the main monitoring node is reduced, and the pressure of resources such as a main monitoring node network and a CPU (central processing unit) is reduced.
By the technical scheme, the alarm detection and performance data acquisition work of the nodes in the cluster can be unaffected, the pressure of the main monitoring node occupied by resources such as a CPU (central processing unit) and network bandwidth caused by information interaction of the main monitoring node on other nodes is reduced, the utilization rate of other monitoring node resources is improved, the pressure of the main monitoring node can be distributed, and a pressure balance can be maintained among the monitoring nodes.
In a preferred embodiment of the present invention, the collecting, by the master monitoring node in the distributed storage cluster, the configuration information of all the monitoring nodes includes:
the main monitoring node respectively collects the number of CPUs, the number of cores of a single CPU, CPU main frequency, the size of a memory and the size of a CPU cache of each monitoring node.
In a preferred embodiment of the present invention, calculating the processing power of each monitoring node based on the collected configuration information comprises:
setting a first proportional parameter and a second proportional parameter of the influence of a CPU and a memory on the performance of a monitoring node;
calculating the CPU performance of each monitoring node based on the collected configuration information of the monitoring nodes;
calculating the memory performance of each monitoring node based on the collected configuration information of the monitoring nodes;
and calculating the processing capacity of each monitoring node based on the calculated CPU performance, memory performance and proportional parameters.
In a preferred embodiment of the present invention, calculating the CPU performance of each monitoring node based on the collected configuration information of the monitoring nodes comprises:
according to the formula: and calculating the CPU performance of each monitoring node by multiplying the CPU performance by the main frequency by the total number of cores by the CPU cache size.
In a preferred embodiment of the present invention, calculating the memory performance of each monitoring node based on the collected configuration information of the monitoring nodes includes:
according to the formula: and calculating the CPU performance of each monitoring node according to the memory performance which is the memory size/1G.
In a preferred embodiment of the present invention, calculating the processing capacity of each monitoring node based on the calculated CPU performance and memory performance and the proportional parameter includes:
according to the formula: and calculating the processing capacity of each monitoring node by the CPU performance multiplied by the first proportional parameter and the memory performance multiplied by the second proportional parameter.
In a preferred embodiment of the present invention, allocating a plurality of common nodes to each monitoring node according to the processing capability of each monitoring node, wherein the monitoring and collecting data of the corresponding common nodes by each monitoring node includes:
allocating a common node for the monitoring node with the maximum processing capacity, and reducing the processing capacity of the monitoring node with the maximum processing capacity by a preset value;
and repeating the steps until all the common nodes are distributed. The calculated processing capacity number can be converted into a weight value, that is, the stronger the processing capacity of the monitoring node is, the larger the weight value is, the master monitoring node selects the node with the largest weight value (the node name is the smallest if the ownership weight values are the same) according to the converted weight values of the monitoring nodes, then allocates a common node to the node, after the allocation is completed, the weight value of the monitoring node is reduced by a certain value, then selects the node with the largest weight value from all the monitoring nodes, allocates the common node to the node, and so on until all the nodes are allocated. After a new node is added into the cluster, the monitoring node for monitoring the newly added node can be distributed by the method, and the increase of the cluster scale can be flexibly responded. In addition, when the performance of the monitoring node is reduced due to the reasons of CPU fault, memory bank damage or network bandwidth reduction, the processing capacity of the monitoring node can be adjusted, so that the nodes beyond the processing capacity range of the monitoring node can be distributed to other nodes.
By the technical scheme, the alarm detection and performance data acquisition work of the nodes in the cluster can be unaffected, the pressure of the main monitoring node occupied by resources such as a CPU (central processing unit) and network bandwidth caused by information interaction of the main monitoring node on other nodes is reduced, the utilization rate of other monitoring node resources is improved, the pressure of the main monitoring node can be distributed, and a pressure balance can be maintained among the monitoring nodes.
It should be noted that, as will be understood by those skilled in the art, all or part of the processes in the methods of the above embodiments may be implemented by instructing relevant hardware through a computer program, and the above programs may be stored in a computer-readable storage medium, and when executed, the programs may include the processes of the embodiments of the methods as described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
Furthermore, the method disclosed according to an embodiment of the present invention may also be implemented as a computer program executed by a CPU, and the computer program may be stored in a computer-readable storage medium. The computer program, when executed by the CPU, performs the above-described functions defined in the method disclosed in the embodiments of the present invention.
In view of the above object, according to a second aspect of the embodiments of the present invention, there is provided an apparatus for node performance data acquisition, as shown in fig. 2, the apparatus 200 includes:
the collection module 201, the collection module 201 is configured to configure the master monitoring node in the distributed storage cluster to collect configuration information of all monitoring nodes;
a calculation module 202, wherein the calculation module 202 is configured to calculate the processing capacity of each monitoring node based on the collected configuration information;
the distribution module 203 is configured to distribute a plurality of common nodes to each monitoring node according to the processing capacity of each monitoring node, and each monitoring node monitors and collects data of the corresponding common node;
and the summarizing module 204, wherein the summarizing module 204 is configured to enable each monitoring node to send the collected data to the main monitoring node.
In view of the above object, a third aspect of the embodiments of the present invention provides a computer device. Fig. 3 is a schematic diagram of an embodiment of a computer device provided by the present invention. As shown in fig. 3, an embodiment of the present invention includes the following means: at least one processor S21; and a memory S22, the memory S22 storing computer instructions S23 executable on the processor, the instructions when executed by the processor implementing the method of:
the method comprises the steps that a main monitoring node in a distributed storage cluster collects configuration information of all monitoring nodes;
calculating the processing capacity of each monitoring node based on the collected configuration information;
distributing a plurality of common nodes for each monitoring node according to the processing capacity of each monitoring node, and monitoring and acquiring data of the corresponding common nodes by each monitoring node;
and each monitoring node sends the collected data to the main monitoring node.
In a preferred embodiment of the present invention, the collecting, by the master monitoring node in the distributed storage cluster, the configuration information of all the monitoring nodes includes:
the main monitoring node respectively collects the number of CPUs, the number of cores of a single CPU, CPU main frequency, the size of a memory and the size of a CPU cache of each monitoring node.
In a preferred embodiment of the present invention, calculating the processing power of each monitoring node based on the collected configuration information comprises:
setting a first proportional parameter and a second proportional parameter of the influence of a CPU and a memory on the performance of a monitoring node;
calculating the CPU performance of each monitoring node based on the collected configuration information of the monitoring nodes;
calculating the memory performance of each monitoring node based on the collected configuration information of the monitoring nodes;
and calculating the processing capacity of each monitoring node based on the calculated CPU performance, memory performance and proportional parameters.
In a preferred embodiment of the present invention, calculating the CPU performance of each monitoring node based on the collected configuration information of the monitoring nodes comprises:
according to the formula: and calculating the CPU performance of each monitoring node by multiplying the CPU performance by the main frequency by the total number of cores by the CPU cache size.
In a preferred embodiment of the present invention, calculating the memory performance of each monitoring node based on the collected configuration information of the monitoring nodes includes:
according to the formula: and calculating the CPU performance of each monitoring node according to the memory performance which is the memory size/1G.
In a preferred embodiment of the present invention, calculating the processing capacity of each monitoring node based on the calculated CPU performance and memory performance and the proportional parameter includes:
according to the formula: and calculating the processing capacity of each monitoring node by the CPU performance multiplied by the first proportional parameter and the memory performance multiplied by the second proportional parameter.
In a preferred embodiment of the present invention, allocating a plurality of common nodes to each monitoring node according to the processing capability of each monitoring node, wherein the monitoring and collecting data of the corresponding common nodes by each monitoring node includes:
allocating a common node for the monitoring node with the maximum processing capacity, and reducing the processing capacity of the monitoring node with the maximum processing capacity by a preset value;
and repeating the steps until all the common nodes are distributed.
In view of the above object, a fourth aspect of the embodiments of the present invention proposes a computer-readable storage medium. FIG. 4 is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. As shown in fig. 4, the computer readable storage medium S31 stores a computer program S32 that when executed by a processor performs the method of:
the method comprises the steps that a main monitoring node in a distributed storage cluster collects configuration information of all monitoring nodes;
calculating the processing capacity of each monitoring node based on the collected configuration information;
distributing a plurality of common nodes for each monitoring node according to the processing capacity of each monitoring node, and monitoring and acquiring data of the corresponding common nodes by each monitoring node;
and each monitoring node sends the collected data to the main monitoring node.
In a preferred embodiment of the present invention, the collecting, by the master monitoring node in the distributed storage cluster, the configuration information of all the monitoring nodes includes:
the main monitoring node respectively collects the number of CPUs, the number of cores of a single CPU, CPU main frequency, the size of a memory and the size of a CPU cache of each monitoring node.
In a preferred embodiment of the present invention, calculating the processing power of each monitoring node based on the collected configuration information comprises:
setting a first proportional parameter and a second proportional parameter of the influence of a CPU and a memory on the performance of a monitoring node;
calculating the CPU performance of each monitoring node based on the collected configuration information of the monitoring nodes;
calculating the memory performance of each monitoring node based on the collected configuration information of the monitoring nodes;
and calculating the processing capacity of each monitoring node based on the calculated CPU performance, memory performance and proportional parameters.
In a preferred embodiment of the present invention, calculating the CPU performance of each monitoring node based on the collected configuration information of the monitoring nodes comprises:
according to the formula: and calculating the CPU performance of each monitoring node by multiplying the CPU performance by the main frequency by the total number of cores by the CPU cache size.
In a preferred embodiment of the present invention, calculating the memory performance of each monitoring node based on the collected configuration information of the monitoring nodes includes:
according to the formula: and calculating the CPU performance of each monitoring node according to the memory performance which is the memory size/1G.
In a preferred embodiment of the present invention, calculating the processing capacity of each monitoring node based on the calculated CPU performance and memory performance and the proportional parameter includes:
according to the formula: and calculating the processing capacity of each monitoring node by the CPU performance multiplied by the first proportional parameter and the memory performance multiplied by the second proportional parameter.
In a preferred embodiment of the present invention, allocating a plurality of common nodes to each monitoring node according to the processing capability of each monitoring node, wherein the monitoring and collecting data of the corresponding common nodes by each monitoring node includes:
allocating a common node for the monitoring node with the maximum processing capacity, and reducing the processing capacity of the monitoring node with the maximum processing capacity by a preset value;
and repeating the steps until all the common nodes are distributed.
Furthermore, the methods disclosed according to embodiments of the present invention may also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. Which when executed by a processor performs the above-described functions defined in the methods disclosed in embodiments of the invention.
Further, the above method steps and system elements may also be implemented using a controller and a computer readable storage medium for storing a computer program for causing the controller to implement the functions of the above steps or elements.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
In one or more exemplary designs, the functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (10)
1. A method for collecting node performance data is characterized by comprising the following steps:
the method comprises the steps that a main monitoring node in a distributed storage cluster collects configuration information of all monitoring nodes;
calculating the processing capacity of each monitoring node based on the collected configuration information;
distributing a plurality of common nodes for each monitoring node according to the processing capacity of each monitoring node, and monitoring and acquiring data of the corresponding common nodes by each monitoring node;
and each monitoring node sends the collected data to the main monitoring node.
2. The method of claim 1, wherein collecting configuration information for all monitoring nodes by a master monitoring node in a distributed storage cluster comprises:
the main monitoring node respectively collects the number of CPUs, the number of cores of a single CPU, CPU main frequency, the size of a memory and the size of a CPU cache of each monitoring node.
3. The method of claim 2, wherein calculating the processing power of each monitoring node based on the collected configuration information comprises:
setting a first proportional parameter and a second proportional parameter of the influence of a CPU and a memory on the performance of a monitoring node;
calculating the CPU performance of each monitoring node based on the collected configuration information of the monitoring nodes;
calculating the memory performance of each monitoring node based on the collected configuration information of the monitoring nodes;
and calculating the processing capacity of each monitoring node based on the calculated CPU performance and memory performance and the proportional parameter.
4. The method of claim 3, wherein calculating the CPU performance of each monitoring node based on the collected configuration information of the monitoring nodes comprises:
according to the formula: and calculating the CPU performance of each monitoring node, wherein the CPU performance is the main frequency multiplied by the total number of cores multiplied by the CPU cache size.
5. The method of claim 3, wherein calculating the memory performance of each monitoring node based on the collected configuration information of the monitoring nodes comprises:
according to the formula: and calculating the CPU performance of each monitoring node according to the memory performance which is the memory size/1G.
6. The method of claim 3, wherein calculating the processing power of each monitoring node based on the calculated CPU and memory performance and the scaling parameters comprises:
according to the formula: and calculating the processing capacity of each monitoring node, wherein the processing capacity of each monitoring node is CPU performance multiplied by the first proportional parameter + memory performance multiplied by the second proportional parameter.
7. The method of claim 1, wherein a plurality of common nodes are allocated to each monitoring node according to the processing capacity of each monitoring node, and wherein the monitoring and collecting of data of the corresponding common nodes by each monitoring node comprises:
allocating a common node for the monitoring node with the maximum processing capacity, and reducing the processing capacity of the monitoring node with the maximum processing capacity by a preset value;
and repeating the steps until all the common nodes are distributed.
8. An apparatus for node performance data acquisition, the apparatus comprising:
the collection module is configured to collect configuration information of all monitoring nodes by a main monitoring node in the distributed storage cluster;
a computing module configured to compute a processing power of each monitoring node based on the collected configuration information;
the distribution module is configured to distribute a plurality of common nodes to each monitoring node according to the processing capacity of each monitoring node, and each monitoring node monitors and acquires data of the corresponding common node;
a summary module configured to cause each monitoring node to send the collected data to the master monitoring node.
9. A computer device, comprising:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111035188.XA CN113806082A (en) | 2021-09-05 | 2021-09-05 | Method, device and equipment for collecting node performance data and readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111035188.XA CN113806082A (en) | 2021-09-05 | 2021-09-05 | Method, device and equipment for collecting node performance data and readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113806082A true CN113806082A (en) | 2021-12-17 |
Family
ID=78894705
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111035188.XA Pending CN113806082A (en) | 2021-09-05 | 2021-09-05 | Method, device and equipment for collecting node performance data and readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113806082A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8135859B1 (en) * | 2005-01-19 | 2012-03-13 | Microsoft Corporation | System and method for providing infrastructure services without a designated network manager |
| CN103905530A (en) * | 2014-03-11 | 2014-07-02 | 浪潮集团山东通用软件有限公司 | High-performance global load balance distributed database data routing method |
| CN108874623A (en) * | 2018-05-31 | 2018-11-23 | 郑州云海信息技术有限公司 | Distributed type assemblies method for monitoring performance, device, equipment, system and storage medium |
| CN109343965A (en) * | 2018-10-31 | 2019-02-15 | 北京金山云网络技术有限公司 | Resource adjusting method, device, cloud platform and server |
| CN110046048A (en) * | 2019-04-18 | 2019-07-23 | 杭州电子科技大学 | A kind of load-balancing method adaptively quickly reassigned based on workload |
| US20190312801A1 (en) * | 2018-04-10 | 2019-10-10 | Vmware, Inc. | Optimized performance data collection at client nodes |
| CN112764993A (en) * | 2021-01-22 | 2021-05-07 | 苏州浪潮智能科技有限公司 | Node information collection method, device, equipment and readable storage medium |
| CN112860393A (en) * | 2021-01-20 | 2021-05-28 | 北京科技大学 | Distributed task scheduling method and system |
-
2021
- 2021-09-05 CN CN202111035188.XA patent/CN113806082A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8135859B1 (en) * | 2005-01-19 | 2012-03-13 | Microsoft Corporation | System and method for providing infrastructure services without a designated network manager |
| CN103905530A (en) * | 2014-03-11 | 2014-07-02 | 浪潮集团山东通用软件有限公司 | High-performance global load balance distributed database data routing method |
| US20190312801A1 (en) * | 2018-04-10 | 2019-10-10 | Vmware, Inc. | Optimized performance data collection at client nodes |
| CN108874623A (en) * | 2018-05-31 | 2018-11-23 | 郑州云海信息技术有限公司 | Distributed type assemblies method for monitoring performance, device, equipment, system and storage medium |
| CN109343965A (en) * | 2018-10-31 | 2019-02-15 | 北京金山云网络技术有限公司 | Resource adjusting method, device, cloud platform and server |
| CN110046048A (en) * | 2019-04-18 | 2019-07-23 | 杭州电子科技大学 | A kind of load-balancing method adaptively quickly reassigned based on workload |
| CN112860393A (en) * | 2021-01-20 | 2021-05-28 | 北京科技大学 | Distributed task scheduling method and system |
| CN112764993A (en) * | 2021-01-22 | 2021-05-07 | 苏州浪潮智能科技有限公司 | Node information collection method, device, equipment and readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 熊齐;唐佳明;: "Linux集群实时监控系统的一种实现方法", 计算机系统应用, no. 09, 15 September 2013 (2013-09-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7890620B2 (en) | Monitoring system and monitoring method | |
| CN102577241B (en) | Method, device and system for scheduling distributed buffer resources | |
| US8892728B2 (en) | Automatic zone-based management of a data center | |
| CN104243405A (en) | Request processing method, device and system | |
| WO2012072344A1 (en) | Endpoint-to-endpoint communications status monitoring | |
| CN108804383B (en) | Support point parallel enumeration method and device based on measurement space | |
| CN110244901B (en) | Task allocation method and device and distributed storage system | |
| CN102426544A (en) | Task allocating method and system | |
| CN112261120B (en) | Cloud-side cooperative task unloading method and device for power distribution internet of things | |
| CN112764993A (en) | Node information collection method, device, equipment and readable storage medium | |
| CN114244718A (en) | Power transmission line communication network equipment management system | |
| CN115509875A (en) | Server health degree evaluation method and device | |
| CN103516734A (en) | Data processing method, device and system | |
| CN110519354A (en) | A kind of distributed objects storage system and its method for processing business and storage medium | |
| CN106021026B (en) | Backup method and device | |
| CN113806082A (en) | Method, device and equipment for collecting node performance data and readable medium | |
| CN107547622B (en) | Resource adjusting method and device | |
| CN114936133A (en) | Method, device, equipment and readable medium for improving system availability | |
| CN112416888B (en) | Dynamic load balancing method and system for distributed file system | |
| CN111651316B (en) | Resource monitoring method, system, electronic equipment and storage medium | |
| CN112468317A (en) | Cluster topology updating method, system, equipment and computer storage medium | |
| LU505329B1 (en) | Management system for transmission line communication network equipment | |
| CN116055496B (en) | Monitoring data acquisition method and device, electronic equipment and storage medium | |
| CN113886172B (en) | Method, device, equipment and readable medium for managing micro-services in cluster | |
| CN109388502A (en) | A kind of service identification distribution method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |