CN117851075B - Resource optimization management method of data monitoring system - Google Patents
Resource optimization management method of data monitoring system Download PDFInfo
- Publication number
- CN117851075B CN117851075B CN202410263488.0A CN202410263488A CN117851075B CN 117851075 B CN117851075 B CN 117851075B CN 202410263488 A CN202410263488 A CN 202410263488A CN 117851075 B CN117851075 B CN 117851075B
- Authority
- CN
- China
- Prior art keywords
- value
- matching
- resource
- computing
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007726 management method Methods 0.000 title claims abstract description 17
- 238000012544 monitoring process Methods 0.000 title claims abstract description 15
- 238000005457 optimization Methods 0.000 title claims abstract description 11
- 238000000034 method Methods 0.000 claims abstract description 17
- 230000005540 biological transmission Effects 0.000 claims description 44
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000012163 sequencing technique Methods 0.000 abstract description 4
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a resource optimization management method of a data monitoring system, which particularly relates to the technical field of data processing and comprises the following steps: respectively collecting resource information of a plurality of computing nodes of the system to obtain a computing node information set, analyzing the computing node information set to obtain a resource value of each computing node, and carrying out resource sequencing to obtain a resource sequencing list; collecting and processing requirements of data processing tasks of a main node of the system, obtaining a required value, performing target matching with a computing node in a resource ordering list according to a preset matching rule, obtaining a primary matching result, verifying, re-matching when the primary matching result does not meet a preset standard, and finally analyzing the primary matching result and obtaining a corresponding matching quality grade; the invention can better schedule each computing node in the computing power network, and realizes the efficient management of the data processing process and improves the data processing efficiency by dynamically scheduling, matching and optimally configuring the computing power resources of the system.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a resource optimization management method of a data monitoring system.
Background
With the advent of the big data age, data monitoring systems have been widely used in various fields. However, due to huge data volume, the conventional resource management method cannot meet the real-time, efficient and accurate data processing requirements, and due to the fact that the monitoring data can generate operation tasks with different difficulty degrees according to different data volumes needing to be analyzed, how to allocate operation nodes in a targeted manner according to the generated operation tasks, so that the optimal management of resources of the whole operation network is realized, and the data processing efficiency is improved, so that the problem to be solved is urgent.
Disclosure of Invention
In order to achieve the above purpose, the present invention provides the following technical solutions:
A resource optimization management method of a data monitoring system comprises the following steps:
step one, respectively collecting resource information of a plurality of computing nodes of a system to obtain a computing node information set;
Analyzing according to the information set of the computing nodes to obtain a resource value of each computing node, and sorting the resources of the computing nodes according to the size of the resource value to obtain a resource sorting list;
step three, collecting and processing the requirements of the data processing tasks of the main nodes of the system to obtain a requirement value;
Performing target matching on the required value of the data processing task and the computing nodes in the resource ordering list according to a preset matching rule to obtain a primary matching result;
Step five, obtaining the last matching quality grade of a target computing node in the first-level matching result, and obtaining the data transmission fluctuation coefficient value of a preset time window of the target computing node in the first-level matching result, when the last matching quality grade and the data transmission fluctuation coefficient value meet preset standards, performing data processing task allocation according to the first-level matching result, otherwise, removing the computing node in the first-level matching result from a resource ordering list and re-performing target matching until the last matching quality grade and the data transmission fluctuation coefficient value meet the preset standards;
And step six, collecting the time spent for completing the primary matching result and carrying out analysis operation to obtain a matching quality value of the primary matching result, and then comparing the matching quality value with a preset matching quality threshold value to obtain a corresponding matching quality grade.
In a preferred embodiment, in the first step, resource information of a plurality of computing nodes of the system is collected respectively, and obtaining a computing node information set refers to:
And collecting the computing resource information, the storage resource information and the network resource information of a plurality of computing nodes of the system respectively, and then summarizing the computing resource information, the storage resource information and the network resource information of all the computing nodes to obtain a computing node information set.
In a preferred embodiment, in the second step, the analysis is performed according to the information set of the computing nodes, and obtaining the resource value of each computing node refers to:
Step S1, marking computing resource information, storage resource information and network resource information in a node information set respectively;
Step S11, marking computing resource data in the computing resource information as JSi;
Step S12, marking the total storage resource data in the storage resource information as ZCi, and marking the existing storage resource data as XCi;
step S13, marking a network fluctuation value of a preset time window in the network resource information as BDi, and transmitting data at a speed SDi;
Step S2, calculating to obtain the resource value of the calculation node through a formula, ，/>P is a preset network fluctuation standard value, q is the total item number of the network fluctuation value BDi, ZYi is the resource value of the calculation node, and f1, f2, f3 and f4 are all preset specific proportionality coefficients and are not 0.
In a preferred embodiment, in the third step, the requirements of the data processing task of the master node of the system are collected and processed, and the obtaining the requirement value refers to:
step W1, collecting data volume and processing time of a data processing task of a main node of a system;
step W2, marking the data quantity of a data processing task of a main node of the system as Si, and marking the processing time as Ti;
step W3, obtaining a required value through formula calculation, A1 represents a preset standard data amount, A2 represents a preset standard processing time, b1 and b2 are both preset specific proportionality coefficients and are not 0, and XQi is a required value.
In a preferred embodiment, in the fourth step, performing target matching on the required value of the data processing task and the computing nodes in the resource ordering list according to a preset matching rule, and obtaining a first-level matching result refers to: multiplying the required value XQi by a preset matching coefficient value to obtain a matching interval [ J1, J2], wherein the matching coefficient value is larger than 1 and smaller than 2, J2 is the result obtained by multiplying the required value XQi by the preset matching coefficient value, the value of J1 is equal to the value of the required value XQi, the computing nodes in the matching interval [ J1, J2] in the resource ordering list are obtained, then the computing nodes are arranged in descending order according to the value of the resource, the computing nodes arranged at the head are used as the matching target of the data processing task, and the two are matched to obtain a primary matching result.
In a preferred embodiment, in step five, the data transmission fluctuation coefficient value is obtained by:
Obtaining data transmission speed SDi under a preset time window in the network resource information, marking the maximum value of the data transmission speed as SDmax, marking the minimum value of the data transmission speed as SDmin, calculating to obtain the average data transmission speed PJ under the preset time window, M is the total item number of the data transmission speed SDi under a preset time window, and then the data transmission fluctuation coefficient is calculated,/>BXi is a data transmission fluctuation coefficient.
In a preferred embodiment, in the fifth step, the matching quality level and the data transmission fluctuation coefficient value of the last time all meet the preset standard means that: the matching quality grade is one level, and the data transmission fluctuation coefficient value is smaller than a preset standard data transmission fluctuation coefficient threshold value.
In a preferred embodiment, in step six, collecting the time spent for completing the first-stage matching result and performing an analysis operation to obtain a matching quality value of the first-stage matching result, and then comparing the matching quality value with a preset matching quality threshold value to obtain a corresponding matching quality level, which means that:
Marking the completion time of the primary matching result as YS, analyzing the processing time Ti of the data processing task of the master node of the system and YS when the primary matching result is completed to obtain the matching quality value of the primary matching result, ZL is the matching quality value of the first-stage matching result, then the matching quality value ZL is compared with a preset matching quality threshold, and if the matching quality value ZL is smaller than or equal to the preset matching quality threshold, the obtained matching quality grade is first-stage; if the matching quality value ZL is larger than a preset matching quality threshold value, the obtained matching quality grade is two-level.
The invention has the technical effects and advantages that:
The invention can acquire the information sets of the computing nodes by collecting the resource information of a plurality of computing nodes of the system, and can comprehensively know the resource condition of the system; analyzing according to the information set of the computing nodes to obtain a resource value of each computing node, and sequencing the resources of the computing nodes according to the size of the resource value to obtain a resource sequencing list, so that reasonable allocation of system resources is facilitated; the requirements of the data processing tasks of the main nodes of the system are collected and processed to obtain the required values, and the basis can be provided for subsequent task allocation; and performing target matching on the required value of the data processing task and the computing nodes in the resource ordering list according to a preset matching rule to obtain a first-level matching result, so that optimal matching of the task and the resource can be realized.
The invention can collect the time spent of the first-stage matching result and carry out analysis operation to obtain the matching quality value of the first-stage matching result, then compare the matching quality value with the preset matching quality threshold value to obtain the corresponding matching quality grade, and can evaluate the quality of the matching result; the running state of the system is monitored and analyzed to monitor the service condition of the resource, so that the system performance bottleneck problem can be found and solved in time; the invention can better schedule each computing node in the computing power network, realizes the high-efficiency management of the data processing process by dynamically scheduling, matching and optimally configuring the computing power resources of the system, improves the data processing efficiency, adopts the task allocation and load balancing strategy, ensures that the computing node allocated with the task reaches a 'balanced state' with the difficulty of task processing to a certain extent, ensures the stable operation of each computing node, and avoids the problem of the performance reduction of the whole system caused by overload of a certain node.
Drawings
For the convenience of those skilled in the art, the present invention will be further described with reference to the accompanying drawings;
fig. 1 is a schematic diagram of a resource optimization management method of a data monitoring system in the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1: a resource optimization management method of a data monitoring system comprises the following steps:
Step one, respectively collecting resource information of a plurality of computing nodes of a system to obtain a computing node information set; such information includes computing resources such as CPU, storage resources such as memory and network resources for each computing node, specifically operating as: the method comprises the steps of respectively collecting computing resource information, storage resource information and network resource information of a plurality of computing nodes of the system, then summarizing the computing resource information, the storage resource information and the network resource information of all the computing nodes to obtain a computing node information set, wherein one computing network consists of a main node and a plurality of computing nodes, the main node is mainly responsible for management and scheduling work of the whole computing network, such as task allocation, and the computing nodes are devices for executing specific computing tasks.
Analyzing according to the information set of the computing nodes to obtain a resource value of each computing node, and sorting the resources of the computing nodes according to the size of the resource value to obtain a resource sorting list; the method specifically comprises the following steps:
Step S1, marking computing resource information, storage resource information and network resource information in a node information set respectively;
Step S11, marking computing resource data in the computing resource information as JSi; i represents only the item number; i=1, 2, 3,4, … …, k is a positive integer;
Step S12, marking the total storage resource data in the storage resource information as ZCi, and marking the existing storage resource data as XCi;
step S13, marking a network fluctuation value of a preset time window in the network resource information as BDi, and transmitting data at a speed SDi;
Step S2, calculating to obtain the resource value of the calculation node through a formula, ，/>P is a preset network fluctuation standard value, q is the total item number of the network fluctuation value BDi, ZYi is the resource value of a calculation node, and f1, f2, f3 and f4 are all preset specific proportion coefficients and are not 0; f1, f2 and f3 respectively represent the influence degree of the computing resource data, the proportion of the residual storage resource occupying the total storage resource data and the network fluctuation value on the resource value of the computing node, and the residual storage resource is obtained by subtracting the existing storage resource data from the total storage resource data;
Step S3, sorting the resources of the plurality of computing nodes according to the size of the resource value ZYi, wherein the resource sorting list is obtained in a descending order mode in the embodiment, the arrangement order of the plurality of computing nodes indicates the computing capacity of the plurality of computing nodes in all the computing nodes, and the higher the sequence is, the stronger the computing capacity is and the more complex the task can be processed; the more posterior the sequence, the weaker its computational power, and the simpler the task that can be handled.
Step three, collecting and processing the requirements of the data processing tasks of the main nodes of the system to obtain a requirement value; the method specifically comprises the following steps:
step W1, collecting data volume and processing time of a data processing task of a main node of a system;
step W2, marking the data quantity of a data processing task of a main node of the system as Si, and marking the processing time as Ti;
step W3, obtaining a required value through formula calculation, A1 represents a preset standard data amount, A2 represents a preset standard processing time, b1 and b2 are both preset specific proportionality coefficients and are not 0, XQi is a required value, the size of the required value XQi represents the complexity of the data processing task, the complexity of the processing task is related to the size of the data amount and the processing time, and b1 and b2 represent the influence degree of the data amount and the processing time of the data processing task on the complexity of the data processing task respectively.
Performing target matching on the required value of the data processing task and the computing nodes in the resource ordering list according to a preset matching rule to obtain a primary matching result; the method comprises the following steps: the method comprises the steps of multiplying a required value XQi by a preset matching coefficient value to obtain a matching interval [ J1, J2], wherein the matching coefficient value is larger than 1 and smaller than 2, J2 is the result obtained by multiplying the required value XQi by the preset matching coefficient value, the value of J1 is equal to the value of the required value XQi, the meaning of multiplying the required value XQi by the preset matching coefficient value is to optimize the configuration of resource operation, better scheduling is carried out on each computing node in a computing power network, the system computing power resource is matched and optimized through dynamic scheduling, the efficient management of the data processing process is realized, the data processing efficiency is improved, a task allocation and load balancing strategy is adopted, the load balancing strategy refers to the computing node which ensures that the computing node allocated tasks meet the requirement of the tasks as much as possible, the computing node with large computing capability is not easy to appear in a simple task matching, the difficulty of task processing reaches a balanced state, the stable operation of each computing node is ensured, the problem that the performance of the whole system is reduced due to the fact that a certain node is overloaded is avoided, the computing node is well ordered, the data is arranged in the first order of the computing node J1, and then the computing node is matched with the first level of the computing node according to the task matching result, and the matching result is arranged at the first order.
Step five, collecting the time spent for completing the primary matching result and carrying out analysis operation to obtain a matching quality value of the primary matching result, and then comparing the matching quality value with a preset matching quality threshold value to obtain a corresponding matching quality grade, wherein the specific steps are as follows: marking the completion time of the primary matching result as YS, analyzing the processing time Ti of the data processing task of the master node of the system and YS when the primary matching result is completed to obtain the matching quality value of the primary matching result,ZL is the matching quality value of the first-stage matching result, then the matching quality value ZL is compared with a preset matching quality threshold, and if the matching quality value ZL is smaller than or equal to the preset matching quality threshold, the obtained matching quality grade is first-stage; if the matching quality value ZL is larger than a preset matching quality threshold value, the obtained matching quality grade is two-grade, when the obtained matching quality grade is one-grade, the difficulty of the calculation node of the allocated task and the task processing reaches an expected balance state, the deviation of the processing time Ti of the data processing task of the YS and the main node of the system accords with the expected balance state when the actual processing is completed, the processing time Ti is expected, and the data processing task is not required to be completed within the processing time Ti, so that the optimal configuration of the system computing power resource is realized; when the obtained matching quality level is two-level, the difficulty level of the distributed task computing node and task processing does not reach the expected balance state, the optimal configuration of the system computing power resource is not realized, the computing power of the computing node is reduced, and the computing node is required to be checked by personnel.
Example 2: a resource optimization management method of a data monitoring system comprises the following steps:
step one, respectively collecting resource information of a plurality of computing nodes of a system to obtain a computing node information set; and collecting the computing resource information, the storage resource information and the network resource information of a plurality of computing nodes of the system respectively, and then summarizing the computing resource information, the storage resource information and the network resource information of all the computing nodes to obtain a computing node information set.
Analyzing according to the information set of the computing nodes to obtain a resource value of each computing node, and sorting the resources of the computing nodes according to the size of the resource value to obtain a resource sorting list; the method comprises the following steps: marking computing resource information, storage resource information and network resource information in the node information set respectively, marking computing resource data in the computing resource information as JSi, marking total storage resource data in the storage resource information as ZCi, marking existing storage resource data as XCi, marking network fluctuation value of a preset time window in the network resource information as BDi, obtaining a resource value of a computing node through formula calculation,，/>P is a preset network fluctuation standard value, q is the total item number of the network fluctuation value BDi, ZYi is the resource value of the calculation node, and f1, f2, f3 and f4 are all preset specific proportionality coefficients and are not 0.
Step three, collecting and processing the requirements of the data processing tasks of the main nodes of the system to obtain a requirement value; firstly collecting the data volume and processing time of the data processing task of the main node of the system, then marking the data volume of the data processing task of the main node of the system as Si, marking the processing time as Ti, then obtaining a required value through formula calculation,A1 represents a preset standard data amount, A2 represents a preset standard processing time, b1 and b2 are both preset specific proportionality coefficients and are not 0, and XQi is a required value.
Performing target matching on the required value of the data processing task and the computing nodes in the resource ordering list according to a preset matching rule to obtain a primary matching result; multiplying the required value XQi by a preset matching coefficient value to obtain a matching interval [ J1, J2], wherein the matching coefficient value is larger than 1 and smaller than 2, J2 is the result obtained by multiplying the required value XQi by the preset matching coefficient value, the value of J1 is equal to the value of the required value XQi, the computing nodes in the matching interval [ J1, J2] in the resource ordering list are obtained, then the computing nodes are arranged in descending order according to the value of the resource, the computing nodes arranged at the head are used as the matching target of the data processing task, and the two are matched to obtain a primary matching result.
Step five, obtaining the last matching quality grade of a target computing node in the first-level matching result, and obtaining the data transmission fluctuation coefficient value of a preset time window of the target computing node in the first-level matching result, when the last matching quality grade and the data transmission fluctuation coefficient value meet preset standards, performing data processing task allocation according to the first-level matching result, otherwise, removing the computing node in the first-level matching result from a resource ordering list and re-performing target matching until the last matching quality grade and the data transmission fluctuation coefficient value meet the preset standards; wherein the data transmission fluctuation coefficient value is obtained by: obtaining data transmission speed SDi under a preset time window in the network resource information, marking the maximum value of the data transmission speed as SDmax, marking the minimum value of the data transmission speed as SDmin, calculating to obtain the average data transmission speed PJ under the preset time window,M is the total item number of the data transmission speed SDi under a preset time window, and then the data transmission fluctuation coefficient is calculated,/>BXi is a data transmission fluctuation coefficient.
Wherein, the matching quality grade and the data transmission fluctuation coefficient value of the last time all accord with the preset standard, which means that: the matching quality level is one level, the data transmission fluctuation coefficient value is smaller than the preset standard data transmission fluctuation coefficient threshold value, and it should be mentioned that if the target computing node in the first level matching result is matched for the first time, only the data transmission fluctuation coefficient value is ensured to meet the preset standard, and the last matching quality level is not existed because the data transmission fluctuation coefficient value is not matched, and in this case, the last matching quality level is defaulted to meet the preset standard.
Step six, collecting the time spent for completing the primary matching result and carrying out analysis operation to obtain a matching quality value of the primary matching result, and then comparing the matching quality value with a preset matching quality threshold value to obtain a corresponding matching quality grade; marking the completion time of the primary matching result as YS, analyzing the processing time Ti of the data processing task of the master node of the system and YS when the primary matching result is completed to obtain the matching quality value of the primary matching result,ZL is the matching quality value of the first-stage matching result, then the matching quality value ZL is compared with a preset matching quality threshold, and if the matching quality value ZL is smaller than or equal to the preset matching quality threshold, the obtained matching quality grade is first-stage; if the matching quality value ZL is larger than a preset matching quality threshold value, the obtained matching quality grade is two-level.
The invention monitors and analyzes the service condition of the resource by monitoring the running state of the system, and is helpful for finding and solving the bottleneck problem of the system performance in time.
The above formulas are all formulas with dimensions removed and numerical values calculated, the formulas are formulas with a large amount of data collected for software simulation to obtain the latest real situation, and preset parameters in the formulas are set by those skilled in the art according to the actual situation.
It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (4)
1. The resource optimization management method of the data monitoring system is characterized by comprising the following steps of:
step one, respectively collecting resource information of a plurality of computing nodes of a system to obtain a computing node information set;
Analyzing according to the information set of the computing nodes to obtain a resource value of each computing node, and sorting the resources of the computing nodes according to the size of the resource value to obtain a resource sorting list;
step three, collecting and processing the requirements of the data processing tasks of the main nodes of the system to obtain a requirement value;
Performing target matching on the required value of the data processing task and the computing nodes in the resource ordering list according to a preset matching rule to obtain a primary matching result; the logic is as follows: multiplying a required value XQi by a preset matching coefficient value to obtain matching intervals [ J1, J2], wherein the matching coefficient value is larger than 1 and smaller than 2, J2 is the result obtained by multiplying the required value XQi by the preset matching coefficient value, the value of J1 is equal to the value of the required value XQi, computing nodes in the matching intervals [ J1, J2] in a resource ordering list are obtained, then the computing nodes are arranged in descending order according to the value of the resource, the computing nodes arranged at the head are used as the matching target of the data processing task, and the computing nodes are matched to obtain a first-level matching result;
Step five, obtaining the last matching quality grade of a target computing node in the first-level matching result, and obtaining the data transmission fluctuation coefficient value of a preset time window of the target computing node in the first-level matching result, when the last matching quality grade and the data transmission fluctuation coefficient value meet preset standards, performing data processing task allocation according to the first-level matching result, otherwise, removing the computing node in the first-level matching result from a resource ordering list and re-performing target matching until the last matching quality grade and the data transmission fluctuation coefficient value meet the preset standards;
the data transmission fluctuation coefficient value is obtained by:
Obtaining data transmission speed SDi under a preset time window in the network resource information, marking the maximum value of the data transmission speed as SDmax, marking the minimum value of the data transmission speed as SDmin, calculating to obtain the average data transmission speed PJ under the preset time window, M is the total item number of the data transmission speed SDi under a preset time window, and then the data transmission fluctuation coefficient is calculated,/>BXi is a data transmission fluctuation coefficient;
The matching quality grade and the data transmission fluctuation coefficient value of the last time all accord with the preset standard, which means that: the matching quality grade is one level, and the data transmission fluctuation coefficient value is smaller than a preset standard data transmission fluctuation coefficient threshold value;
Step six, collecting the time spent for completing the primary matching result and carrying out analysis operation to obtain a matching quality value of the primary matching result, and then comparing the matching quality value with a preset matching quality threshold value to obtain a corresponding matching quality grade; the logic is as follows: marking the completion time of the primary matching result as YS, analyzing the processing time Ti of the data processing task of the master node of the system and YS when the primary matching result is completed to obtain the matching quality value of the primary matching result, ZL is the matching quality value of the first-stage matching result, then the matching quality value ZL is compared with a preset matching quality threshold, and if the matching quality value ZL is smaller than or equal to the preset matching quality threshold, the obtained matching quality grade is first-stage; if the matching quality value ZL is larger than a preset matching quality threshold value, the obtained matching quality grade is two-level.
2. The method for resource optimization management of a data monitoring system according to claim 1, wherein in the first step, resource information of a plurality of computing nodes of the system is collected respectively, and obtaining a computing node information set refers to:
And collecting the computing resource information, the storage resource information and the network resource information of a plurality of computing nodes of the system respectively, and then summarizing the computing resource information, the storage resource information and the network resource information of all the computing nodes to obtain a computing node information set.
3. The method for optimizing and managing resources of a data monitoring system according to claim 2, wherein in the second step, the analysis is performed according to the information set of the computing nodes, and the obtaining of the resource value of each computing node means:
Step S1, marking computing resource information, storage resource information and network resource information in a node information set respectively;
Step S11, marking computing resource data in the computing resource information as JSi;
Step S12, marking the total storage resource data in the storage resource information as ZCi, and marking the existing storage resource data as XCi;
step S13, marking a network fluctuation value of a preset time window in the network resource information as BDi, and transmitting data at a speed SDi;
Step S2, calculating to obtain the resource value of the calculation node through a formula, ，P is a preset network fluctuation standard value, q is the total item number of the network fluctuation value BDi, ZYi is the resource value of the calculation node, and f1, f2, f3 and f4 are all preset specific proportionality coefficients and are not 0.
4. A resource optimization management method of a data monitoring system according to claim 3, wherein in step three, the requirements of the data processing tasks of the master node of the system are collected and processed, and the obtained requirement value means:
step W1, collecting data volume and processing time of a data processing task of a main node of a system;
step W2, marking the data quantity of a data processing task of a main node of the system as Si, and marking the processing time as Ti;
step W3, obtaining a required value through formula calculation, A1 represents a preset standard data amount, A2 represents a preset standard processing time, b1 and b2 are both preset specific proportionality coefficients and are not 0, and XQi is a required value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410263488.0A CN117851075B (en) | 2024-03-08 | 2024-03-08 | Resource optimization management method of data monitoring system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410263488.0A CN117851075B (en) | 2024-03-08 | 2024-03-08 | Resource optimization management method of data monitoring system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117851075A CN117851075A (en) | 2024-04-09 |
| CN117851075B true CN117851075B (en) | 2024-05-14 |
Family
ID=90536722
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410263488.0A Active CN117851075B (en) | 2024-03-08 | 2024-03-08 | Resource optimization management method of data monitoring system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117851075B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018059029A1 (en) * | 2016-09-30 | 2018-04-05 | 华为技术有限公司 | Resource allocation method, related device and system |
| CN113204411A (en) * | 2021-06-08 | 2021-08-03 | 广东利元亨智能装备股份有限公司 | Data processing method, intermediate processing equipment and storage medium |
| WO2023125493A1 (en) * | 2021-12-27 | 2023-07-06 | 华为技术有限公司 | Resource management method and apparatus, and resource management platform |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102088735B (en) * | 2010-03-24 | 2014-12-03 | 电信科学技术研究院 | Method and equipment for balancing inter-sub-frame traffic load and processing inter-cell interference (ICI) |
| CN111901573A (en) * | 2020-08-17 | 2020-11-06 | 泽达易盛（天津）科技股份有限公司 | Fine granularity real-time supervision system based on edge calculation |
| US11983568B2 (en) * | 2021-04-23 | 2024-05-14 | Kabushiki Kaisha Toshiba | Allocation of heterogeneous computational resource |
-
2024
- 2024-03-08 CN CN202410263488.0A patent/CN117851075B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018059029A1 (en) * | 2016-09-30 | 2018-04-05 | 华为技术有限公司 | Resource allocation method, related device and system |
| CN113204411A (en) * | 2021-06-08 | 2021-08-03 | 广东利元亨智能装备股份有限公司 | Data processing method, intermediate processing equipment and storage medium |
| WO2023125493A1 (en) * | 2021-12-27 | 2023-07-06 | 华为技术有限公司 | Resource management method and apparatus, and resource management platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117851075A (en) | 2024-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107968802A (en) | The method, apparatus and filtering type scheduler of a kind of scheduling of resource | |
| CN110308980A (en) | Batch processing method, device, equipment and the storage medium of data | |
| CN112261120B (en) | Cloud-side cooperative task unloading method and device for power distribution internet of things | |
| CN116708451B (en) | Edge cloud cooperative scheduling method and system | |
| CN111176840A (en) | Distributed task allocation optimization method and device, storage medium and electronic device | |
| CN115237568A (en) | Mixed weight task scheduling method and system for edge heterogeneous equipment | |
| CN117271142B (en) | Load balancing method and task scheduler for analyzing probability security analysis model | |
| CN117851075B (en) | Resource optimization management method of data monitoring system | |
| Armony et al. | Asymptotically optimal appointment schedules | |
| CN116560811A (en) | Simulation system and method applied to scheduling system | |
| CN116521335A (en) | Distributed task scheduling method and system for inclined image model production | |
| CN115314400A (en) | Network system anomaly detection method and device, electronic equipment and storage medium | |
| CN115168014A (en) | Job scheduling method and device | |
| CN113419852B (en) | Method, device, equipment and storage medium for responding request of micro service | |
| CN110084507A (en) | The scientific workflow method for optimizing scheduling of perception is classified under cloud computing environment | |
| CN112749202A (en) | Information operation strategy determination method, device, equipment and storage medium | |
| CN110825493A (en) | Virtual machine tuning method and device | |
| CN117828389B (en) | Distributed system monitoring method based on intelligent operation and maintenance | |
| CN114330879B (en) | Order distribution method and system of multi-dimensional constraint | |
| CN114706667B (en) | Streaming media forwarding method based on heterogeneous computation | |
| CN112383949B (en) | Edge computing and communication resource allocation method and system | |
| CN117155947B (en) | High-reliability real-time sharing method and system for data resources | |
| CN117729204B (en) | K8S container scheduling method and system based on monitoring perception | |
| CN112905351B (en) | GPU and CPU load scheduling method, device, equipment and medium | |
| CN113450078A (en) | Processing system of communication big data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |