CN115061891A - System load capacity prediction method and device based on block chain - Google Patents

System load capacity prediction method and device based on block chain Download PDF

Info

Publication number
CN115061891A
CN115061891A CN202210736445.0A CN202210736445A CN115061891A CN 115061891 A CN115061891 A CN 115061891A CN 202210736445 A CN202210736445 A CN 202210736445A CN 115061891 A CN115061891 A CN 115061891A
Authority
CN
China
Prior art keywords
load capacity
data
block chain
model
target system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210736445.0A
Other languages
Chinese (zh)
Inventor
贝飞
崔东晓
赵星驰
刘征彦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202210736445.0A priority Critical patent/CN115061891A/en
Publication of CN115061891A publication Critical patent/CN115061891A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
    • G06F11/3433—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment for load management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3034—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a storage system, e.g. DASD based or network based
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
    • G06F11/324—Display of status information
    • G06F11/327—Alarm or error message display
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/80—Database-specific techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/875—Monitoring of systems including the internet

Abstract

The invention discloses a block chain-based system load capacity prediction method and device, and relates to the technical field of block chains and intelligent operation and maintenance, wherein the method comprises the following steps: acquiring load capacity data of a target system from a preset block chain network, wherein block chain nodes in the block chain network upload real-time load capacity data of the target system to the block chain network; generating load capacity time sequence data of the target system according to the load capacity data; and inputting the load capacity time sequence data into a preset system load capacity prediction model to obtain a load capacity prediction result of the target system. The invention effectively reduces the system fault risk caused by the conditions of rapid increase of the system capacity and the like.

Description

System load capacity prediction method and device based on block chain
Technical Field
The invention relates to the technical field of system operation and maintenance, in particular to a block chain-based system load capacity prediction method and device.
Background
With the continuous development of banking business, the construction demand of application systems of banks is continuously expanded, and the dependence on system resources and budget are also increased more and more. In the process of monitoring system resources, the alarm depending on real-time resources is easy to cause the situations of insufficient resources or resource waste and the like caused by inaccurate estimation of the service data growth rate and the system resource occupancy rate, and the best use of the resources cannot be achieved.
In terms of determining whether system resources are sufficient, event monitoring of application deployment is currently generally relied on. After the system is on line, the application usually deploys event monitoring influencing the service development according to the actual needs of the service function, so as to improve the operation and maintenance capability of the system. The method can not pre-judge the system resource, can monitor and alarm after the event occurs, and can not avoid the business influence caused by the occurrence of the event. In addition, the alarm of the monitoring data depends heavily on the availability of the monitoring system, and the situations of false alarm, missing alarm and the like of the monitoring alarm often occur, so that the monitoring result is inconsistent with the actual result.
In summary, a monitoring method that performs resource early warning through real data of a system online for a long time and can get rid of strong dependence of the monitoring system is absent at present, how to provide a new solution, and solving the technical problems is a technical problem to be solved urgently in the field.
Disclosure of Invention
In order to solve at least one technical problem in the background art, the present invention provides a method and an apparatus for predicting system load capacity based on a block chain.
In order to achieve the above object, according to an aspect of the present invention, there is provided a system load capacity prediction method based on a block chain, the method including:
acquiring load capacity data of a target system from a preset block chain network, wherein block chain nodes in the block chain network upload real-time load capacity data of the target system to the block chain network;
generating load capacity time sequence data of the target system according to the load capacity data;
and inputting the load capacity time sequence data into a preset system load capacity prediction model to obtain a load capacity prediction result of the target system, wherein the system load capacity prediction model is obtained by training a preset machine learning model by adopting training data, and the training data is load capacity time sequence data generated according to historical load capacity data of the target system.
Optionally, the method for predicting system load capacity based on a block chain further includes:
creating a producer of a message queue to transmit real-time load capacity data of the target system detected by a monitoring system to the message queue through the producer;
creating a consumer of the message queue at the blockchain node to transmit real-time load capacity data of the target system to the blockchain node through the message queue.
Optionally, the obtaining load capacity data of the target system from the preset block chain network specifically includes:
when new load capacity data corresponding to the target system is uploaded to the block chain network, acquiring the new load capacity data from the block chain network;
the generating load capacity time series data of the target system according to the load capacity data specifically includes:
and combining the new load capacity data with the acquired load capacity data of the target system according to a time sequence to obtain load capacity time sequence data.
Optionally, the method for predicting system load capacity based on a block chain further includes:
acquiring the training data;
and training an ARMA model according to the training data to obtain the system load capacity prediction model.
Optionally, the training the ARMA model according to the training data to obtain the system load capacity prediction model includes:
and verifying the prediction accuracy of the trained ARMA model according to preset verification data, and if the verification prediction accuracy is higher than a preset threshold value, determining the ARMA model at the moment as the system load capacity prediction model.
Optionally, the training the ARMA model according to the training data to obtain the system load capacity prediction model specifically includes:
performing white noise check on the training data;
for the training data which is detected to have no noise data, the ARMA model is trained by adopting the training data;
and for the training data with the detected noise data, carrying out model parameter estimation on the ARMA model by adopting a maximum likelihood ratio method, then carrying out order determination on the model by using a BIC information criterion, determining p and q parameter estimation of the ARMA model in an order determination mode, then establishing a BIC matrix to carry out recursive calculation on p and q parameter values of the ARMA model, taking out the corresponding p and q parameters of the minimum BIC value as the optimal order determination parameters, and establishing a sequence model by establishing the optimal order determination parameters.
Optionally, the method for predicting system load capacity based on a block chain further includes:
and if the load capacity prediction result meets a preset early warning condition, generating early warning information and sending the early warning information to a management terminal.
In order to achieve the above object, according to another aspect of the present invention, there is provided a block chain-based system load capacity prediction apparatus, including:
the system comprises a data acquisition unit, a data processing unit and a data processing unit, wherein the data acquisition unit is used for acquiring load capacity data of a target system from a preset block chain network, and block chain nodes in the block chain network upload real-time load capacity data of the target system to the block chain network;
the time sequence data generation unit is used for generating load capacity time sequence data of the target system according to the load capacity data;
and the system load capacity prediction unit is used for inputting the load capacity time sequence data into a preset system load capacity prediction model to obtain a load capacity prediction result of the target system, wherein the system load capacity prediction model is obtained by training a preset machine learning model by adopting training data, and the training data is load capacity time sequence data generated according to historical load capacity data of the target system.
In order to achieve the above object, according to another aspect of the present invention, there is also provided a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the above system load capacity prediction method based on block chains when executing the computer program.
To achieve the above object, according to another aspect of the present invention, there is also provided a computer readable storage medium having stored thereon a computer program/instructions which, when executed by a processor, implement the steps of the above block chain based system load capacity prediction method.
The invention has the beneficial effects that:
according to the method and the device, the load capacity time sequence data of the target system are obtained from a preset block chain network, the load capacity time sequence data of the target system are generated according to the load capacity data, and then the load capacity time sequence data are input into a preset system load capacity prediction model to obtain the load capacity prediction result of the target system.
Drawings
In order to more clearly illustrate the embodiments or technical solutions of the present invention, the drawings used in the embodiments or technical solutions in the prior art are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts. In the drawings:
fig. 1 is a first flowchart of a block chain-based system load capacity prediction method according to an embodiment of the present invention;
fig. 2 is a second flowchart of a block chain-based system load capacity prediction method according to an embodiment of the present invention;
FIG. 3 is a third flowchart of a block chain-based system load capacity prediction method according to an embodiment of the present invention;
FIG. 4 is a block chain network according to an embodiment of the present invention;
fig. 5 is a block diagram of a system load capacity prediction apparatus based on a block chain according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a computer device according to an embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It should be noted that the terms "comprises" and "comprising," and any variations thereof, in the description and claims of the present invention and the above-described drawings, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
It should be noted that, in the technical solution of the present application, the acquisition, storage, use, processing, etc. of data all conform to the relevant regulations of the national laws and regulations.
It should be noted that the block chain-based system load capacity prediction method and apparatus of the present invention may be applied to the financial field, and may also be applied to other technical fields.
The invention provides a method for extracting load capacity data, constructing a data model for predicting load capacity, carrying out risk alarm on the load capacity of a system through a model output value, constructing a network monitoring model through a block chain network, getting rid of strong dependence of the monitoring system and solving the technical problem that system faults are caused by the fact that the disk capacity resources of the existing system are easy to fill.
Fig. 1 is a first flowchart of a block chain based system load capacity prediction method according to an embodiment of the present invention, and as shown in fig. 1, the block chain based system load capacity prediction method according to an embodiment of the present invention includes steps S101 to S103.
Step S101, acquiring load capacity data of a target system from a preset block chain network, wherein block chain nodes in the block chain network upload real-time load capacity data of the target system to the block chain network.
The invention utilizes the block chain technology to build the block chain network, and particularly can store the real-time load capacity data of the target system collected by the monitoring system in an uplink manner as shown in fig. 4, thereby ensuring the data to be not falsified and credible and ensuring the information transmission and information sharing among nodes.
In the present invention, the step may specifically be performed by using any one of the blockchain nodes in the blockchain network to acquire data on the blockchain. In an embodiment of the present invention, a group of block link nodes is created for each group of monitoring systems, a consensus algorithm is used to ensure the credibility and the non-tamper-ability of data, each group of block link nodes can write monitoring data (i.e., load capacity data), if a single block link node fails, the availability of the monitoring system can still be ensured, and an independent group of nodes is created for a corresponding monitoring output system to obtain data on the block link.
In the invention, real-time load capacity data of a target system acquired by a monitoring system is uploaded to a block chain through any block chain link point in the block chain network.
In an embodiment of the present invention, the load capacity data specifically includes: attribute ID, acquisition time, server IP information, existing capacity of a disk, total capacity of the disk and the like.
In one embodiment of the invention, the load capacity may specifically refer to the disk capacity of the servers of the system.
And step S102, generating load capacity time sequence data of the target system according to the load capacity data.
In the invention, the load capacity data acquired currently and the load capacity data installation time data acquired before can be arranged to generate the load capacity time sequence data.
In one embodiment of the present invention, the number of pieces of data in the load capacity timing data is a fixed value.
Step S103, inputting the load capacity time sequence data into a preset system load capacity prediction model to obtain a load capacity prediction result of the target system, wherein the system load capacity prediction model is obtained by training a preset machine learning model by using training data, and the training data is load capacity time sequence data generated according to historical load capacity data of the target system.
In an embodiment of the present invention, the load capacity prediction result may be a load capacity value of the target system at a next time corresponding to the predicted current time. In an embodiment of the present invention, the load capacity value may specifically be a system resource usage rate. In an embodiment of the present invention, the system resource usage may specifically be a disk usage.
In an embodiment of the present invention, a process of uploading real-time load capacity data of a target system to a block chain network specifically includes:
the block chain nodes receive load capacity data, a consensus module is called to complete consensus of the information, then an intelligent contract is executed, main body information, namely a regular data source output by a transformation unit of a data preprocessing device, is registered into each block chain node, the consistency of the main body information on all block chain nodes is ensured, and a hash value (assumed as H) is calculated as a characteristic value according to a splicing result of key element attribute ID, acquisition time, the existing capacity of a disk and the total capacity of the disk of the main body and is stored into the block chain for subsequent comparison.
After receiving information registration requests from a plurality of main bodies, the block chain network is responsible for receiving the information attached to the requests, including attribute ID numbers and batch numbers of the reported information, verifying signature and confirming identity, and adopting a PBFT (physical vapor transport) practical Byzantine fault-tolerant algorithm to sequence nodes, generate blocks and broadcast to all the nodes to achieve common identification chaining.
When receiving a query verification request from an identity verification system, querying according to an attribute ID number or a delivery information batch number, calculating a hash value (assumed to be H '), checking data in a block chain, querying an on-chain address according to the attribute ID number or the delivery information batch number if a record characteristic value H is H', and querying load capacity data corresponding to the number in the block chain network according to the queried on-chain address.
As shown in fig. 2, in an embodiment of the present invention, the method for predicting system load capacity based on a block chain of the present invention further includes step S201 and step S202.
Step S201, creating a producer of a message queue, so as to transmit real-time load capacity data of the target system detected by a monitoring system to the message queue through the producer.
In one embodiment of the invention, the message queue may employ Kafka. Kafka is a source-opened message engine system, has obvious performance advantage, can support to accurately release messages in real time, can be compatible with cross-platform transmission through an AMQP protocol, and can be adapted to different information systems of various monitoring servers. In an embodiment of the present invention, the message queue Kafka used in the present invention may specifically be a distributed Kafka cluster.
In an embodiment of the present invention, in this step, a producer may be specifically created for the monitoring system corresponding to the target system, that is, the monitoring system corresponding to the target system is used as the producer to directly upload data to the Kafka cluster.
Step S202, a consumer of the message queue is created at the blockchain node, so as to transmit the real-time load capacity data of the target system to the blockchain node through the message queue.
The invention uses Kafka to upload data, constructs a transmission channel for providers and consumers, and reports transaction information to a block chain node in a block chain network.
In the invention, the invention creates a program unit (namely a producer) which issues messages to the Kafka cluster, can issue the messages to any object of the topic, and after the unit receives a report data source transmitted by a data preprocessing device, the unit module is used for issuing the messages to the topic of the cluster unit.
Published messages typically have two modes: a queue mode (queuing) and a publish-subscribe mode (publish-subscribe). In the queue mode, the consumer concurers can read messages from the server simultaneously, and each message is read by only one of the consumer concurers; messages in publish-subscribe mode are broadcast to all consumer consumers. In order to ensure that data can be accurately retrieved by consumers, the invention adopts a queue mode to realize message publishing.
The client application program requesting the message from the message queue can subscribe one or more topic, acquire the data in the cluster unit and be responsible for transmitting the data to the nodes in the block chain network.
In an embodiment of the present invention, the acquiring load capacity data of the target system from the preset block chain network in step S101 specifically includes:
and when new load capacity data corresponding to the target system is uploaded to the block chain network, acquiring the new load capacity data from the block chain network.
In an embodiment of the present invention, the generating load capacity time series data of the target system according to the load capacity data in step S102 specifically includes:
and combining the new load capacity data with the acquired load capacity data of the target system according to a time sequence to obtain load capacity time sequence data.
As shown in fig. 3, in an embodiment of the present invention, the method for predicting system load capacity based on a block chain of the present invention further includes step S301 and step S302.
Step S301, acquiring the training data.
In one embodiment of the invention, the invention acquires historical load capacity data of the server of the target system and extracts load capacity data of the newly added server every day at a daily timing. And then, removing the obtained repeated and useless data, and reserving key capacity attribute information, wherein the attribute information comprises an attribute ID, data acquisition time, server IP information and the current use amount of each mounted disk.
In one embodiment of the invention, the cleaned data is constructed according to attributes such as an attribute ID, acquisition time, the existing capacity of a disk, the total capacity of the disk and the like, the historical load capacity data acquired for the first time is used as modeling source data of a system modeling device, if the total acquisition information is m records, the last n records are used as verification data, m-n records are used as training data, and the total capacity of the disk is set as a fixed value and is not constructed repeatedly.
In one embodiment of the invention, the invention registers the training data after attribute construction in a database and carries out regularized output, uniformly converts the information into string types in character format, converts the used capacity and the total capacity into numerical types, and converts field information comprising attribute ID, acquisition time, the existing capacity of a disk, the total capacity of the disk and the like into a modeling data source capable of modeling and identifying.
And S302, training an ARMA model according to the training data to obtain the system load capacity prediction model.
In an embodiment of the present invention, the present invention may further use an AR model or an MA model for model training to obtain a system load capacity prediction model.
In an embodiment of the present invention, the training the ARMA model according to the training data in step S302 to obtain the system load capacity prediction model specifically includes:
performing white noise check on the training data;
for the training data which is detected to have no noise data, the ARMA model is trained by adopting the training data;
and for the training data with the detected noise data, carrying out model parameter estimation on the ARMA model by adopting a maximum likelihood ratio method, then carrying out order determination on the model by using a BIC information criterion, determining p and q parameter estimation of the ARMA model in an order determination mode, then establishing a BIC matrix to carry out recursive calculation on p and q parameter values of the ARMA model, taking the corresponding p and q parameters of the minimum BIC value as the optimal order determination parameters, and establishing a sequence model by establishing the optimal order determination parameters.
When the ARMA model is trained according to training data, stability test and white noise test are firstly carried out on an input sequence value (namely the training data), for a traditional information monitoring system, only the original monitoring data is generally output as data, and pseudo regression phenomenon of the model is easy to generate, so that the stability test is added to the model, the stability test is used for determining that no determined trend or random trend exists in the original data sequence, and the white noise test is used for determining that useful information in the sequence is completely extracted and has no random noise.
For noisy data that has been checked for white noise, an ARMA model may be used for training. For data with noise, a maximum likelihood ratio method is adopted for model parameter estimation, then, the model is subjected to order determination by using a BIC information criterion according to different models, p and q parameter estimation of a modeling ARMA model is determined in an order determination mode, the maximum parameter values of p and q generally do not exceed one tenth of the length of a data set, then, a BIC matrix is established, the p and q values are subjected to recursive calculation by using an ARIMA model, the corresponding p and q parameters of the minimum BIC value are taken out to serve as the optimal order determination parameters, and a complete sequence model is established after the parameters are set.
After the model is determined, whether the residual sequence is white noise is checked, if not, useful information still exists, model identification needs to be carried out on the residual sequence again, the minimum BIC value of p and q parameter calculation is determined, and therefore the optimal model is trained.
In an embodiment of the present invention, the training the ARMA model according to the training data in step S302 to obtain the system load capacity prediction model includes:
and verifying the prediction accuracy of the trained ARMA model according to preset verification data, and if the verification prediction accuracy is higher than a preset threshold value, determining the ARMA model at the moment as the system load capacity prediction model.
In the invention, when data is acquired, total acquisition information is recorded as m records, n records at the last moment are used as verification data, m-n records are used as training data, the prediction results of the n records are compared with the real results, and the precision threshold of the prediction value is determined by establishing the indexes of average absolute error, root mean square error and average absolute percentage error, and the threshold also needs to be dynamically adjusted according to the actual service requirement of the system, generally, the threshold is higher for the system with larger service volume. And if the predicted value of the model is determined to reach the standard through error analysis, determining the model at the moment as a system load capacity prediction model, and performing subsequent model prediction application.
In an embodiment of the present invention, the method for predicting system load capacity based on a block chain further includes:
and if the load capacity prediction result meets a preset early warning condition, generating early warning information and sending the early warning information to a management terminal.
In one embodiment of the invention, the capacity ratio threshold is set individually, and different early warning conditions are set according to the importance of different systems. In an embodiment of the present invention, the early warning condition specifically includes 5 early warning levels, and the capacity ratio of each level is a, B, c, d, e, where a < B < c < d < e, and for system a, if the actual traffic growth rate is slow, the predicted value is set to be greater than e, an alarm is generated, and for system B, if the actual traffic growth rate is fast, the predicted value is set to be greater than d, an alarm is generated.
In the invention, the load capacity prediction result is compared with each early warning condition, and if the early warning condition is met, early warning information is generated and issued.
In an embodiment of the present invention, the early warning information may be in the form of an early warning information indication table, and table 1 below is an example of the early warning information indication table, and the early warning information indication table is finally issued through a system identifier, an early warning level, a server IP, early warning indication information, and early warning time to prompt a monitoring administrator to predict an early warning result and prompt the monitoring administrator to pay attention to a system server that needs attention at present.
Figure BDA0003716031430000101
TABLE 1
It can be seen from the above embodiments that, in order to solve the problem in the field of performance capacity monitoring that system resources cannot perform system early warning through the real usage amount after the system is online and get rid of the strong dependence of the monitoring system, the present invention provides a method for extracting disk data, constructing a data model of disk data performance capacity, performing risk alarm on the disk capacity through the model output value, and constructing a network monitoring model through a block chain network to get rid of the strong dependence of the monitoring system. Its advantages are as follows:
1. according to the input data of the time sequence, the long-time resource utilization rate of the system after the system is on line is predicted really through an intelligent algorithm, and the system fault risk caused by the conditions that the system capacity is increased rapidly and the like is reduced effectively;
2. the problem of short board monitoring of application deployment events can be solved through event pre-judgment, and the condition that monitoring alarm can be triggered only after an event occurs is avoided;
3. the authenticity and high availability of monitoring information can be guaranteed through a block chain technology, the strong dependence on a monitoring system is reduced, the situations of misinformation or missing report are greatly reduced, and the availability of the whole monitoring system is improved;
4. according to the resource usage prediction of the time series algorithm, the resource usage rates of different systems can be judged more truly, so that the resource can be more accurately provided for application systems needing resources in resource allocation, and the resource usage rate is improved.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
Based on the same inventive concept, an embodiment of the present invention further provides a system load capacity prediction apparatus based on a block chain, which can be used to implement the system load capacity prediction method based on the block chain described in the foregoing embodiment, as described in the following embodiment. As the principle of solving the problem of the system load capacity prediction apparatus based on the block chain is similar to that of the system load capacity prediction method based on the block chain, the embodiment of the system load capacity prediction apparatus based on the block chain can refer to the embodiment of the system load capacity prediction method based on the block chain, and repeated details are omitted. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 5 is a block diagram of a system load capacity prediction apparatus based on a block chain according to an embodiment of the present invention, and as shown in fig. 5, in an embodiment of the present invention, the system load capacity prediction apparatus based on a block chain according to the present invention includes:
the system comprises a data acquisition unit 1, a block chain node and a block chain link control unit, wherein the data acquisition unit is used for acquiring load capacity data of a target system from a preset block chain network, and the block chain link node in the block chain network uploads real-time load capacity data of the target system to the block chain network;
a time series data generating unit 2, configured to generate load capacity time series data of the target system according to the load capacity data;
and the system load capacity prediction unit 3 is configured to input the load capacity time series data into a preset system load capacity prediction model to obtain a load capacity prediction result of the target system, where the system load capacity prediction model is obtained by training a preset machine learning model with training data, and the training data is load capacity time series data generated according to historical load capacity data of the target system.
In an embodiment of the present invention, the apparatus for predicting system load capacity based on a block chain further includes:
a producer creating unit, configured to create a producer of a message queue, so as to transmit real-time load capacity data of the target system, detected by a monitoring system, to the message queue through the producer;
a consumer creating unit for creating a consumer of the message queue at the block chain node to transmit real-time load capacity data of the target system to the block chain node through the message queue.
In an embodiment of the present invention, the data obtaining unit 1 is specifically configured to obtain new load capacity data corresponding to the target system from the blockchain network when the new load capacity data is uploaded to the blockchain network;
in an embodiment of the present invention, the time series data generating unit 2 is specifically configured to combine the new load capacity data with the acquired load capacity data of the target system according to a time sequence to obtain load capacity time series data.
In an embodiment of the present invention, the apparatus for predicting system load capacity based on a block chain further includes:
a training data acquisition unit for acquiring the training data;
and the model training unit is used for training the ARMA model according to the training data to obtain the system load capacity prediction model.
In an embodiment of the present invention, the model training unit includes:
and the training effect verification module is used for verifying the prediction accuracy of the trained ARMA model according to preset verification data, and if the verification prediction accuracy is higher than a preset threshold value, determining the ARMA model at the moment as the system load capacity prediction model.
In an embodiment of the present invention, the apparatus for predicting system load capacity based on a block chain further includes:
and the early warning unit is used for generating early warning information and sending the early warning information to the management terminal if the load capacity prediction result meets a preset early warning condition.
To achieve the above object, according to another aspect of the present application, there is also provided a computer apparatus. As shown in fig. 6, the computer device comprises a memory, a processor, a communication interface and a communication bus, wherein a computer program that can be run on the processor is stored in the memory, and the steps of the method of the above embodiment are realized when the processor executes the computer program.
The processor may be a Central Processing Unit (CPU). The Processor may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or a combination thereof.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and units, such as the corresponding program units in the above-described method embodiments of the present invention. The processor executes various functional applications of the processor and the processing of the work data by executing the non-transitory software programs, instructions and modules stored in the memory, that is, the method in the above method embodiment is realized.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor, and the like. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and such remote memory may be coupled to the processor via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more units are stored in the memory and when executed by the processor perform the method of the above embodiments.
The specific details of the computer device may be understood by referring to the corresponding related descriptions and effects in the above embodiments, and are not described herein again.
In order to achieve the above object, according to another aspect of the present application, there is also provided a computer-readable storage medium storing a computer program which, when executed in a computer processor, implements the steps in the above block chain based system load capacity prediction method. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD) or a Solid State Drive (SSD), etc.; the storage medium may also comprise a combination of memories of the kind described above.
To achieve the above object, according to another aspect of the present application, there is also provided a computer program product comprising computer program/instructions which, when executed by a processor, implement the steps of the above block chain based system load capacity prediction method.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A method for predicting system load capacity based on block chains is characterized by comprising the following steps:
acquiring load capacity data of a target system from a preset block chain network, wherein block chain nodes in the block chain network upload real-time load capacity data of the target system to the block chain network;
generating load capacity time sequence data of the target system according to the load capacity data;
and inputting the load capacity time sequence data into a preset system load capacity prediction model to obtain a load capacity prediction result of the target system, wherein the system load capacity prediction model is obtained by training a preset machine learning model by adopting training data, and the training data is load capacity time sequence data generated according to historical load capacity data of the target system.
2. The method of predicting system load capacity based on block chains according to claim 1, further comprising:
creating a producer of a message queue to transmit real-time load capacity data of the target system detected by a monitoring system to the message queue through the producer;
creating a consumer of the message queue at the blockchain node to transmit real-time load capacity data of the target system to the blockchain node through the message queue.
3. The method for predicting system load capacity based on a block chain according to claim 1, wherein the acquiring load capacity data of a target system from a preset block chain network specifically includes:
when new load capacity data corresponding to the target system is uploaded to the block chain network, acquiring the new load capacity data from the block chain network;
the generating load capacity time series data of the target system according to the load capacity data specifically includes:
and combining the new load capacity data with the acquired load capacity data of the target system according to a time sequence to obtain load capacity time sequence data.
4. The method of predicting system load capacity based on block chains according to claim 1, further comprising:
acquiring the training data;
and training an ARMA model according to the training data to obtain the system load capacity prediction model.
5. The method according to claim 4, wherein the training an ARMA model according to the training data to obtain the system load capacity prediction model comprises:
and verifying the prediction accuracy of the trained ARMA model according to preset verification data, and if the verification prediction accuracy is higher than a preset threshold value, determining the ARMA model at the moment as the system load capacity prediction model.
6. The block chain-based system load capacity prediction method according to claim 4, wherein the training an ARMA model according to the training data to obtain the system load capacity prediction model specifically comprises:
performing white noise check on the training data;
for the training data which is detected to have no noise data, the ARMA model is trained by adopting the training data;
and for the training data with the detected noise data, carrying out model parameter estimation on the ARMA model by adopting a maximum likelihood ratio method, then carrying out order determination on the model by using a BIC information criterion, determining p and q parameter estimation of the ARMA model in an order determination mode, then establishing a BIC matrix to carry out recursive calculation on p and q parameter values of the ARMA model, taking the corresponding p and q parameters of the minimum BIC value as the optimal order determination parameters, and establishing a sequence model by establishing the optimal order determination parameters.
7. The method of predicting system load capacity based on block chains according to claim 1, further comprising:
and if the load capacity prediction result meets a preset early warning condition, generating early warning information and sending the early warning information to a management terminal.
8. An apparatus for predicting system load capacity based on block chains, comprising:
the system comprises a data acquisition unit, a data processing unit and a data processing unit, wherein the data acquisition unit is used for acquiring load capacity data of a target system from a preset block chain network, and block chain nodes in the block chain network upload real-time load capacity data of the target system to the block chain network;
the time sequence data generation unit is used for generating load capacity time sequence data of the target system according to the load capacity data;
and the system load capacity prediction unit is used for inputting the load capacity time sequence data into a preset system load capacity prediction model to obtain a load capacity prediction result of the target system, wherein the system load capacity prediction model is obtained by training a preset machine learning model by adopting training data, and the training data is load capacity time sequence data generated according to historical load capacity data of the target system.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method according to any of claims 1 to 7 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, on which a computer program/instructions are stored, characterized in that the computer program/instructions, when executed by a processor, implement the steps of the method of any one of claims 1 to 7.
CN202210736445.0A 2022-06-27 2022-06-27 System load capacity prediction method and device based on block chain Pending CN115061891A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210736445.0A CN115061891A (en) 2022-06-27 2022-06-27 System load capacity prediction method and device based on block chain

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210736445.0A CN115061891A (en) 2022-06-27 2022-06-27 System load capacity prediction method and device based on block chain

Publications (1)

Publication Number Publication Date
CN115061891A true CN115061891A (en) 2022-09-16

Family

ID=83202512

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210736445.0A Pending CN115061891A (en) 2022-06-27 2022-06-27 System load capacity prediction method and device based on block chain

Country Status (1)

Country Link
CN (1) CN115061891A (en)

Similar Documents

Publication Publication Date Title
CN111966289B (en) Partition optimization method and system based on Kafka cluster
CN106815254B (en) Data processing method and device
CN109684052B (en) Transaction analysis method, device, equipment and storage medium
US20140215055A1 (en) Monitoring network entities via a central monitoring system
CN111970195B (en) Data transmission method and streaming data transmission system
CN110147470B (en) Cross-machine-room data comparison system and method
CN113221163B (en) Model training method and system
CN112632129A (en) Code stream data management method, device and storage medium
CN113420323B (en) Data sharing method and terminal equipment
CN113760677A (en) Abnormal link analysis method, device, equipment and storage medium
CN112732756A (en) Data query method, device, equipment and storage medium
CN116308824A (en) Knowledge graph-based group fraud risk identification method and related equipment
CN115061891A (en) System load capacity prediction method and device based on block chain
CN116910144A (en) Computing power network resource center, computing power service system and data processing method
CN115220131A (en) Meteorological data quality inspection method and system
CN114756301A (en) Log processing method, device and system
CN111294553B (en) Method, device, equipment and storage medium for processing video monitoring service signaling
CN114048512A (en) Method and device for processing sensitive data
CN113158497A (en) Online service experiment method and device, computer equipment and storage medium
CN114428704A (en) Method and device for full-link distributed monitoring, computer equipment and storage medium
CN111711537B (en) Method, device and equipment for updating standby main node list
CN112491589B (en) Object storage network early warning method, system, terminal and storage medium
CN114422324B (en) Alarm information processing method and device, electronic equipment and storage medium
CN115396319B (en) Data stream slicing method, device, equipment and storage medium
CN113114612B (en) Determination method and device for distributed system call chain

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination