CN117455666A - Transaction technical index prediction method, device and equipment based on neural network - Google Patents
Transaction technical index prediction method, device and equipment based on neural network Download PDFInfo
- Publication number
- CN117455666A CN117455666A CN202311336618.0A CN202311336618A CN117455666A CN 117455666 A CN117455666 A CN 117455666A CN 202311336618 A CN202311336618 A CN 202311336618A CN 117455666 A CN117455666 A CN 117455666A
- Authority
- CN
- China
- Prior art keywords
- data
- index
- prediction
- transaction
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 30
- 238000012549 training Methods 0.000 claims abstract description 38
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 238000003860 storage Methods 0.000 claims abstract description 13
- 238000000605 extraction Methods 0.000 claims abstract description 12
- 230000004044 response Effects 0.000 claims abstract description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 44
- 230000006870 function Effects 0.000 claims description 24
- 230000001932 seasonal effect Effects 0.000 claims description 21
- 230000002159 abnormal effect Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 2
- 230000000737 periodic effect Effects 0.000 abstract description 3
- 230000008859 change Effects 0.000 description 11
- 238000012544 monitoring process Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000012423 maintenance Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000005856 abnormality Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000007774 longterm Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000011835 investigation Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
- G06Q30/0202—Market predictions or forecasting for commercial activities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/02—Banking, e.g. interest calculation or account maintenance
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Theoretical Computer Science (AREA)
- Development Economics (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Data Mining & Analysis (AREA)
- General Business, Economics & Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Technology Law (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Game Theory and Decision Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a transaction technical index prediction method, a device, equipment and a storage medium based on a neural network, which comprises the following steps: acquiring historical index data, preprocessing the historical index data, including data division and data feature extraction, to obtain a training data set, wherein the historical index data comprises average response time, transaction success rate and transaction number; inputting the training data set into a model constructed based on an LSTM network for training to obtain an index prediction model, and determining an upper baseline and a lower baseline in the obtained index prediction model; and inputting the index data to be predicted into the index prediction model to predict, and obtaining a prediction result of a future time period. Periodic fluctuation can be more effectively identified, the threshold value can be dynamically adjusted, the method is more in line with actual application scenes, and the prediction accuracy is improved.
Description
Technical Field
The present invention relates to the field of banking monitoring technologies, and in particular, to a method, an apparatus, and a device for predicting a transaction technical index based on a neural network.
Background
Single index prediction is an important class of algorithms in intelligent operation and maintenance index analysis. In the traditional mode, operation and maintenance personnel can know after the fault occurs, and then find the cause of the fault to solve the problem. The method has certain hysteresis, and some faults can be seriously influenced after occurrence, so that the method is unfavorable for the normal operation of the production system.
In the prior art, most of the prediction algorithms perform curve fitting on collected historical data samples, calculate the overall running trend of the data, and thus predict the trend of future data. The algorithm has the defects that the function is difficult to fit some fluctuation with time periodicity or nonlinear fluctuation, the fitting function cannot be set with preset parameters under the condition that the data change rule is not known, the prediction result is inaccurate and untimely, the nonlinear data change trend cannot be successfully simulated, and a reasonable data fluctuation interval cannot be calculated. Therefore, how to find an algorithm capable of predicting nonlinear periodic time series data is an important technical problem of the current intelligent operation and maintenance work.
Disclosure of Invention
Therefore, the present invention aims to provide a transaction technical index prediction method, device and equipment based on a neural network, which aims to solve the nonlinear data prediction and monitoring problems.
In order to achieve the above object, the present invention provides a transaction technical index prediction method based on a neural network, the method comprising:
acquiring historical index data, preprocessing the historical index data, including data division and data feature extraction, to obtain a training data set, wherein the historical index data comprises average response time, transaction success rate and transaction number;
inputting the training data set into a model constructed based on an LSTM network for training to obtain an index prediction model, and determining an upper baseline and a lower baseline in the obtained index prediction model;
and inputting the index data to be predicted into the index prediction model to predict, and obtaining a prediction result of a future time period.
Preferably, the method further comprises:
judging whether the prediction result exceeds an alarm threshold value, if so, sending out an abnormal alarm; the alarm threshold is constructed by using baseline.
Preferably, the preprocessing of the historical index data including data division and data feature extraction is performed to obtain a training data set, including:
dividing the historical index data according to preset input length, output length and sliding window step length division to obtain a plurality of groups of input and output data pairs;
and extracting the data characteristics of each input and output data pair through an STL algorithm to obtain the training data set, wherein the data characteristics comprise seasonal characteristics, trend characteristics and residual characteristics.
Preferably, the STL algorithm includes an inner loop; the extracting the data characteristic of each input and output data pair through the STL algorithm comprises the following steps:
automatically deducing a corresponding seasonal period according to the index of the historical index data to obtain the period length;
converging the historical index data at different moments in one period to obtain a subsequence;
subtracting the previous trend feature obtained in the previous round from the historical index data in each internal cycle, and carrying out regression on each subsequence by using a LOESS algorithm to obtain a first result;
carrying out sliding average of preset length on the first result and carrying out regression on each subsequence through an LOESS algorithm to obtain a second result;
subtracting the second result from the first result to obtain the seasonal feature;
subtracting the seasonal features from the historical index data and carrying out regression on each subsequence through a LOESS algorithm to obtain the trend features;
and decomposing the seasonal features and the trend features based on the historical index data to obtain the residual features.
Preferably, the network architecture of the index prediction model includes an input layer, two LSTM network layers and a full connection layer, where the hidden layer of the two LSTM network layers including the lower LSTM is fully connected with the higher LSTM through a feed-forward connection.
Preferably, the inputting the training data set into a model constructed based on the LSTM network for training includes:
model optimization is performed by using the average absolute error loss function and function activation is performed by using the softsign function.
Preferably, the determining the upper and lower baselines includes:
calculating according to the predicted previous day history index data and the width coefficient to obtain the upper and lower baselines; wherein,
the upper base line is upper (x) t )＝estimation(x t )+width*x magnitude (x t )/2，
Lower baseline is lower (x) t )＝estimation(x t )-width*x magnitude (x t ) And/2, wherein,
x magnitude ＝(mean(last day )+median(last day ))/2，last day represents the previous day history index data, and width represents the width factor.
In order to achieve the above object, the present invention further provides a transaction technical index prediction device based on a neural network, the device comprising:
the preprocessing unit is used for acquiring historical index data, preprocessing the historical index data, including data division and data feature extraction, to obtain a training data set, wherein the historical index data comprises average response time, transaction success rate and transaction number;
the training unit is used for inputting the training data set into a model constructed based on an LSTM network for training to obtain an index prediction model, and determining an upper baseline and a lower baseline in the obtained index prediction model;
and the prediction unit is used for inputting the index data to be predicted into the index prediction model to predict, so as to obtain a prediction result of a future time period.
In order to achieve the above object, the present invention also proposes an apparatus comprising a processor, a memory, and a computer program stored in the memory, the computer program being executed by the processor to implement the steps of a neural network-based transaction technical index prediction method according to the above embodiment.
In order to achieve the above object, the present invention further proposes a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of a neural network-based transaction technical index prediction method according to the above embodiments.
The beneficial effects are that:
according to the scheme, through learning the data history rule based on the index prediction model constructed by the LSTM network and predicting future data according to the data of the current period of time, operation and maintenance personnel can be informed of possible faults in the next process in advance. Periodic fluctuation can be more effectively identified, the threshold value can be dynamically adjusted, the method is more in line with actual application scenes, and the prediction accuracy is improved.
According to the scheme, the index prediction model constructed based on the LSTM network can have memory capacity on long-term data, high data utilization rate and more sufficient feature learning, and the algorithm is high in automation degree, can autonomously learn data rules, and does not need manual parameter adjustment and adjustment effects.
According to the scheme, abnormal changes of the transaction technical indexes are caused by the occurrence of the abnormality when the transaction technical indexes are predicted, or the indexes are beyond a safety range, so that an alarm is generated and sent to corresponding management personnel for investigation, and the service interruption rate and the interruption time can be effectively reduced.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a transaction technical index prediction method based on a neural network according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of dividing index data according to an embodiment of the invention.
Fig. 3 is a schematic diagram of a network structure of an LSTM training procedure according to an embodiment of the present invention.
Fig. 4 (a) and (B) are diagrams illustrating a screen shot of an alarm generated by a prediction process according to an embodiment of the present invention.
Fig. 5 is a schematic diagram of data fluctuation corresponding to the occurrence of the anomaly in fig. 4 (B) according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating an example of a prediction process anomaly.
Fig. 7 is a schematic structural diagram of a transaction technical index prediction device based on a neural network according to an embodiment of the invention.
The realization of the object, the functional characteristics and the advantages of the invention will be further described with reference to the accompanying drawings in connection with the embodiments.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, based on the embodiments of the invention, which are apparent to those of ordinary skill in the art without inventive faculty, are intended to be within the scope of the invention. Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, based on the embodiments of the invention, which are apparent to those of ordinary skill in the art without inventive faculty, are intended to be within the scope of the invention.
In the description of the present invention, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature.
The following describes the invention in detail with reference to examples.
Taking an application scenario of the prediction algorithm in the technical operation and maintenance as an example, because of the operation rule of the service, part of tasks can be operated only on fixed dates of each month, related batches on the dates can be operated relatively slowly, and how to enable the prediction algorithm to remember information such as time rules becomes a key point and a difficulty of research. Based on this, the present invention proposes to utilize a machine learning algorithm for processing. According to the condition of the historical data, the development trend of a period of time in the future can be predicted; or based on current data, predict that the normal track will not deviate significantly in the following. These scenarios are significant in practical applications, such as predicting server capacity data to determine if capacity expansion is needed, when capacity expansion is needed in the future; and early warning can be generated by short-time prediction and real-time monitoring.
Referring to fig. 1, a flow chart of a transaction technical index prediction method based on a neural network according to an embodiment of the invention is shown.
In this embodiment, the method includes:
s11, acquiring historical index data, preprocessing the historical index data, including data division and data feature extraction, to obtain a training data set, wherein the historical index data comprises average response time, transaction success rate and transaction number.
Further, in step S11, the preprocessing including data division and data feature extraction is performed on the historical index data to obtain a training data set, including:
s11-1, dividing the historical index data according to preset input length, output length and sliding window step length division to obtain a plurality of groups of input and output data pairs;
s11-2, extracting data characteristics of each input and output data pair through an STL algorithm to obtain the training data set, wherein the data characteristics comprise seasonal characteristics, trend characteristics and residual characteristics.
Wherein the STL algorithm comprises an inner loop; the extracting the data characteristic of each input and output data pair through the STL algorithm comprises the following steps:
automatically deducing a corresponding seasonal period according to the index of the historical index data to obtain the period length;
converging the historical index data at different moments in one period to obtain a subsequence;
subtracting the previous trend feature obtained in the previous round from the historical index data in each internal cycle, and carrying out regression on each subsequence by using a LOESS algorithm to obtain a first result;
carrying out sliding average of preset length on the first result and carrying out regression on each subsequence through an LOESS algorithm to obtain a second result;
subtracting the second result from the first result to obtain the seasonal feature;
subtracting the seasonal features from the historical index data and carrying out regression on each subsequence through a LOESS algorithm to obtain the trend features;
and decomposing the seasonal features and the trend features based on the historical index data to obtain the residual features.
Further, the STL algorithm includes an outer loop; the regression is performed on each sub-sequence by using the LOESS algorithm to obtain a first result, including:
multiplying a regression result obtained when each subsequence is regressed by a LOESS algorithm by a preset weight value to obtain the first result; wherein the preset weight value is introduced based on an outer loop.
In this embodiment, the method is applied to index data prediction of a scientific and technological operation and maintenance system in banking industry. The model constructed based on LSTM (long and short time memory) is selected to learn the historical data, and the model has the biggest characteristics that the effect of processing the data with larger relevance in time sequence is better. The LSTM model training data needs a structure of one input and one output, which means that the input is processed to obtain the output, and when the algorithm fully learns all possible conditions, the value of the future output length can be predicted according to the data of the current input length. Illustrating: in one application scenario, assuming that a piece of data has a length of 1000 units, a training set is fabricated by adopting a sliding window method according to the time sequence characteristics of the LSTM network. Reference is made to figure 2. For example, the input length of each input data is set to be 5, the output length is set to be 2, the step length of the sliding window is set to be 1, and after the setting, the data is divided into approximately 1000 groups of input and output groups corresponding to each other, and the corresponding input and output data set is used as a data set. Based on the setting mode, when the network is used for testing and practical application, the data of the first five continuous time points are input, and the network predicts the data of the last two time points.
When LSTM is used for time series prediction, time series data are not always used as the input of an algorithm, features are extracted from original data to the greatest extent for the algorithm and a model to use, and more complex relations between data are captured, so that the algorithm learning is more accurate. Specifically, indexes such as average response time, total transaction number, transaction success rate and the like of historical transactions of each transaction system in a bank are taken as original data, and the following three characteristics of the original data are extracted, wherein the three characteristics comprise: seasonal characteristics that contain information on day of week, date, month, week, day of work, holiday, etc.; trend characteristics including information such as running average, running median, running maximum, running minimum, difference, exponentially weighted running average, exponentially weighted running standard deviation, etc.; residual characteristics, including the residual after the original data has decomposed the two characteristics (seasonal characteristics and trend characteristics). The algorithm for extracting the features is a "season and trend decomposition algorithm based on a local weighted regression technique" (Seasonal and Trend Decomposition Using LOESS algorithm, hereinafter referred to as STL algorithm).
First, the LSTM model automatically deduces the seasonal period of the data according to the index of the data, and in this case, deduces that characteristic items such as weekends, holidays and the like, which affect the operation rule of the system, and obtains the length of the period (also referred to as a time window). And gathering the data at different moments in a period to obtain a subsequence. The STL algorithm can be mainly divided into an inner loop and an outer loop, in each inner loop, the trend characteristic T obtained by subtracting the result of the previous round of inner loop from the data is first subtracted, then regression is performed on each subsequence by using the LOESS algorithm (q=n1, d=1), (n is the number of samples in one cycle, n1 is the minimum odd number of n or more), and the obtained result is marked as C. The length of C is sequentially taken as a running average of n, n,3, and then a regression of the LOESS (q= 9,d =1) is taken to obtain the result L as a low flux of the cyclic subsequence. The seasonal feature S is the result obtained by C-L. And subtracting the seasonal feature S from the raw data, and performing regression of the log (q=n2, d=1), wherein n2 is an odd number between n and 2n, so as to obtain the trend feature T. The extracted three types of features cause the original 1-dimensional time series data to become x+1-dimensional data (x is the number of features) after feature extraction. While the outer loop of the STL algorithm introduces a robustness weight term, the inner loop is used to control the effect of outliers in the data. If the outlier of the data point is large, the weight becomes small, and the weight is used in the LoESS regression process of the inner loop, and the regression result is multiplied by the weight as the final result.
In one application direction of the present embodiment, it is assumed that the capacity change of one server needs to be predicted, and the following features of sample data need to be paid attention to: time information corresponding to each capacity data, and whether there is a situation that a large change occurs due to the specificity of the date; the following are various indexes of the capacity changing along with time, trend, residual numerical values and the like, and the indexes are extracted so as to facilitate the calculation of the neural network on the change rule. The data after feature extraction can be used as the input of the neural network.
S12, inputting the training data set into a model constructed based on an LSTM network for training to obtain an index prediction model, and determining an upper baseline and a lower baseline in the obtained index prediction model.
S13, inputting the index data to be predicted into the index prediction model for prediction, and obtaining a prediction result of a future time period.
In this embodiment, the network architecture of the index prediction model constructed based on the LSTM network is shown in fig. 3. LSTM is a recurrent neural network that can solve the long-term dependency problem, and compared with the traditional recurrent neural network RNN, it adds three layers of valve nodes, namely, forget gate, input gate and output gate, so that each layer of the network can judge which information can be input into the layer of computation and which information can be output to the next layer of computation. It retains long-term information and is more suitable for prediction of time series. The index prediction model comprises an input layer, two LSTM network layers and a full-connection layer, wherein the hidden layer of the low-level LSTM is fully connected with the high-level LSTM through feedforward connection. Wherein the input layer is operative to receive input data for delivery to the LSTM network layer. The LSTM network layer functions to convert data acquired from the input layer into a higher level feature representation for extracting features of the data. The function of the fully connected layer is to integrate each node with all the nodes necklace of the LSTM layer and integrate the previously extracted features. The loss function in the present network uses a mean absolute error loss function and the activation function uses a softsign function.
And inputting the training data set with the characteristics, which is obtained by extracting the characteristics from the input set obtained by dividing, into an LSTM network, taking the predicted data as output, and obtaining a set of proper index prediction models through iterative learning. By giving empirical parameters before this, the fitting speed and effect of the algorithm is guaranteed and a dropout mechanism is used in the network to prevent model overfitting.
And obtaining an index prediction model through learning historical data, and predicting future values according to the model. However, since the actual scene is complex and changeable, the prediction result cannot completely conform to the actual value, so that in order to increase the fault tolerance of the algorithm, an upper base line and a lower base line are added to the prediction result, the base line is calculated according to the data of the previous day of prediction and a width coefficient width,
upper(x t )＝estimation(x t )+width*x magnitude (x t )/2
lower(x t )＝estimation(x t )-width*x magnitude (x t )/2
wherein estimation (x t ) Is the value of the predicted result, x magnitude ＝(mean(last day )+median(last day ))/2，last day For the previous day history, mean is a function of the mean and mean is a function of the median. After the result after processing, i.e. after the LSTM learns the change rule of the sample data, a function for predicting the change trend of future data is formed inside the network, the data stream to be predicted is input into the function, and the predicted output is obtained, i.e. the prediction of the change of the input data stream in the next time or the next period of time.
In another embodiment, the method further comprises:
s14, judging whether the prediction result exceeds an alarm threshold value, and if so, sending out an abnormal alarm; the alarm threshold is constructed by using baseline.
In specific implementation, baseline modeling is used to expand the predicted data value into a threshold interval, so that monitoring can be conveniently performed according to the predicted data value, and the specific monitoring process comprises the following steps: in x minutes (once every minute, x is a manually set value, that is, abnormality occurs in consecutive x minutes), if the real-time data exceeds the upper limit/lower limit of the predicted threshold interval and the transaction number exceeds y (y is a manually set value, this step is to avoid occasional non-abnormal situations as much as possible), an alarm is given, and notification is given.
The machine learning has more application scenes in the my intelligent operation and maintenance system, and the machine learning also comprises monitoring of other indexes besides the transaction index monitoring, for example, in the capacity prediction of the server, the output of the network is the capacity change condition in a later period of time, so that the manager can conveniently judge whether the capacity of the server is required to be expanded or maintained.
At present, the method is carried on a transaction technical index analysis center of an intelligent operation and maintenance data analysis platform of the self-service and is used for monitoring key technical indexes such as transaction success rates, transaction average response times, transaction numbers, response rates and the like of a plurality of service channels in a line in real time, calculating the expected numerical value of the current index according to historical data of operation of the service system, giving a threshold value, and generating an alarm when abnormal changes occur to the index due to network fluctuation, server faults and the like and exceeding a safety range, and sending the alarm to corresponding management staff for investigation, so that the service interruption rate and the service interruption time can be effectively reduced. As shown in fig. 4 (a) and (B), two alarms generated when the average response time of the service exceeds the predicted alarm value, it can be seen that the predicted value generated by the prediction method used in the present invention is dynamic, and the target predicted value and the safety range are set differently at different time points. Meanwhile, at the front end of a system carrying the algorithm, the data fluctuation when the abnormality occurs can be intuitively seen, as shown in fig. 5 (the diagram is an alarm occurring at 9 am on 21 days of 2 months, and corresponds to the alarm screenshot displayed above). In fig. 5, a light blue region represents a normal range of the predicted value obtained by calculation, and a cyan curve is a real-time change of actual production data. When the real-time data exceeds the normal range at the time of 9 point 01, an alarm is sent out to inform a technician to analyze the health condition of the system, so that the problems of service interruption and the like are avoided.
Referring to fig. 6, a screen shot of the system carrying the algorithm for monitoring the same index for 21 to 27 days of 2023, 9 and a prediction interval using the method is shown. In the figure, a light blue region represents a prediction section obtained by calculation, and a cyan curve is a real-time change of actual production data. In the part marked by the red square frame, the data are fluctuated in the same time period of three continuous days, and the predicted interval is correspondingly fluctuated, so that the method can effectively learn the periodicity rule of the data; the purple square marked part generates abnormal fluctuation of real-time data, and the abnormal fluctuation exceeds the prediction range, is recognized as an abnormal point by a system, and is marked by a red dot.
Referring to fig. 7, a schematic structural diagram of a transaction technical index prediction device based on a neural network according to an embodiment of the invention is shown.
In this embodiment, the apparatus 60 includes:
a preprocessing unit 61, configured to obtain historical index data, and perform preprocessing including data division and data feature extraction on the historical index data to obtain a training data set, where the historical index data includes average response time, transaction success rate, and transaction number;
the training unit 62 is configured to input the training data set to a model constructed based on an LSTM network for training, obtain an index prediction model, and determine an upper baseline and a lower baseline in the obtained index prediction model;
and the prediction unit 63 is configured to input the index data to be predicted into the index prediction model to perform prediction, so as to obtain a prediction result of the future time period.
In another embodiment, the apparatus 60 further comprises:
the alarm unit is used for judging whether the prediction result exceeds an alarm threshold value, and if so, an abnormal alarm is sent out; the alarm threshold is constructed by using baseline.
The respective unit modules of the apparatus 60 may perform the corresponding steps in the above method embodiments, so that the detailed description of the respective unit modules is omitted herein.
The embodiment of the invention also provides a device, which comprises the transaction technical index prediction device based on the neural network, wherein the transaction technical index prediction device based on the neural network can adopt the structure of the embodiment of fig. 7, correspondingly, the technical scheme of the method embodiment shown in fig. 1 can be executed, the implementation principle and the technical effect are similar, and details can be referred to relevant records in the embodiment and are not repeated herein.
The apparatus comprises: a device with a photographing function such as a mobile phone, a digital camera or a tablet computer, or a device with an image processing function, or a device with an image display function. The device may include a memory, a processor, an input unit, a display unit, a power source, and the like.
The memory may be used to store software programs and modules, and the processor executes the software programs and modules stored in the memory to perform various functional applications and data processing. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (e.g., an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide access to the memory by the processor and the input unit.
The input unit may be used to receive input digital or character or image information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. Specifically, the input unit of the present embodiment may include a touch-sensitive surface (e.g., a touch display screen) and other input devices in addition to the camera.
The display unit may be used to display information entered by a user or provided to a user as well as various graphical user interfaces of the device, which may be composed of graphics, text, icons, video and any combination thereof. The display unit may include a display panel, and alternatively, the display panel may be configured in the form of an LCD (Liquid Crystal Display ), an OLED (Organic Light-Emitting Diode), or the like. Further, the touch-sensitive surface may overlay the display panel, and upon detection of a touch operation thereon or thereabout, the touch-sensitive surface is communicated to the processor to determine the type of touch event, and the processor then provides a corresponding visual output on the display panel based on the type of touch event.
The embodiment of the present invention also provides a computer readable storage medium, which may be a computer readable storage medium contained in the memory in the above embodiment; or may be a computer-readable storage medium, alone, that is not assembled into a device. The computer readable storage medium has stored therein at least one instruction that is loaded and executed by a processor to implement the neural network based transaction technical index prediction method shown in fig. 1. The computer readable storage medium may be a read-only memory, a magnetic disk or optical disk, etc.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described as different from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other. For the device embodiments, the apparatus embodiments and the storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.
Also, herein, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
While the foregoing description illustrates and describes the preferred embodiments of the present invention, it is to be understood that the invention is not limited to the forms disclosed herein, but is not to be construed as limited to other embodiments, but is capable of use in various other combinations, modifications and environments and is capable of changes or modifications within the scope of the inventive concept, either as described above or as a matter of skill or knowledge in the relevant art. And that modifications and variations which do not depart from the spirit and scope of the invention are intended to be within the scope of the appended claims.
Claims (10)
1. A transaction technical index prediction method based on a neural network, the method comprising:
acquiring historical index data, preprocessing the historical index data, including data division and data feature extraction, to obtain a training data set, wherein the historical index data comprises average response time, transaction success rate and transaction number;
inputting the training data set into a model constructed based on an LSTM network for training to obtain an index prediction model, and determining an upper baseline and a lower baseline in the obtained index prediction model;
and inputting the index data to be predicted into the index prediction model to predict, and obtaining a prediction result of a future time period.
2. The neural network-based transaction technical index prediction method according to claim 1, further comprising:
judging whether the prediction result exceeds an alarm threshold value, if so, sending out an abnormal alarm; the alarm threshold is constructed by using baseline.
3. The method for predicting transaction technical indexes based on neural network according to claim 1, wherein the preprocessing of the historical index data including data division and data feature extraction is performed to obtain a training data set, and the method comprises the following steps:
dividing the historical index data according to preset input length, output length and sliding window step length division to obtain a plurality of groups of input and output data pairs;
and extracting the data characteristics of each input and output data pair through an STL algorithm to obtain the training data set, wherein the data characteristics comprise seasonal characteristics, trend characteristics and residual characteristics.
4. A transaction technical index prediction method based on a neural network according to claim 3, wherein the STL algorithm includes an inner loop and an outer loop; the extracting the data characteristic of each input and output data pair through the STL algorithm comprises the following steps:
automatically deducing a corresponding seasonal period according to the index of the historical index data to obtain the period length;
converging the historical index data at different moments in one period to obtain a subsequence;
subtracting the previous trend feature obtained in the previous round from the historical index data in each internal cycle, and carrying out regression on each subsequence by using a LOESS algorithm to obtain a first result;
carrying out sliding average of preset length on the first result and carrying out regression on each subsequence through an LOESS algorithm to obtain a second result;
subtracting the second result from the first result to obtain the seasonal feature;
subtracting the seasonal features from the historical index data and carrying out regression on each subsequence through a LOESS algorithm to obtain the trend features;
and decomposing the seasonal features and the trend features based on the historical index data to obtain the residual features.
5. The method of claim 1, wherein the network architecture of the index prediction model comprises an input layer, two LSTM network layers and a full connection layer, wherein hidden layers of the two LSTM network layers including a lower LSTM are fully connected with a higher LSTM through a feed-forward connection.
6. The neural network-based transaction technical index prediction method according to claim 1, wherein the inputting the training dataset into a model constructed based on an LSTM network for training comprises:
model optimization is performed by using the average absolute error loss function and function activation is performed by using the softsign function.
7. The method for predicting transaction specifications based on a neural network according to claim 1, wherein the determining the upper and lower baselines comprises:
calculating according to the predicted previous day history index data and the width coefficient to obtain the upper and lower baselines; wherein,
the upper base line is upper (x) t )＝estimation(x t )+width*x magnitude (x t )/2，
Lower baseline is lower (x) t )＝estimation(x t )-width*x magnitude (x t ) And/2, wherein,
x magnitude ＝(mean(last day )+median(last day ))/2，last day represents the previous day history index data, and width represents the width factor.
8. A transaction technical index prediction device based on a neural network, the device comprising:
the preprocessing unit is used for acquiring historical index data, preprocessing the historical index data, including data division and data feature extraction, to obtain a training data set, wherein the historical index data comprises average response time, transaction success rate and transaction number;
the training unit is used for inputting the training data set into a model constructed based on an LSTM network for training to obtain an index prediction model, and determining an upper baseline and a lower baseline in the obtained index prediction model;
and the prediction unit is used for inputting the index data to be predicted into the index prediction model to predict, so as to obtain a prediction result of a future time period.
9. A neural network based transaction technical index prediction device, comprising a processor, a memory and a computer program stored in the memory, the computer program being executed by the processor to implement the steps of a neural network based transaction technical index prediction method according to any one of claims 1 to 7.
10. A computer readable storage medium having stored thereon a computer program to be executed by a processor to implement the steps of a neural network based transaction technical indicator prediction method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311336618.0A CN117455666A (en) | 2023-10-16 | 2023-10-16 | Transaction technical index prediction method, device and equipment based on neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311336618.0A CN117455666A (en) | 2023-10-16 | 2023-10-16 | Transaction technical index prediction method, device and equipment based on neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117455666A true CN117455666A (en) | 2024-01-26 |
Family
ID=89582654
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311336618.0A Pending CN117455666A (en) | 2023-10-16 | 2023-10-16 | Transaction technical index prediction method, device and equipment based on neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117455666A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118245350A (en) * | 2024-05-27 | 2024-06-25 | 深圳市天创伟业科技有限公司 | Solid-state storage health condition detection method and device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108197845A (en) * | 2018-02-28 | 2018-06-22 | 四川新网银行股份有限公司 | A kind of monitoring method of the transaction Indexes Abnormality based on deep learning model LSTM |
| CN109587713A (en) * | 2018-12-05 | 2019-04-05 | 广州数锐智能科技有限公司 | A kind of network index prediction technique, device and storage medium based on ARIMA model |
| WO2020143409A1 (en) * | 2019-01-07 | 2020-07-16 | 阿里巴巴集团控股有限公司 | Method and device for predicting business indicators |
| CN112115024A (en) * | 2020-09-03 | 2020-12-22 | 上海上讯信息技术股份有限公司 | Training method and device for fault prediction neural network model |
| WO2021212752A1 (en) * | 2020-04-23 | 2021-10-28 | 平安科技（深圳）有限公司 | Device index data-based anomaly detection method and apparatus, device, and storage medium |
| WO2021212756A1 (en) * | 2020-04-23 | 2021-10-28 | 平安科技（深圳）有限公司 | Index anomaly analysis method and apparatus, and electronic device and storage medium |
| CN114218462A (en) * | 2021-12-28 | 2022-03-22 | 天翼物联科技有限公司 | Data classification method, device, equipment and storage medium based on LSTM |
| CN115982412A (en) * | 2021-10-14 | 2023-04-18 | 北京航天长峰科技工业集团有限公司 | STL decomposition and LSTM network-based time sequence prediction method |
-
2023
- 2023-10-16 CN CN202311336618.0A patent/CN117455666A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108197845A (en) * | 2018-02-28 | 2018-06-22 | 四川新网银行股份有限公司 | A kind of monitoring method of the transaction Indexes Abnormality based on deep learning model LSTM |
| CN109587713A (en) * | 2018-12-05 | 2019-04-05 | 广州数锐智能科技有限公司 | A kind of network index prediction technique, device and storage medium based on ARIMA model |
| WO2020143409A1 (en) * | 2019-01-07 | 2020-07-16 | 阿里巴巴集团控股有限公司 | Method and device for predicting business indicators |
| WO2021212752A1 (en) * | 2020-04-23 | 2021-10-28 | 平安科技（深圳）有限公司 | Device index data-based anomaly detection method and apparatus, device, and storage medium |
| WO2021212756A1 (en) * | 2020-04-23 | 2021-10-28 | 平安科技（深圳）有限公司 | Index anomaly analysis method and apparatus, and electronic device and storage medium |
| CN112115024A (en) * | 2020-09-03 | 2020-12-22 | 上海上讯信息技术股份有限公司 | Training method and device for fault prediction neural network model |
| CN115982412A (en) * | 2021-10-14 | 2023-04-18 | 北京航天长峰科技工业集团有限公司 | STL decomposition and LSTM network-based time sequence prediction method |
| CN114218462A (en) * | 2021-12-28 | 2022-03-22 | 天翼物联科技有限公司 | Data classification method, device, equipment and storage medium based on LSTM |
Non-Patent Citations (1)
| Title |
|---|
| 田春华: "《大数据与智能+产教融合丛书 工业大数据分析算法实战》", 30 September 2022, 北京：机械工业出版社, pages: 234 - 239 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118245350A (en) * | 2024-05-27 | 2024-06-25 | 深圳市天创伟业科技有限公司 | Solid-state storage health condition detection method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10809704B2 (en) | Process performance issues and alarm notification using data analytics | |
| Lade et al. | Manufacturing analytics and industrial internet of things | |
| CN107622308B (en) | Power generation equipment parameter early warning method based on DBN (database-based network) | |
| CN107085415A (en) | Regular composer in process control network | |
| JP2018112903A (en) | Plant operation support apparatus, plant operation support method, plant operation support program, and recording medium | |
| CN117455666A (en) | Transaction technical index prediction method, device and equipment based on neural network | |
| US10921755B2 (en) | Method and system for competence monitoring and contiguous learning for control | |
| US20210065086A1 (en) | System and method for failure curve analytics | |
| Goel et al. | A data-driven alarm and event management framework | |
| Al-Dabbagh et al. | Toward the advancement of decision support tools for industrial facilities: Addressing operation metrics, visualization plots, and alarm floods | |
| Kumar et al. | Reliability prediction analysis of aspect-oriented application using soft computing techniques | |
| Liu et al. | Online conditional outlier detection in nonstationary time series | |
| US20220067990A1 (en) | Monitoring apparatus, monitoring method, and computer-readable medium having recorded thereon monitoring program | |
| CN115858291A (en) | System index detection method and device, electronic equipment and storage medium thereof | |
| CN113377595B (en) | Fault diagnosis method, device, electronic equipment and storage medium | |
| US11669082B2 (en) | Online fault localization in industrial processes without utilizing a dynamic system model | |
| Adesanwo et al. | Smart alarming for intelligent surveillance of Electrical Submersible Pump Systems | |
| Schnelle et al. | A real-time expert system for quality control | |
| JP2013182471A (en) | Load evaluation device for plant operation | |
| CN112510699A (en) | Transformer substation secondary equipment state analysis method and device based on big data | |
| CN111859798A (en) | Flow industrial fault diagnosis method based on bidirectional long-time and short-time neural network | |
| Onnes et al. | Bayesian network conflict detection for normative monitoring of black-box systems | |
| JP7506208B1 (en) | Information processing device, information processing method, and information processing program | |
| JP3040853B2 (en) | Plant monitoring system | |
| JP7452509B2 (en) | Monitoring devices, monitoring methods, and monitoring programs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |