CN114239949A - Website access amount prediction method and system based on two-stage attention mechanism - Google Patents
Website access amount prediction method and system based on two-stage attention mechanism Download PDFInfo
- Publication number
- CN114239949A CN114239949A CN202111507467.1A CN202111507467A CN114239949A CN 114239949 A CN114239949 A CN 114239949A CN 202111507467 A CN202111507467 A CN 202111507467A CN 114239949 A CN114239949 A CN 114239949A
- Authority
- CN
- China
- Prior art keywords
- data
- training
- attention mechanism
- website
- set data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 82
- 230000007246 mechanism Effects 0.000 title claims abstract description 80
- 238000012549 training Methods 0.000 claims abstract description 124
- 238000012360 testing method Methods 0.000 claims abstract description 41
- 238000012795 verification Methods 0.000 claims abstract description 37
- 238000007781 pre-processing Methods 0.000 claims abstract description 17
- 238000012545 processing Methods 0.000 claims description 26
- 238000010606 normalization Methods 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 15
- 238000013507 mapping Methods 0.000 claims description 14
- 230000015654 memory Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 11
- 238000010276 construction Methods 0.000 claims description 4
- 230000007774 longterm Effects 0.000 abstract description 10
- 230000006870 function Effects 0.000 description 17
- 230000000694 effects Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 11
- 238000013528 artificial neural network Methods 0.000 description 9
- 238000012986 modification Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 230000004083 survival effect Effects 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Development Economics (AREA)
- Game Theory and Decision Science (AREA)
- Probability & Statistics with Applications (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a website visit amount prediction method and a website visit amount prediction system based on a two-stage attention mechanism, which relate to the field of time sequence prediction, and the method comprises the following steps: obtaining original time sequence data of website access amount; preprocessing the original time sequence data, and generating training set data, verification set data and test set data; constructing a website access amount prediction model based on a two-stage attention mechanism; training the training set data, setting a hyper-parameter based on the verification set data, and obtaining a trained two-stage attention mechanism prediction model; and inputting the test set data into the dual-stage attention mechanism prediction model for training to obtain a second training result, wherein the second training result comprises a website visit amount prediction result of the original time series data. The technical problem that the traditional time series prediction method in the prior art is difficult to capture the long-term dependence input characteristics of the historical information of the website access amount is solved.
Description
Technical Field
The application relates to the field of time series prediction, in particular to a website access amount prediction method and system based on a two-stage attention mechanism.
Background
With the development of information technology, websites have indispensable tools for browsing information, shopping, entertainment and the like in daily life, the access amount of the website is important information that the website depends on survival, and the access amount directly influences the operation condition of a certain website, some common website access amount prediction methods are currently used for realizing website access amount data prediction at a future moment by researching the rules of historical information, and the common prediction methods include a linear time series prediction model, a nonlinear time series prediction model, a neural network time series prediction model, a Boosting prediction model, a GM prediction model and the like.
However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the inventors of the present application find that the above-mentioned technology has at least the following technical problems:
the technical problems that the traditional time sequence prediction method is poor in nonlinear fitting capability, low in accuracy of website visit amount prediction and difficult to capture long-term dependence of website visit amount historical information exist in the prior art.
Disclosure of Invention
The embodiment of the application provides a website visit amount prediction method and system based on a two-stage attention mechanism, and aims to solve the technical problems that in the prior art, a traditional time series prediction method is poor in nonlinear fitting capability, accuracy of website visit amount prediction is low, and long-term dependence of website visit amount historical information is difficult to capture.
In view of the foregoing problems, embodiments of the present application provide a method and a system for predicting website visitation amount based on a two-stage attention mechanism.
A first aspect of an embodiment of the present application provides a website visitation amount prediction method based on a two-stage attention mechanism, where the method is applied to a website visitation amount prediction system based on a two-stage attention mechanism, and the method includes: obtaining original time sequence data of website access amount; preprocessing the original time sequence data, and dividing the original time sequence data in proportion to generate training set data, verification set data and test set data; constructing a website access amount prediction model based on a two-stage attention mechanism; inputting the training set data into the website visitation amount prediction model for training, setting a hyper-parameter based on the verification set data, and obtaining a first training result, wherein the first training result comprises a two-stage attention mechanism prediction model after training is completed; inputting the test set data into the dual-stage attention mechanism prediction model for training to obtain a second training result, wherein the second training result comprises a website visitation amount prediction result of the original time series data.
A second aspect of an embodiment of the present application provides a website visitation amount prediction system based on a two-stage attention mechanism, where the system includes: a first obtaining unit: the first obtaining unit is used for obtaining original time series data of website access amount; a first generation unit: the first generation unit is used for preprocessing the original time series data and carrying out proportion division to generate training set data, verification set data and test set data; a first building unit: the first construction unit is used for constructing a website visit amount prediction model based on a two-stage attention mechanism; a first input unit: the first input unit is used for inputting the training set data into the website visit amount prediction model for training, setting a hyper-parameter based on the verification set data, and obtaining a first training result, wherein the first training result comprises a training-finished two-stage attention mechanism prediction model; a second input unit: the second input unit is used for inputting the test set data into the dual-stage attention mechanism prediction model for training to obtain a second training result, and the second training result comprises a website visit amount prediction result of the original time series data.
A third aspect of the embodiments of the present application provides a website visitation amount prediction system based on a two-stage attention mechanism, including: a processor coupled to a memory for storing a program that, when executed by the processor, causes a system to perform the steps of the method according to the first aspect.
One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:
the method comprises the steps of obtaining original time sequence data of website access amount; preprocessing the original time sequence data, and dividing the original time sequence data in proportion to generate training set data, verification set data and test set data; constructing a website access amount prediction model based on a two-stage attention mechanism; inputting the training set data into the website visitation amount prediction model for training, setting a hyper-parameter based on the verification set data, and obtaining a first training result, wherein the first training result comprises a two-stage attention mechanism prediction model after training is completed; inputting the test set data into the dual-stage attention mechanism prediction model for training to obtain a second training result, wherein the second training result comprises a website visitation amount prediction result of the original time series data. By using the two-stage attention mechanism prediction method, the technical effect of effectively improving the accuracy of website access amount prediction is achieved.
The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.
Drawings
Fig. 1 is a schematic flowchart of a website visitation amount prediction method based on a two-stage attention mechanism according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of a preprocessed data set for generating the original time-series data in a website visitation amount prediction method based on a two-stage attention mechanism according to an embodiment of the present disclosure;
fig. 3 is a schematic flowchart illustrating a process of generating the test set data in a website visitation amount prediction method based on a two-stage attention mechanism according to an embodiment of the present disclosure;
FIG. 4 is a schematic structural diagram of a system for predicting website visitation capacity based on a two-stage attention mechanism according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of an exemplary electronic device according to an embodiment of the present application.
Description of reference numerals: the device comprises a first obtaining unit 11, a first generating unit 12, a first constructing unit 13, a first input unit 14, a second input unit 15, an electronic device 300, a memory 301, a processor 302, a communication interface 303 and a bus architecture 304.
Detailed Description
The embodiment of the application provides a website visitation capacity prediction method and system based on a two-stage attention mechanism, and aims to solve the technical problems that in the prior art, a traditional time series prediction method is poor in nonlinear fitting capacity, accuracy of website visitation capacity prediction is low, and long-term dependence of website visitation capacity historical information is difficult to capture. By using the two-stage attention mechanism prediction method, the technical effect of effectively improving the accuracy of website access amount prediction is achieved.
Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are merely some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.
Summary of the application
With the development of information technology, websites have indispensable tools for browsing information, shopping, entertainment and the like in daily life, the access amount of the website is important information that the website depends on survival, and the access amount directly influences the operation condition of a certain website, some common website access amount prediction methods are currently used for realizing website access amount data prediction at a future moment by researching the rules of historical information, and the common prediction methods include a linear time series prediction model, a nonlinear time series prediction model, a neural network time series prediction model, a Boosting prediction model, a GM prediction model and the like. However, the technical problems that the traditional time series prediction method is poor in nonlinear fitting capability, low in accuracy of website visitation amount prediction and difficult to capture long-term dependence of website visitation amount historical information exist in the prior art.
In view of the above technical problems, the technical solution provided by the present application has the following general idea: obtaining original time sequence data of website access amount; preprocessing the original time sequence data, and dividing the original time sequence data in proportion to generate training set data, verification set data and test set data; constructing a website access amount prediction model based on a two-stage attention mechanism; inputting the training set data into the website visitation amount prediction model for training, setting a hyper-parameter based on the verification set data, and obtaining a first training result, wherein the first training result comprises a two-stage attention mechanism prediction model after training is completed; inputting the test set data into the dual-stage attention mechanism prediction model for training to obtain a second training result, wherein the second training result comprises a website visitation amount prediction result of the original time series data.
Having described the basic principles of the present application, the following embodiments will be described in detail and fully with reference to the accompanying drawings, it being understood that the embodiments described are only some embodiments of the present application, and not all embodiments of the present application, and that the present application is not limited to the exemplary embodiments described herein. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application. It should be further noted that, for the convenience of description, only some but not all of the elements relevant to the present application are shown in the drawings.
Example one
As shown in fig. 1, an embodiment of the present application provides a website visitation amount prediction method based on a dual-stage attention mechanism, where the method is applied to the website visitation amount prediction method based on the dual-stage attention mechanism, and the method includes:
step S100: obtaining original time sequence data of website access amount;
step S200: preprocessing the original time sequence data, and dividing the original time sequence data in proportion to generate training set data, verification set data and test set data;
specifically, the time series data is data collected in different times, where the data is used to describe a change situation of a phenomenon changing over time, the original time series data of the website visitation amount reflects an increase or decrease visitation amount of the website visitation amount over time, and the preprocessing of the original time series data refers to performing operations such as screening, cleaning, denoising, and the like on the original time series data, so as to remove bad data in the original time series data and reduce interference of spam data on subsequent analysis, and further according to different proportions of a total data set, a preprocessed data set is divided into a training set, a verification set, and a test set, where a sum of proportions of the training set, the verification set, and the test set is 1, for example, a proportion division method is as follows: the method comprises the steps that 80% of a total data set is used as a training set, 10% of the total data set is used as a test set, and 10% of the total data set is used as a verification set, specifically, training set data is used for training a model, the test set is used for obtaining generalization errors of a final model in a corresponding reality scene, the verification set is used for obtaining hyper-parameters in a neural network, the hyper-parameters refer to the number of layers of the neural network, the number of neurons of each layer of the neural network and some regularized parameters, and through preprocessing and dividing original time sequence data of website visitation, the technical effect of providing data support for a subsequent training website visitation prediction model is achieved.
Step S300: constructing a website access amount prediction model based on a two-stage attention mechanism;
specifically, the website visitation amount prediction model is constructed as a Long Short-Term Memory network (LSTM), the LSTM network is a time-cycle neural network and is specially designed for solving the Long-Term dependence problem existing in a general cycle neural network, the two-stage attention mechanism is to add an attention mechanism in both the input stage of an encoder and the output stage of a decoder, so as to achieve the purposes of selecting a characteristic factor and grasping the Long-Term time sequence dependence relationship, the attention mechanism is to automatically search hidden states most relevant to a current output target from hidden state lists of all moments of an input sequence in decoding, and then to use the hidden states and the output of the previous moment in decoding as the input of the decoding current moment, the website visit amount prediction model is constructed based on the two-stage attention mechanism, so that the technical effect of improving the prediction capability of the model is achieved.
Step S400: inputting the training set data into the website visitation amount prediction model for training, setting a hyper-parameter based on the verification set data, and obtaining a first training result, wherein the first training result comprises a two-stage attention mechanism prediction model after training is completed;
specifically, the training set data is input into the website visitation amount prediction model for training, the purpose of the method is to complete data fitting of the prediction model, inputting the training set data into the website visitation amount prediction model for training, and setting the hyper-parameters based on the verification set data means that, the purpose of improving the generalization capability of the neural network model is achieved by adjusting the hyper-parameters in the neural network through the verification set, the generalization ability refers to the adaptability of the neural network to unknown samples, and the specific method is based on the optimization methods such as random forest hyper-parameter optimization, Bayesian optimization, random search and the like, and the optimal value based on the objective function is obtained through training the verification set, the optimal hyper-parameter is obtained, the purpose of setting the hyper-parameter based on the verification set data is completed, and a first training result is further obtained, wherein the first training result comprises a two-stage attention mechanism prediction model after training is completed. And setting hyper-parameters through the verification set data to obtain a two-stage attention mechanism prediction model for completing the hyper-parameter setting, thereby achieving the technical effect of strengthening the generalization capability of the prediction model.
Step S500: inputting the test set data into the dual-stage attention mechanism prediction model for training to obtain a second training result, wherein the second training result comprises a website visitation amount prediction result of the original time series data.
Specifically, the two-stage attention mechanism prediction model is a prediction model for completing hyper-parameter setting, at this time, test set data is input into the prediction model for the purpose of evaluating the generalization capability of a final model, the test set data cannot be used as a basis related to algorithms such as parameter adjustment, feature selection and the like, and the test set data is only a small part of an original data set due to the fact that the test set data is used once and the data volume is large, training of the two-stage attention mechanism prediction model is completed through the test set data, and a website visit amount prediction result of original time series data is obtained. And completing the training of the two-stage attention mechanism prediction model through the test set, and achieving the technical effect of obtaining the website visit amount prediction result of the original time sequence data.
Further, as shown in fig. 2, step S200 in the method provided in the embodiment of the present application includes:
step S210: filling missing values in the original time sequence data based on a mean difference compensation method;
step S220: performing minimum-maximum normalization processing on the filled data set to generate a preprocessed data set of the original time series data;
specifically, the mean difference compensation method is a method for solving the problem that a data set has a lost value, and is implemented by calculating an average value of K bits of data before and after the lost data value when the data set has a data loss, and using the average value as data of the lost value, where K is generally 1 or 2, and performing min-max normalization processing on the filled data set by first finding a minimum value and a maximum value in the filled data set, and when there are a plurality of same minimum values or maximum values, taking only one of the minimum values or maximum values, and for a data set requiring normalization processing, the calculation method for normalization is as follows:
wherein, x is data needing normalization processing, and x*For normalized data, xmaxIs the maximum value, x, in the filled datasetminIs the minimum value of the filled data set. And filling the original time sequence data by a mean value difference filling method, and further performing minimum-maximum normalization processing on the filled data to obtain the technical effect of preprocessing the original time sequence data.
Further, as shown in fig. 3, step S200 in the method provided in the embodiment of the present application includes:
step S230: based on a first preset proportion value, carrying out proportion division on the preprocessed data set to generate training set data;
step S240: based on a second preset proportion value, carrying out proportion division on the preprocessed data set to generate verification set data;
step S250: and carrying out proportional division on the preprocessed data set based on a third preset proportional value to generate the test set data.
Specifically, the first preset proportion value is a proportion of a preset training set to the original time series data before the website visitation amount prediction model is trained, a specific value is 80%, the second preset proportion value is a proportion of a preset verification set to the original time series data before the website visitation amount prediction model is trained, a collective value is 10%, the third preset proportion value is a proportion of a preset test set to the original time series data before the website visitation amount prediction model is trained, a specific value is 10%, the preprocessed data set is subjected to proportion division through the first, second and third preset proportion values to further generate a training set, a verification set and a test set respectively, the training set is a data sample used for training the model and completing model fitting, the validation set is a sample set left alone during model training, which can be used to adjust the hyper-parameters of the model and to make a preliminary assessment of the model's capabilities. The method is generally used for verifying the generalization capability of the current model, including accuracy, recall rate and the like, when the model is iteratively trained so as to decide whether to stop continuing training. The test set is used for evaluating the generalization ability of the model final model, and the preprocessing data set is subjected to proportion division through the first preset proportion value, the second preset proportion value and the third preset proportion value, so that the technical effects of generating the training set, the verification set and the test set and providing data support for training the website visit amount prediction model are further achieved.
Further, step S400 in the method provided in the embodiment of the present application includes:
step S410: constructing an input sequence X based on a first LSTM network as a mapping functiontTo encoder hidden layer state htThe mapping relationship of (2);
step S420: calculating X of the kth driving sequence according to the mapping relationkGenerating a first calculation result;
step S430: and carrying out normalization processing on the first calculation result.
In particular, the first LSTM network refers to a first attention mechanism based LSTM network used in the encoding stage, and the first LSTM network as a mapping function refers to using LSTM as a function between the input and other elements, i.e. establishing the input sequence X by LSTMtTo encoder hidden layer state htThe LSTM function is better able to capture long term dependencies in the time series. The specific mapping relation is ht＝f1(ht-1，Xt) Calculating X of the k-th driving sequence according to the mapping relationkThe specific method for calculating the weight is to pass the hidden layer state h of the LSTM unit at the moment t-1t-1Cell state st-1And the kth drive sequence
To calculate XkFurther generating a first calculation result, and performing normalization processing on the first calculation result, wherein the normalization processing function is a softmax function, and an input sequence X is constructedtTo encoder hidden layer state htTo obtain X of the k-th driving sequencekThe technical effect of coding is further achieved.
Further, the method provided by the embodiment of the present application further includes:
step S431: according to the formula:
Specifically, the softmax function is also called as a normalization index function, and aims to display the result of multiple classifications in a probability form, the first step of softmax is to convert the prediction result of the model into the index function, so that the nonnegativity of the probability is ensured, further only the converted result needs to be normalized, and the normalization processing method is to divide the converted result by the sum of all converted results, so that the converted result can be understood as the percentage of the converted result in the total number, and further the approximate probability can be obtained. The calculation formula of the softmax function is as follows:
wherein,
for the parameter to be learned, the sum of all attention weights processed by the softmax function is 1. Calculating attention weight at t moment and weighting the attention weight to the input sequence to obtain new input
The formula is as follows:
according to
And ht-1And completing LSTM mapping to obtain a hidden layer state ht at the time t, wherein the formula is as follows:
the technical effect of performing normalization processing on the first calculation result is achieved based on the softmax function.
Further, the method provided by the embodiment of the present application further includes:
step S632: decoding by a decoder based on a second LSTM network to obtain a target sequence output
Specifically, the second LSTM network refers to a second attention mechanism-based LSTM network used in the encoding stage to obtain the target sequence output
The specific method comprises the following steps: using encoder hidden layer states hiDecoder hidden layer state at time t-1
And memory cell state
Calculating hidden layer state weight of encoder at t moment
The formula is as follows:
wherein,
the temporal attention mechanism can automatically select the encoder hidden layer state with larger relevance for the parameter to be learned. Computing a context vector c using attention weights of encoder hidden layerstEach time instant has a corresponding context vector, and the formula is as follows:
using the true value y of y at time t-1t-1And a context vector ct-1Calculate out new
Wherein
Will be provided with
Mapped to the input dimensions of the decoder, the formula is as follows:
according to
Using LSTM as the mapping function f2Computing hidden layer state d of a decodert，
And
will [ d ]T；cT]Converting into hidden layer dimension of decoder, calculating predicted value of target sequence of website access amount at current time
The formula is as follows:
based on the second LSTM network, the target sequence output is obtained through further decoding by a decoder
The technical effect of completing the decoding work.
Compared with the prior art, the method has the following beneficial effects:
1. the method comprises the steps of obtaining original time sequence data of website access amount; preprocessing the original time sequence data, and dividing the original time sequence data in proportion to generate training set data, verification set data and test set data; constructing a website access amount prediction model based on a two-stage attention mechanism; inputting the training set data into the website visitation amount prediction model for training, setting a hyper-parameter based on the verification set data, and obtaining a first training result, wherein the first training result comprises a two-stage attention mechanism prediction model after training is completed; inputting the test set data into the dual-stage attention mechanism prediction model for training to obtain a second training result, wherein the second training result comprises a website visitation amount prediction result of the original time series data. The method solves the technical problems that the traditional time sequence prediction method is poor in nonlinear fitting capability, low in accuracy of website access amount prediction and difficult to capture long-term dependence of historical website access amount information in the prior art. By using the two-stage attention mechanism prediction method, the technical effect of effectively improving the accuracy of website access amount prediction is achieved.
2. And filling the original time sequence data by a mean value difference filling method, and further performing minimum-maximum normalization processing on the filled data to obtain the technical effect of preprocessing the original time sequence data.
3. Through the first, second and third preset proportion values, the preprocessing data set is subjected to proportion division, and the technical effects of generating the training set, the verification set and the test set and providing data support for training the website visit amount prediction model are further achieved.
Example two
Based on the same inventive concept as the method for predicting website visitation amount based on the dual-stage attention mechanism in the foregoing embodiment, the present application further provides a system for predicting website visitation amount based on the dual-stage attention mechanism, please refer to fig. 5, where the system includes:
first obtaining unit 11: the first obtaining unit 11 is configured to obtain original time series data of website visitation amounts;
the first generation unit 12: the first generating unit 12 is configured to preprocess the original time series data, and perform proportional division to generate training set data, verification set data, and test set data;
the first building element 13: the first construction unit 13 is configured to construct a website visitation amount prediction model based on a two-stage attention mechanism;
first input unit 14: the first input unit 14 is configured to input the training set data to the website visitation amount prediction model for training, set a hyper-parameter based on the validation set data, and obtain a first training result, where the first training result includes a two-stage attention mechanism prediction model after training is completed;
second input unit 15: the second input unit 15 is configured to input the test set data into the two-stage attention mechanism prediction model for training, and obtain a second training result, where the second training result includes a website visitation amount prediction result for the original time-series data.
Further, the system further comprises:
a first filling unit: the first filling unit is used for filling missing values in the original time series data based on a mean difference compensation method;
a second generation unit: the second generating unit is used for performing minimum-maximum normalization processing on the filled data set to generate a preprocessed data set of the original time series data.
Further, the system further comprises:
a third generation unit: the third generation unit is used for carrying out proportional division on the preprocessed data set based on a first preset proportional value to generate the training set data;
a fourth generation unit: the fourth generating unit is used for carrying out proportion division on the preprocessed data set based on a second preset proportion value to generate the verification set data;
a fifth generation unit: the fifth generating unit is configured to perform proportional division on the preprocessed data set based on a third preset proportional value, and generate the test set data.
Further, the system further comprises:
a second building element: the second construction unit is used for constructing an input sequence X based on the first LSTM network as a mapping functiontTo encoder hidden layer state htThe mapping relationship of (2);
a sixth generation unit: the sixth generating unit is used for calculating X of the kth driving sequence based on the mapping relationkGenerating a first calculation result;
a first processing unit: the first processing unit is used for carrying out normalization processing on the first calculation result.
Further, the system further comprises:
a first processing unit: the first processing unit is configured to:
carrying out normalization processing on the first calculation result; wherein,
are parameters to be learned.
Further, the system further comprises:
a first decoding unit: the first decoding unit is used for decoding the decoder based on the second LSTM network to obtain the target sequence output
In the present specification, each embodiment is described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the foregoing method for predicting website visitation amount based on the two-stage attention mechanism in the first embodiment of fig. 1 and the specific example are also applicable to the system for predicting website visitation amount based on the two-stage attention mechanism in this embodiment. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Exemplary electronic device
The electronic device of the embodiment of the present application is described below with reference to fig. 5.
Fig. 5 illustrates a schematic structural diagram of an electronic device according to an embodiment of the present application.
Based on the inventive concept of the website visitation amount prediction method based on the two-stage attention mechanism in the foregoing embodiments, the present application also provides a website visitation amount prediction method system based on the two-stage attention mechanism, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps of any one of the foregoing website visitation amount prediction methods based on the two-stage attention mechanism.
Where in fig. 5 a bus architecture (represented by bus 300), bus 300 may include any number of interconnected buses and bridges, bus 300 linking together various circuits including one or more processors, represented by processor 302, and memory, represented by memory 304. The bus 300 may also link together various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface 305 provides an interface between the bus 300 and the receiver 301 and transmitter 303. The receiver 301 and the transmitter 303 may be the same element, i.e., a transceiver, providing a means for communicating with various other apparatus over a transmission medium. The processor 302 is responsible for managing the bus 300 and general processing, and the memory 304 may be used for storing data used by the processor 302 in performing operations.
The embodiment of the application provides a website visit amount prediction method based on a dual-stage attention mechanism, wherein the method is applied to a website visit amount prediction system based on the dual-stage attention mechanism, and the method comprises the following steps: preprocessing the original time sequence data, and dividing the original time sequence data in proportion to generate training set data, verification set data and test set data; constructing a website access amount prediction model based on a two-stage attention mechanism; inputting the training set data into the website visitation amount prediction model for training, setting a hyper-parameter based on the verification set data, and obtaining a first training result, wherein the first training result comprises a two-stage attention mechanism prediction model after training is completed; inputting the test set data into the dual-stage attention mechanism prediction model for training to obtain a second training result, wherein the second training result comprises a website visitation amount prediction result of the original time series data. The method solves the technical problems that the traditional time sequence prediction method is poor in nonlinear fitting capability, low in accuracy of website access amount prediction and difficult to capture long-term dependence of historical website access amount information in the prior art. By using the two-stage attention mechanism prediction method, the technical effect of effectively improving the accuracy of website access amount prediction is achieved.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, the present application may take the form of an entirely software embodiment, an entirely hardware embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application is in the form of a computer program product that may be embodied on one or more computer-usable storage media having computer-usable program code embodied therewith. And such computer-usable storage media include, but are not limited to: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk Memory, a Compact Disc Read-Only Memory (CD-ROM), and an optical Memory.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a system for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (8)
1. A website visit amount prediction method based on a two-stage attention mechanism is applied to a website visit amount prediction system based on the two-stage attention mechanism, and comprises the following steps:
obtaining original time sequence data of website access amount;
preprocessing the original time sequence data, and dividing the original time sequence data in proportion to generate training set data, verification set data and test set data;
constructing a website access amount prediction model based on a two-stage attention mechanism;
inputting the training set data into the website visitation amount prediction model for training, setting a hyper-parameter based on the verification set data, and obtaining a first training result, wherein the first training result comprises a two-stage attention mechanism prediction model after training is completed;
inputting the test set data into the dual-stage attention mechanism prediction model for training to obtain a second training result, wherein the second training result comprises a website visitation amount prediction result of the original time series data.
2. The method of claim 1, wherein the pre-processing the raw time-series data comprises:
filling missing values in the original time sequence data based on a mean difference compensation method;
and performing minimum-maximum normalization processing on the filled data set to generate a preprocessed data set of the original time series data.
3. The method of claim 1, wherein the scaling comprises:
based on a first preset proportion value, carrying out proportion division on the preprocessed data set to generate training set data;
based on a second preset proportion value, carrying out proportion division on the preprocessed data set to generate verification set data;
and carrying out proportional division on the preprocessed data set based on a third preset proportional value to generate the test set data.
4. The method of claim 1, wherein the first training result comprises a trained two-stage attention mechanism prediction model comprising:
constructing an input sequence X based on a first LSTM network as a mapping functiontTo encoder hidden layer state htThe mapping relationship of (2);
calculating X of the kth driving sequence according to the mapping relationkGenerating a first calculation result;
and carrying out normalization processing on the first calculation result.
5. The method of claim 4, wherein the normalizing the first calculation result comprises:
according to the formula:
carrying out normalization processing on the first calculation result;
wherein,
are parameters to be learned.
6. The method of claim 5, wherein the method comprises:
decoding by a decoder based on a second LSTM network to obtain a target sequence output
7. A website visitation amount prediction system based on a two-stage attention mechanism, wherein the system comprises:
a first obtaining unit: the first obtaining unit is used for obtaining original time series data of website access amount;
a first generation unit: the first generation unit is used for preprocessing the original time series data and carrying out proportion division to generate training set data, verification set data and test set data;
a first building unit: the first construction unit is used for constructing a website visit amount prediction model based on a two-stage attention mechanism;
a first input unit: the first input unit is used for inputting the training set data into the website visit amount prediction model for training, setting a hyper-parameter based on the verification set data, and obtaining a first training result, wherein the first training result comprises a training-finished two-stage attention mechanism prediction model;
a second input unit: the second input unit is used for inputting the test set data into the dual-stage attention mechanism prediction model for training to obtain a second training result, and the second training result comprises a website visit amount prediction result of the original time series data.
8. A system for predicting website visitation capacity based on a two-stage attention mechanism, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1-6 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111507467.1A CN114239949A (en) | 2021-12-10 | 2021-12-10 | Website access amount prediction method and system based on two-stage attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111507467.1A CN114239949A (en) | 2021-12-10 | 2021-12-10 | Website access amount prediction method and system based on two-stage attention mechanism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114239949A true CN114239949A (en) | 2022-03-25 |
Family
ID=80754705
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111507467.1A Pending CN114239949A (en) | 2021-12-10 | 2021-12-10 | Website access amount prediction method and system based on two-stage attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114239949A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116502959A (en) * | 2023-06-21 | 2023-07-28 | 南京航空航天大学 | Product manufacturing quality prediction method based on meta learning |
| CN117370272A (en) * | 2023-10-25 | 2024-01-09 | 浙江星汉信息技术股份有限公司 | File management method, device, equipment and storage medium based on file heat |
-
2021
- 2021-12-10 CN CN202111507467.1A patent/CN114239949A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116502959A (en) * | 2023-06-21 | 2023-07-28 | 南京航空航天大学 | Product manufacturing quality prediction method based on meta learning |
| CN116502959B (en) * | 2023-06-21 | 2023-09-08 | 南京航空航天大学 | Product manufacturing quality prediction method based on meta learning |
| CN117370272A (en) * | 2023-10-25 | 2024-01-09 | 浙江星汉信息技术股份有限公司 | File management method, device, equipment and storage medium based on file heat |
| CN117370272B (en) * | 2023-10-25 | 2024-06-11 | 浙江星汉信息技术股份有限公司 | File management method, device, equipment and storage medium based on file heat |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112784092B (en) | Cross-modal image text retrieval method of hybrid fusion model | |
| CN111797122B (en) | Method and device for predicting change trend of high-dimensional reappearance concept drift stream data | |
| US11650968B2 (en) | Systems and methods for predictive early stopping in neural network training | |
| CN109902222A (en) | Recommendation method and device | |
| WO2020190543A1 (en) | Differential bit width neural architecture search | |
| US20220215298A1 (en) | Method for training sequence mining model, method for processing sequence data, and device | |
| CN114239949A (en) | Website access amount prediction method and system based on two-stage attention mechanism | |
| WO2021004324A1 (en) | Resource data processing method and apparatus, and computer device and storage medium | |
| CN108876044B (en) | Online content popularity prediction method based on knowledge-enhanced neural network | |
| WO2020190526A1 (en) | Mixed precision training of an artificial neural network | |
| CN110264270A (en) | A kind of behavior prediction method, apparatus, equipment and storage medium | |
| CN113139570A (en) | Dam safety monitoring data completion method based on optimal hybrid valuation | |
| CN111428181A (en) | Bank financing product recommendation method based on generalized additive model and matrix decomposition | |
| CN114936701A (en) | Real-time monitoring method and device for comprehensive energy consumption and terminal equipment | |
| CN113656707B (en) | Financial product recommendation method, system, storage medium and equipment | |
| CN114880538A (en) | Attribute graph community detection method based on self-supervision | |
| CN113506175A (en) | Method, device, equipment and storage medium for optimizing risk early warning model of medium and small enterprises | |
| CN112561050B (en) | Neural network model training method and device | |
| CN114418158A (en) | Cell network load index prediction method based on attention mechanism learning network | |
| CN110728292A (en) | Self-adaptive feature selection algorithm under multi-task joint optimization | |
| CN115905546A (en) | Graph convolution network document identification device and method based on resistive random access memory | |
| CN117348837A (en) | Quantization method and device for floating point precision model, electronic equipment and storage medium | |
| CN113723593A (en) | Load shedding prediction method and system based on neural network | |
| CN115836298A (en) | Automatic selection and filter removal optimization of quantization under energy constraints | |
| CN111626472A (en) | Scene trend judgment index computing system and method based on deep hybrid cloud model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |