CN116881854B - XGBoost-fused time sequence prediction method for calculating feature weights - Google Patents

XGBoost-fused time sequence prediction method for calculating feature weights Download PDF

Info

Publication number
CN116881854B
CN116881854B CN202311156329.2A CN202311156329A CN116881854B CN 116881854 B CN116881854 B CN 116881854B CN 202311156329 A CN202311156329 A CN 202311156329A CN 116881854 B CN116881854 B CN 116881854B
Authority
CN
China
Prior art keywords
time sequence
xgboost
feature
prediction model
characteristic information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311156329.2A
Other languages
Chinese (zh)
Other versions
CN116881854A (en
Inventor
郭宇红
马海森
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Relations, University of
Original Assignee
International Relations, University of
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Relations, University of filed Critical International Relations, University of
Priority to CN202311156329.2A priority Critical patent/CN116881854B/en
Publication of CN116881854A publication Critical patent/CN116881854A/en
Application granted granted Critical
Publication of CN116881854B publication Critical patent/CN116881854B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/24323—Tree-organised classifiers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2123/00—Data types
    • G06F2123/02—Data types in the time domain, e.g. time-series data
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

The invention provides a time sequence prediction method for calculating feature weights by fusing XGBoost, which belongs to the technical field of data processing, and comprises the following steps: acquiring a plurality of types of characteristic information related to the predicted time sequence variable from a database; according to the time sequence variable, the characteristic information and the XGBoost model, learning to obtain each decision tree; determining the weight of each characteristic information according to the number of nodes in each decision tree and the splitting gain of the nodes; based on the time sequence, the feature information and the feature weight, training the time sequence prediction model fused with XGBoost to calculate the feature weight to obtain a trained time sequence prediction model, solving the problem of low accuracy of time sequence prediction caused by the fact that the features are included in the analysis by the same weight or the correlation coefficient or the rough weight formed by the common attention mechanism in the prior art, and improving the accuracy of time sequence prediction in the related field.

Description

XGBoost-fused time sequence prediction method for calculating feature weights
Technical Field
The invention relates to the technical field of data processing, in particular to a time sequence prediction method for calculating feature weights by fusing XGBoost.
Background
There are many time-series variables in real life that can be predicted by arithmetic processing of some existing data. The trend of the evolution of the related time sequence can be mastered in time through the prediction of the time sequence, the risk is avoided, and effective support information is provided for effective decision.
In the related art, there may be a plurality of influencing factors of the time series, for example, in the prediction process of the traffic field, there may be a plurality of influencing factors of the peak of the traffic flow, so how to predict the time series variable-the traffic flow based on the plurality of influencing factors is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
Aiming at the problems in the prior art, the embodiment of the invention provides a time sequence prediction method for calculating feature weights by fusing XGBoost.
Specifically, the embodiment of the invention provides the following technical scheme:
in a first aspect, an embodiment of the present invention provides a training method for a time-series prediction model that fuses feature weights calculated by a gradient lifting model XGBoost, including:
acquiring a plurality of types of characteristic information related to the predicted time sequence variable from a database;
according to the characteristic information of a plurality of types and the gradient lifting model XGBoost, each decision tree is learned; the decision tree is used for determining the influence degree of the characteristic information of each type on the predicted time sequence variable;
Determining the weight of each characteristic information according to the number of nodes in each decision tree and the splitting gain of the nodes;
training the time sequence prediction model fused with the XGBoost calculation feature weights according to the feature information of the plurality of types and the weights of the feature information of the plurality of types to obtain a trained time sequence prediction model fused with the XGBoost calculation feature weights; the time sequence prediction model fused with XGBoost to calculate the feature weight is constructed based on a cyclic neural network model GRU; the XGBoost computing feature weight fusion time series prediction model is used for predicting the expected result of the time series variable.
Further, determining the weight of each feature information according to the number of nodes in each decision tree and the splitting gain of the nodes, including:
the weight of each feature information is determined based on the following formula:
；
wherein,representing characteristic information +.>Weights of (2); />Representing characteristic information +.>The number of nodes of the corresponding decision tree; t represents the number of all decision trees; n (t) represents the number of non-leaf nodes of the t-th tree; />A partition feature representing an ith non-leaf node of the nth tree; i () represents an indication function; />Representing the sum of the second derivatives of all samples falling on the ith non-leaf node of the nth tree; n represents the number of features.
Further, determining whether the decision tree stops growing based on the target rule; the target rule includes at least one of:
the target gain before and after splitting of the decision tree nodes is smaller than or equal to a first threshold value;
the number of samples contained in the decision tree node is less than or equal to a second threshold;
the number of splitting layers of the decision tree reaches a third threshold.
Further, optimizing preset parameters of the time sequence prediction model fused with the XGBoost calculation feature weights based on a whale optimization algorithm WOA to obtain the optimized time sequence prediction model fused with the XGBoost calculation feature weights.
Further, optimizing preset parameters of the time sequence prediction model fused with XGBoost to calculate the feature weight based on a whale optimization algorithm WOA to obtain an optimized time sequence prediction model fused with XGBoost to calculate the feature weight, and the method comprises the following steps:
and optimizing the number of neurons and the training iteration times of the time sequence prediction model fused with the XGBoost to calculate the feature weights based on the WOA algorithm to obtain the optimized time sequence prediction model fused with the XGBoost to calculate the feature weights.
In a second aspect, the embodiment of the present invention further provides a method for predicting a time sequence of feature weights calculated by fusing a gradient lifting model XGBoost, including:
Acquiring a plurality of types of characteristic information related to the predicted time sequence and weights of the characteristic information from a database; the weight of each characteristic information is used for representing the influence degree of each type of characteristic information on the predicted time sequence;
inputting a plurality of types of feature information related to the predicted time sequence and the weight of each feature information into a trained fusion XGBoost computing feature weight time sequence prediction model to obtain a time sequence prediction result; the time sequence prediction model fused with the XGBoost to calculate the feature weight is obtained by training based on the training method of the time sequence prediction model fused with the gradient lifting model XGBoost to calculate the feature weight according to the first aspect.
In a third aspect, an embodiment of the present invention further provides a training device for a time-series prediction model that fuses a feature weight calculated by a gradient lifting model XGBoost, including:
an acquisition module for acquiring a plurality of types of characteristic information related to the predicted time series variable from a database;
the determining module is used for learning to obtain each decision tree according to the characteristic information of a plurality of types and the gradient lifting model XGBoost; the decision tree is used for determining the influence degree of the characteristic information of each type on the predicted time sequence variable;
Determining the weight of each characteristic information according to the number of nodes in each decision tree and the splitting gain of the nodes;
the training module is used for training the time sequence prediction model fused with the XGBoost to calculate the characteristic weight according to the characteristic information of a plurality of types and the weight of the characteristic information of a plurality of types to obtain a trained time sequence prediction model fused with the XGBoost to calculate the characteristic weight; the time sequence prediction model fused with XGBoost to calculate the feature weight is constructed based on a cyclic neural network model GRU; the XGBoost computing feature weight fusion time series prediction model is used for predicting the expected result of the time series variable.
In a fourth aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements a training method of a time-series prediction model for computing feature weights by using a fusion gradient lifting model XGBoost according to the first aspect or a time-series prediction method for computing feature weights by using a fusion gradient lifting model XGBoost according to the second aspect when the processor executes the program.
In a fifth aspect, an embodiment of the present invention further provides a non-transitory computer readable storage medium, on which a computer program is stored, where the computer program when executed by a processor implements a training method of a time-series prediction model for computing feature weights by fusing a gradient lifting model XGBoost according to the first aspect or a time-series prediction method for computing feature weights by fusing a gradient lifting model XGBoost according to the second aspect.
In a sixth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, where the computer program when executed by a processor implements a training method of a time-series prediction model for computing feature weights by fusing a gradient lifting model XGBoost according to the first aspect or a time-series prediction method for computing feature weights by fusing a gradient lifting model XGBoost according to the second aspect.
According to the embodiment of the invention, the decision tree corresponding to each type of characteristic information is obtained by acquiring the plurality of types of characteristic information related to the predicted time sequence from the database and according to the plurality of types of characteristic information and the gradient lifting model related to the predicted time sequence, and the weight of each characteristic information is determined according to the number of nodes in each decision tree, so that the influence degree of each type of characteristic information on the predicted time sequence can be rapidly, effectively and accurately determined; further, according to the multiple types of feature information related to the predicted time sequence and the weights of the multiple types of feature information, the time sequence prediction model fused with the XGBoost computing feature weights is trained, so that the trained time sequence prediction model fused with the XGBoost computing feature weights can comprehensively and accurately capture important feature information related to the predicted time sequence, the problem that important information cannot be fully learned when the features are more is solved, and the trained time sequence prediction model fused with the XGBoost computing feature weights can more accurately conduct time sequence prediction.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a training method of a time series prediction model fused with XGBoost calculation feature weights according to an embodiment of the present invention;
FIG. 2 is a second flow chart of a training method of a time series prediction model for fusing XGBoost calculation feature weights according to the embodiment of the present invention;
FIG. 3 is a third flow chart of a training method of a time series prediction model for fusing XGBoost calculation feature weights according to the embodiment of the present invention;
FIG. 4 is a schematic diagram of a training device for fusing XGBoost computing feature weights in a time series prediction model according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The method of the embodiment of the invention can be applied to a time sequence prediction scene to realize accurate prediction of a time sequence result.
In the related art, there may be a plurality of influencing factors of the time series to be predicted, for example, in the prediction process of the traffic field, there may be a plurality of influencing factors influencing the peak of the traffic flow, and how to predict the time series based on the plurality of influencing factors is a technical problem that needs to be solved by those skilled in the art.
According to the training method of the time sequence prediction model fused with XGBoost computing feature weights, disclosed by the embodiment of the invention, a plurality of types of feature information related to a predicted time sequence are obtained from a database, decision trees corresponding to the feature information of each type are obtained according to the feature information of the plurality of types and the gradient lifting model related to the predicted time sequence, and the weight of each feature information is determined according to the number of nodes in each decision tree, so that the influence degree of the feature information of each type on the predicted time sequence can be rapidly, effectively and accurately determined; further, according to the feature information of a plurality of types and the weights of the feature information of a plurality of types related to the predicted time sequence, the time sequence prediction model fused with the XGBoost computing feature weights is trained, so that the trained time sequence prediction model fused with the XGBoost computing feature weights can comprehensively and accurately grasp the feature information related to the predicted time sequence, the problem that important information cannot be fully learned when the features are more is solved, and the trained time sequence prediction model fused with the XGBoost computing feature weights can more accurately predict the time sequence.
The following describes the technical scheme of the present invention in detail with reference to fig. 1 to 5. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.
FIG. 1 is a flowchart of an embodiment of a training method for merging XGBoost computing feature weights in a time series prediction model according to an embodiment of the present invention. As shown in fig. 1, the method provided in this embodiment includes:
step 101, obtaining a plurality of types of characteristic information related to predicted time sequence variables from a database;
in particular, there are many time-series variables in real life, which can be predicted by an arithmetic process on some existing data. The time sequence occurrence trend can be mastered in time through the time sequence prediction, risks are avoided, and effective support information is provided for effective decision.
In the related art, there may be a plurality of influencing factors of the time series to be predicted, for example, there may be a plurality of influencing factors of the weather change in the prediction process of the weather change; in the prediction process of the amount of the tourist, a plurality of factors influencing the amount of the tourist can exist; in the process of predicting the traffic flow, a plurality of factors affecting the traffic flow may exist; in the course of stock index prediction, there may be a plurality of factors affecting the stock index. How to accurately predict each time series based on a plurality of influencing factors is a technical problem that needs to be solved by those skilled in the art.
In order to solve the above-mentioned problem, in an embodiment of the present invention, first, a plurality of types of feature information related to a predicted time series are acquired from a database; alternatively, the method provided by the invention can be applied to various time series predictions; the predicted time series variable may be, for example, a highest weather temperature, a peak guest volume, a peak traffic volume, a stock quote, etc. Alternatively, the plurality of types of feature information related to the predicted time series may be feature information directly or indirectly related to a predicted result of the predicted time series; for example, when the weather maximum temperature is predicted the next day, the characteristic information related to the weather temperature includes a plurality of factors such as cloud cover, visibility, barometric pressure, air temperature, humidity, wind direction, wind speed, evaporation amount, and the like; in predicting the peak of the vehicle flow rate, the characteristic information related to the vehicle flow rate includes: traffic conditions, traffic run time, traffic control conditions, weather factors, etc.; in making a prediction of a stock closing price, the characteristic information related to the stock closing price may include: open price, maximum price, amount of transactions, etc.
Step 102, learning to obtain each decision tree according to the characteristic information of a plurality of types and the gradient lifting model XGBoost; the decision tree is used for determining the influence degree of the characteristic information of each type on the predicted time sequence variable;
Specifically, after a plurality of types of feature information related to the predicted time sequence are obtained from the database, a decision tree corresponding to each type of feature information can be obtained according to the plurality of types of feature information related to the predicted time sequence and the gradient lifting model; alternatively, a plurality of types of feature information related to the predicted time series and corresponding time series variable values may be input as samples to the gradient lifting model, thereby generating a decision tree for determining the degree of influence of each type of feature information on the predicted time series; optionally, the gradient lifting model may be an XGBoost model, n types of feature information related to the predicted time sequence are formed through a Boosting algorithm of gradient lifting, n classifiers are formed based on a Boosting algorithm of gradient lifting, and each classifier (decision tree) counts the number of own nodes to determine the influence of each feature on the predicted tag, so that the influence degree of each type of feature information on the predicted time sequence can be rapidly, effectively and accurately determined based on the decision tree, and the effect of accurately determining the influence degree of each type of feature information on the predicted time sequence based on the gradient lifting model is achieved.
For example, traffic flow data over a period of time is selected and n characteristic information associated with the traffic flow data, such as traffic characteristics, time characteristics, traffic control characteristics, weather characteristics, are determined.
Then, the n-dimensional feature sequence data is normalized, and each feature in the n-dimensional feature sequence is ordered so that the segmentation points can be rapidly selected in the training of the gradient lifting model based on residual errors.
And then, inputting the ordered sequences corresponding to each type of characteristic information, forming n classifiers by a Boosting algorithm based on gradient lifting, and counting the number of nodes of each classifier, so that the influence of each characteristic on the peak value of the traffic flow of the predictive label can be rapidly, effectively and accurately determined.
Optionally, the method of the present application may also be used for predicting the closing price of a stock, and the specific process is as follows:
stock data within a period of time is selected, and n pieces of characteristic information, such as a starting price, a highest price and a trading volume, related to the stock data are determined.
And then, normalizing the n-dimensional feature sequence data, and sequencing each feature in the n-dimensional feature sequence so as to quickly candidate the segmentation points in the training of the gradient lifting model based on the residual error.
And then, inputting the ordered sequences corresponding to each type of characteristic information, forming n classifiers by a Boosting algorithm based on gradient promotion, and counting the number of nodes of each classifier, so that the influence of each characteristic on the predicted tag receiving price can be rapidly, effectively and accurately determined.
Step 103, determining the weight of each feature information according to the number of nodes in each decision tree and the splitting gain of the nodes;
specifically, after decision trees corresponding to the feature information of each type are obtained according to the feature information of a plurality of types and the gradient lifting model related to the predicted time sequence, the weight of each feature information corresponding to each decision tree can be determined according to the number of nodes in each decision tree; optionally, in the decision tree corresponding to the feature information, the node corresponds to a division of a feature, and the number of nodes corresponds to the number of times the feature is taken as a division point, and the more the feature appears in the division nodes of the decision tree, the higher its importance is relatively. Optionally, the more the number of nodes in the decision tree corresponding to the predicted time sequence, the greater the influence degree of the feature information corresponding to the decision tree on the predicted time sequence is, the greater the weight of the feature information corresponding to the decision tree is, and the greater the influence degree of the feature information on the result of the predicted time sequence is. Therefore, according to the number of nodes in each decision tree and the node splitting gain, the calculation of each characteristic weight can be effectively carried out, and the accurate calculation of the weights of a plurality of types of characteristic information related to the predicted time sequence is realized.
104, training the time sequence prediction model fused with the XGBoost calculation feature weights according to the feature information of the plurality of types and the weights of the feature information of the plurality of types to obtain a trained time sequence prediction model fused with the XGBoost calculation feature weights; the time sequence prediction model fused with XGBoost to calculate the feature weight is constructed based on a cyclic neural network model GRU; the XGBoost computing feature weight fusion time series prediction model is used for predicting the expected result of the time series variable.
Specifically, after determining weights of the feature information corresponding to each decision tree according to the number of nodes in each decision tree, training a time sequence prediction model fused with XGBoost computing feature weights according to the feature information of a plurality of types and the weights of the feature information of a plurality of types related to the predicted time sequence to obtain a trained time sequence prediction model fused with XGBoost computing feature weights; optionally, a time series prediction model fused with XGBoost to calculate the feature weight is constructed based on the recurrent neural network model GRU; alternatively, the GRU model works well in multi-dimensional feature extraction while processing large volumes of data with advantages in training speed.
Optionally, according to the predicted time series related multiple types of feature information and weights of the multiple types of feature information, the time series prediction model fused with XGBoost computing feature weights is trained, that is, XGBoost is introduced as an attention mechanism in the training process of the GRU model to realize accurate computation of multidimensional feature weights, so that the GRU model can learn more deeply the predicted time series related multiple types of feature information based on the weights of the multiple types of feature information, learning efficiency is higher, the trained time series prediction model fused with XGBoost computing feature weights can comprehensively capture the predicted time series related feature information, the problem that important information cannot be fully learned when the features are more is solved, the trained time series prediction model fused with XGBoost computing feature weights can more accurately conduct time series prediction, and the problem that the learning of the same weight is not enough to cause low time series prediction accuracy in the prior art is solved.
According to the method, the plurality of types of characteristic information related to the predicted time sequence are obtained from the database, decision trees corresponding to the characteristic information of each type are obtained according to the plurality of types of characteristic information related to the predicted time sequence and the gradient lifting model, the weight of each characteristic information corresponding to each decision tree is determined according to the number of nodes in each decision tree and the gains generated before and after node splitting, and therefore the influence degree of each type of characteristic information on the predicted time sequence can be rapidly, effectively and accurately determined; further, according to the multiple types of feature information related to the predicted time sequence and the weights of the multiple types of feature information, the time sequence prediction model fused with XGBoost computing feature weights is trained, so that the time sequence prediction model fused with XGBoost computing feature weights can learn more deeply the multiple types of feature information related to the predicted time sequence based on the weights of the multiple types of feature information, learning efficiency is higher, the trained time sequence prediction model fused with XGBoost computing feature weights can comprehensively capture the feature information related to the predicted time sequence, the problem that important information cannot be fully learned when features are more is solved, and the trained time sequence prediction model fused with XGBoost computing feature weights can more accurately conduct time sequence prediction. The method of the embodiment of the invention solves the problem that the accuracy of the time sequence prediction is lower because a plurality of important features participate in training with the same weight in the prior art and the degree of learning is insufficient, simultaneously solves the problem that the common attention mechanism is fuzzy and rough in feature weight calculation, and improves the accuracy of the time sequence prediction result of the target object.
In an embodiment, determining the weight of each feature information according to the number of nodes in each decision tree and the splitting gain of the nodes includes:
the weight of each feature information is determined based on the following formula:
；
wherein,representing characteristic information +.>Weights of (2); />Representing characteristic information +.>The number of nodes of the corresponding decision tree; t represents the number of all decision trees; n (t) represents the number of non-leaf nodes of the t-th tree; />A partition feature representing an ith non-leaf node of the nth tree; i () represents an indication function; />Representing the sum of the second derivatives of all samples falling on the ith non-leaf node of the nth tree; n represents the number of feature information associated with the predicted time series.
Specifically, in the embodiment of the invention, after decision trees corresponding to each type of feature information are obtained according to a plurality of types of feature information and gradient lifting models related to the predicted time sequence, the weight of each feature information corresponding to each decision tree can be determined according to the number of nodes in each decision tree and the gain difference generated before and after node splitting caused by the features; alternatively, in the decision tree, a node corresponds to a split of a feature, which feature is such that the number of nodes of the decision tree that are split corresponds to the number of times that feature is a split point, the more a feature appears in a split node of the decision tree, the relatively higher its importance. Optionally, the greater the number of nodes split by a certain feature in the decision tree corresponding to the predicted time sequence, the greater the influence degree of the feature information on the predicted time sequence, the greater the weight of the feature information, and the greater the influence degree of the feature information on the predicted result of the predicted time sequence. According to the number of nodes in each decision tree, the weight of each corresponding characteristic information in the forest can be effectively calculated, and the accurate calculation of the weights of a plurality of types of characteristic information related to the predicted time sequence is realized.
Optionally, when determining the weight of each piece of feature information corresponding to the decision tree according to the number of nodes in the decision tree, the weight of each piece of feature information corresponding to the decision tree may be determined according to the following formula, so as to accurately and effectively quantify the influence degree of the feature information on the predicted time sequence, where the specific quantization formula is as follows:
；
wherein,representing characteristic information +.>Weights of (2); />Representing characteristic information +.>The number of nodes of the corresponding decision tree; t represents the number of all decision trees; n (t) represents the number of non-leaf nodes of the t-th tree; />A partition feature representing an ith non-leaf node of the nth tree; i () represents an indication function; />Representing all samples falling on the ith non-leaf node of the nth treeIs the sum of the second derivatives of (a); n represents the number of feature information associated with the predicted time series.
Optionally, after determining the weight of each corresponding feature information in the forest, the weight of each feature information may be multiplied by the corresponding feature information, so as to give the network attention to the time sequence prediction model fused with the XGBoost computing feature weight, and improve the prediction accuracy of the time sequence prediction model fused with the XGBoost computing feature weight.
According to the method, when the weight of each piece of characteristic information corresponding to each decision tree is determined according to the number of nodes in the forest, the weight of each piece of characteristic information corresponding to each decision tree can be determined according to a target formula, so that the influence degree of the characteristic information corresponding to each decision tree on a predicted time sequence is accurately and effectively quantified, the accurate calculation of the weight of a plurality of types of characteristic information related to the predicted time sequence according to the number of the nodes in the forest is realized, the weight of each piece of characteristic information can be multiplied by the corresponding characteristic information, the attention of a time sequence prediction model network fused with XGBoost computing characteristic weights is endowed, and the prediction accuracy of a time sequence prediction model fused with XGBoost computing characteristic weights is improved.
In an embodiment, determining whether the decision tree stops growing is based on a target rule, the target rule comprising at least one of:
the target gain before and after splitting of the decision tree nodes is smaller than or equal to a first threshold value;
the number of samples contained in the decision tree node is less than or equal to a second threshold;
the number of splitting layers of the decision tree reaches a third threshold.
Specifically, in the embodiment of the invention, a plurality of types of characteristic information and time sequence variables related to the predicted time sequence can be input into a gradient lifting model as samples, so that a decision tree for determining the influence degree of each type of characteristic information on the predicted time sequence is generated; alternatively, in the process of generating the decision tree for determining the degree of influence of each type of feature information on the predicted time series, the number of nodes in the decision tree corresponding to each feature information may be determined based on the following target rule; wherein the target rule includes at least one of: target gain before and after splitting of the decision tree node, the number of samples contained in the decision tree node and the splitting layer number of the decision tree; the target gain is the difference value of the loss function before and after splitting of the decision tree nodes; when a node is split, the gain relation of the loss function of the current node and the child node after the split can be analyzed; alternatively, the larger the gain value, the better the current structure is; optionally, when at least one of three conditions in the target rule is met, suspending splitting of the current node of the decision tree corresponding to the characteristic information; the decision tree corresponding to the characteristic information is determined according to the structure of the decision tree corresponding to the characteristic information, and the structure of the decision tree corresponding to the determined characteristic information is used as a trained classifier. Finally, a plurality of classifiers (decision trees) are formed, the number of nodes split by the corresponding features in each classifier, namely the number of times of feature splitting, is counted, and the gain value generated by the features on the node splitting in the process of node splitting is fused to serve as the influence of the features, so that the effect of accurately determining the influence degree of the feature information of each type on the predicted time sequence based on the gradient lifting model is achieved.
According to the method, based on the target rule, the effect of accurately determining the number of the nodes in the decision tree corresponding to the feature information is achieved, and further the influence degree of the feature information corresponding to the decision tree on the prediction time sequence can be effectively determined based on the number of the nodes in the decision tree corresponding to the feature information and the splitting gain of each node, so that the accurate quantification of the weight of the feature information is achieved.
In an embodiment, based on whale optimization algorithm WOA, preset parameters of the time sequence prediction model fused with XGBoost to calculate the feature weight are optimized, and the optimized time sequence prediction model fused with XGBoost to calculate the feature weight is obtained.
Specifically, in order to enable the time sequence prediction model fusing XGBoost computing feature weights to more accurately conduct time sequence prediction of a target object, in the embodiment of the invention, accurate computation of weights of multidimensional feature information is achieved, so that the time sequence prediction model fusing XGBoost computing feature weights can learn more deeply a plurality of types of feature information related to a predicted time sequence based on the weights of the plurality of types of feature information, and on the basis of higher learning efficiency, preset parameters of the time sequence prediction model fusing XGBoost computing feature weights are optimized through a whale optimization algorithm WOA, and therefore the time sequence prediction model fusing XGBoost computing feature weights after parameter optimization can be conducted more accurately.
Optionally, optimizing preset parameters of the time sequence prediction model fused with the XGBoost computing feature weights based on a whale optimization algorithm WOA to obtain the optimized time sequence prediction model fused with the XGBoost computing feature weights, including:
and optimizing the number of neurons and the training iteration times of the time sequence prediction model fused with the XGBoost to calculate the feature weights based on the WOA algorithm to obtain the optimized time sequence prediction model fused with the XGBoost to calculate the feature weights.
Specifically, in the embodiment of the invention, based on whale optimization algorithm WOA, the number of neurons and training iteration times of a time sequence prediction model fused with XGBoost computation feature weights are optimized to obtain the optimized time sequence prediction model fused with XGBoost computation feature weights, so that the time sequence prediction model fused with XGBoost computation feature weights after the optimization of the number of neurons and training iteration times can be more accurately predicted. Optionally, the time sequence prediction model fused with the XGBoost computing feature weight needs to process huge data volume in the process of feature extraction and training, when the time sequence prediction model fused with the XGBoost computing feature weight processes the data, two parameters, namely the neuron number of the time sequence prediction model fused with the XGBoost computing feature weight and the training iteration number of the time sequence prediction model fused with the XGBoost computing feature weight, can change along with the change of the data format and the number, the two parameters mutually influence, the parameter requirement accuracy is higher, and if the prediction result is improperly selected, the prediction result can be greatly influenced. The preset parameters of the time sequence prediction model fused with XGBoost calculation feature weights are optimized through a whale optimization algorithm WOA, so that the occurrence of over-fitting or under-fitting can be effectively avoided, the prediction precision and efficiency are improved, and the system running time and the system load are reduced.
According to the method, the time sequence prediction model fused with the XGBoost to calculate the feature weights is trained according to the feature information of the types and the weights of the feature information of the types related to the predicted time sequence, so that the time sequence prediction model fused with the XGBoost to calculate the feature weights can grasp the feature information related to the predicted time sequence comprehensively, and the problem that important information cannot be learned sufficiently when the features are more is solved. On the other hand, the preset parameters of the time sequence prediction model fused with XGBoost calculation feature weights are optimized through a whale optimization algorithm WOA, and the occurrence of over-fitting or under-fitting can be effectively avoided, so that the prediction precision and efficiency are improved, and the system running time and the system load are reduced.
Exemplary, as shown in fig. 2, the training process of the time series prediction model fusing XGBoost computing feature weights is as follows:
acquiring a plurality of types of characteristic information related to the predicted time sequence from a database:
for example, when predicting the traffic flow in the traffic domain, feature information related to the traffic flow in the traffic domain in a period of time, such as traffic conditions, traffic running time, traffic control conditions, and weather factors, is selected and used as n pieces of feature information.
Or, when predicting the weather change, selecting weather related characteristic information such as cloud cover, visibility, air pressure, air temperature, humidity, wind direction, wind speed and evaporation capacity in a period of time, and taking the characteristic information as n pieces of characteristic information.
Or, when predicting the stock closing price, selecting stock data such as the opening price, the highest price and the trading volume in a period of time, and determining n indexes of the stock sequence data as characteristics.
And normalizing the n-dimensional feature sequence data, and sequencing each feature in the n-dimensional feature sequence so as to quickly candidate the segmentation points in the training of the gradient lifting model based on the residual error.
And sequentially inputting each type of characteristic information, forming a plurality of classifiers by a Boosting algorithm based on gradient lifting, and counting the number of nodes of each classifier so as to determine the influence of each type of characteristic information on the prediction label. The specific process is as follows:
after a plurality of types of characteristic information related to the predicted time sequence are input into a gradient lifting model, when training to a t sub-tree, i sample data, dividing a regularization term of an objective function into a first t-1 term and a t term, and enabling an original objective of the gradient lifting model to be converted into:
；
And performing Taylor expansion processing on the objective function of the gradient lifting model, simplifying the objective function of the gradient lifting model, and further parameterizing the model formula. The complexity of the decision tree corresponding to the feature information is mainly influenced by the number of leaf nodes and node weights, and the formula of the complexity of the decision tree corresponding to the feature information is as follows:
；
wherein T represents the leaf node number of the decision tree corresponding to the characteristic information,representing the norm of the leaf node value. Then it is brought into the function formula of the plaque, for simplicity of this will +.>，/>Gradient lifting can be finally obtainedThe objective function of the model is:
；
the relationship between the leaf node value w and the optimization target value Obj (t) of the decision tree corresponding to the feature information has been determined. And then adopting a greedy algorithm to determine the structure of a decision tree corresponding to the characteristic information. When one node is split, only the relation between the node of the decision tree corresponding to the current characteristic information and the Obj (t) value of the child node is seen. At the current node, gain=obj (pre-split) -Obj (post-split) is defined, with larger gain values indicating better current structure. Three splitting rules are thus defined: (1) gain (t)<=10 -5 The method comprises the steps of carrying out a first treatment on the surface of the (2) The number of samples contained in the nodes of the decision tree corresponding to the characteristic information <=1; (3) Decision tree splitting layer number corresponding to characteristic information>=6; the current node pauses splitting when at least one of the three conditions is met. The nodes are not split in a decision tree corresponding to the characteristic information according to three formulated conditions, at the moment, the structure of the decision tree corresponding to the characteristic information is determined, and the structure of the decision tree corresponding to the determined characteristic information is used as a trained classifier. Counting the number of nodes split by the same feature in each classifier, namely, fusing the number of times of splitting by the feature in the decision tree, fusing the gain value generated by splitting by the feature to the node in the process of splitting by the feature, and finally calculating to obtainAs an influence of the feature information.
And converting the influence of the characteristic information into the weight corresponding to the characteristic information and giving attention to the GRU network in real time. Will firstScaling to a (0, 1) interval, wherein a scaling formula is as follows: />. Then the scaled +.>Constructed as an n-dimensional influence vector V' = (V 1 ’，V 2 ’，V 3 ’……，V n '), the vector is given to the built GRU network input layer. When the n-dimensional feature vector is input, the corresponding feature value is multiplied by the feature weight, so that the attention of the GRU network is given.
Network parameters of a time series prediction model fused with XGBoost calculation feature weights are optimized in real time by using a Whale Optimization Algorithm (WOA). The method comprises the following steps:
The number of whales was first formulated as m, i.e. X1, X2, … …, xm. The position of each whale is then initialized, expressed in a vector of dimension D:
；
the dimension here represents the number of optimization targets, and the number of optimization targets is 2, namely the number of GRU neurons of the GRU layer and the training iteration times of the GRU layer in the time sequence prediction model of which the optimization targets are fused with XGBoost to calculate the feature weights are drawn, so that X in the model is a two-dimensional vector.
Second, each whale randomly chooses to surround or drive the prey with air bubbles in one iteration. If the choice is to surround the prey, the whale will swim towards the best position whale or randomly towards one whale to update its position, the position update formula is as follows:
；
a and C are both random number vectors, when A is (-1, 1), the whale moves towards the whale at the optimal position, otherwise, the whale moves towards the other whale at random; if the whale chooses to drive the prey with bubbles, in order to make the bubbles which are sequentially spitted out in time reach the water surface simultaneously and enclose a circle to trap the prey, the whale can choose the spiral to float upwards when spitting out the bubbles, so the position updating formula is as follows:
；
where b will determine the shape of the floating spiral and l is a random number between [ -1,1 ].
Finally, after the iteration is completed for all times, all whales can reach a position to capture a prey, and the coordinates of the whales are the optimal position at the moment, so that the optimal GRU neuron number and GRU layer training iteration times of the time sequence prediction model fused with XGBoost calculation feature weights are obtained.
And after obtaining the optimal parameters of the time sequence prediction model network fused with the XGBoost computing feature weights through WOA, inputting the prediction data into a trained time sequence prediction model fused with the XGBoost computing feature weights, so that a time sequence prediction result can be obtained.
In one embodiment, a method for predicting a time sequence of feature weights by fusing XGBoost calculation includes:
acquiring a plurality of types of characteristic information related to the predicted time sequence and weights of the characteristic information from a database; the weight of each characteristic information is used for representing the influence degree of each type of characteristic information on the predicted time sequence;
and inputting the feature information of a plurality of types related to the predicted time sequence and the weight of each feature information into a trained fusion XGBoost computing feature weight time sequence prediction model to obtain a time sequence prediction result.
Specifically, in the time sequence prediction process, the embodiment of the invention can firstly acquire a plurality of types of feature information related to the predicted time sequence and the weight of each feature information from a database; alternatively, the method provided by the invention can be applied to various objects and various types of time series predictions. For example, the predicted time series may be, for example, a highest weather temperature, a peak guest volume, a peak traffic volume, a stock quote, and the like. Alternatively, the plurality of types of characteristic information related to the predicted time series may be characteristic information directly or indirectly related to the time series; for example, in the case of predicting the weather change, the characteristic information related to the weather change may include a plurality of factors such as cloud cover, visibility, air pressure, air temperature, humidity, wind direction, wind speed, evaporation amount, and the like; in making predictions of traffic conditions, the characteristic information associated with traffic flow may include: traffic conditions, traffic run time, traffic control conditions, weather factors, etc.; in making a prediction of a stock closing price, the characteristic information related to the stock closing price may include: open price, maximum price, amount of transactions, etc. Alternatively, the weight of each feature information may be determined by the number of nodes of the decision tree corresponding to the feature information.
After obtaining a plurality of types of feature information related to the predicted time sequence and the weight of each feature information, the method can input the plurality of types of feature information related to the predicted time sequence and the weight of each feature information into a trained time sequence prediction model fused with XGBoost computing feature weights, and the problem that the accuracy of the time sequence prediction is low due to the fact that the time sequence prediction model fused with XGBoost computing feature weights is deeper in learning degree and higher in learning efficiency of the plurality of types of feature information related to the predicted time sequence or the rough weight formed by adopting a common attention mechanism is solved.
Exemplary, as shown in fig. 3, a time series prediction system based on gradient lifting and automatic optimization is provided in an embodiment of the present application, which specifically includes:
And a data acquisition module:
for example, when predicting the traffic flow in the traffic domain, the database selects the characteristic information related to the traffic flow in the traffic domain in a period of time, such as traffic conditions, traffic running time, traffic control conditions and weather factors, and uses the characteristic information as n pieces of characteristic information.
Or, when predicting the weather change, selecting weather related characteristic information such as cloud cover, visibility, air pressure, air temperature, humidity, wind direction, wind speed and evaporation amount in a database, and taking the characteristic information as n pieces of characteristic information.
Or, when predicting the stock closing price, the multi-dimensional characteristic data of predicting the stock daily line is obtained through the api of the stock exchange website, and the data is imported into the system. Then, a multidimensional label for extracting features such as a price for opening, a highest price, a volume of transaction, etc. is determined, and a price for closing is determined as a label for prediction.
And a data preprocessing module:
the imported whole data is integrated into a complete data set, and then the data is cleaned to remove empty data and error data. Dividing the cleaned data into two parts of an n-dimensional characteristic label and a predictive label, and normalizing the two parts of data by using a z-score standard to form a data set X.
The feature attention generation module:
the feature label part of the data set X is sequenced according to each feature, then each type of sequenced sequence is input, a plurality of classifiers are finally formed through Boosting algorithm training based on gradient lifting, each classifier counts the number of own nodes and calculates the gain of each node to a model when splitting, n feature importance values are finally formed, and the n values are normalized in equal proportion to generate an attention weight sequence of the current weather change condition, traffic flow or stranding feature.
Feature extraction and training module:
the dataset X is input into a feature extraction network. The characteristic part of each piece of data in the data set X passes through the input layer, and the characteristic attention weight sequence generated by the last module is multiplied by the characteristic part of each piece of data in the data set X to give weight to the corresponding characteristic data. And inputting the weighted data into a GRU network layer, and extracting weather change conditions, traffic flow or strand index data characteristics. And the dropout layer is set to filter out 30% of the characteristic data to prevent overfitting. And finally, the extracted characteristic data are integrated through a full-connection layer to form a predicted value and a predicted label (a true value) for training, and the whole process is iterated for k times.
Model automatic optimization module:
and (3) optimizing the number of GRU neurons and the number of network iterations in the feature extraction and training module in real time by using a Whale Optimization Algorithm (WOA), and carrying out the feature extraction and training to obtain optimal parameters and then carrying out a trained feature extraction model.
Model prediction module:
and predicting weather change conditions, traffic flow or stock fingers by using the trained and optimized model to obtain a predicted value. In the embodiment of the invention, after predicting the peak traffic flow, the highest weather temperature or the stock price, the user can make decisions in advance, which provides help and reference for the decision aspect of the user.
According to the embodiment of the invention, the characteristic information of a plurality of types related to the predicted time sequence is input into XGBoost for training, the influence of the characteristic on the predicted label is determined through the characteristic times used by a Boosting algorithm in node splitting, then the influence is converted into the characteristic weight, the characteristic weight is used as attention to be introduced into a GRU model, and two important parameters of the neuron number and the iteration times of the model are automatically selected by a Whale Optimization Algorithm (WOA) are fused, so that the situation of over-fitting or under-fitting of the model is effectively avoided.
For the purpose of evaluating the time series prediction method in the embodiment of the application, an attention mechanism used in the prior art is selected for comparison. The comparative experimental model is as follows:
data preprocessing:
in the embodiment of the present application, characteristic information (traffic conditions, traffic running time, traffic control conditions, weather factors, traffic flow peaks), weather-related characteristic information (cloud cover, visibility, air pressure, air temperature, humidity, wind direction, air speed, evaporation amount, weather maximum temperature), characteristic information (price of a stock, highest price, lowest price, amount of a transaction, amount of a rise, amount of a fall, amount of a take-off) in traffic in the traffic field of 2016 month 1 to 2021 month 12 are acquired, and the data are subjected to cleaning, wherein the traffic peaks, the weather maximum temperature, the amount of the take-off are used as predictive labels, and the remaining labels are used as characteristic labels. The data are divided into a training set and a testing set according to the ratio of 5:5, namely, the first two years are half training sets, the second two years are half testing sets, and the data are normalized to improve the convergence rate of the model. Every 2 adjacent days is formulated as an input data.
Multi-feature LSTM model: when predicting the traffic flow in the traffic field, selecting the characteristic information related to the traffic flow in the traffic field within a period of time from a database, selecting traffic conditions, traffic running time, traffic control conditions and weather factors, taking the traffic conditions, the traffic running time, the traffic control conditions and the weather factors as n pieces of characteristic information, and taking a peak value of the traffic flow as a predicted time sequence.
When the weather change condition is predicted, the characteristic information related to weather in a period of time is selected from a database, cloud cover, visibility, air pressure, air temperature, humidity, wind direction, wind speed and evaporation capacity are selected and used as n pieces of characteristic information, and the highest weather temperature is used as a predicted time sequence.
When predicting the stock closing price, the multi-dimensional characteristic data of predicting the stock daily line is obtained through the api of the stock exchange website, and the data is imported into the system. Then, a multi-dimensional label for extracting the characteristics is determined, the opening price, the highest price, the lowest price, the amount of the deal and the amount of the deal are selected, the rising and falling range is used as 6-dimensional characteristic information, and the closing price is determined as a label for prediction.
Multi-feature GRU model: when predicting the traffic flow in the traffic field, selecting the characteristic information related to the traffic flow in the traffic field within a period of time from a database, selecting traffic conditions, traffic running time, traffic control conditions and weather factors, taking the traffic conditions, the traffic running time, the traffic control conditions and the weather factors as n pieces of characteristic information, and taking a peak value of the traffic flow as a predicted time sequence.
When the weather change condition is predicted, the characteristic information related to weather in a period of time is selected from a database, cloud cover, visibility, air pressure, air temperature, humidity, wind direction, wind speed and evaporation capacity are selected and used as n pieces of characteristic information, and the highest weather temperature is used as a predicted time sequence.
When predicting the stock closing price, the multi-dimensional characteristic data of predicting the stock daily line is obtained through the api of the stock exchange website, and the data is imported into the system. Then, a multi-dimensional label for extracting the features is determined, the opening price, the highest price, the lowest price, the amount of the deal and the amount of the rise and fall are selected as 6-dimensional feature information, and the closing price is determined as a label for prediction.
Attention-recurrent neural network model Attention-GRU model: a common attention mechanism is introduced in the multi-feature GRU model.
Gradient lifting-circulating neural network model XGBoost-GRU: XGBoost attention mechanisms are introduced in the multi-feature GRU model.
Whale optimization-gradient lifting-circulation neural network model WOA-XGB-GRU model: and optimizing the XGBoost-GRU model by using a WOA algorithm.
All models are initially set, the number of hidden layer neurons of LSTM and GRU models is 250, the random forgetting rate is 0.3, the input dimension of each neuron is set to be (6, 3), adam 'is selected as an optimizer, the learning rate is set to be 0.1, the batch processing size is 2000, mse' is taken as a loss function, and the iteration number is 500.
The experimental results are shown in table 1: the model predicts more than 700 continuous time sequence results, the prediction width covers a plurality of periods, the prediction results are more comprehensive, and the root mean square error has reference significance.
TABLE 1
The root mean square error of the multi-feature LSTM and GRU were 43.0 and 41.28, respectively, which suggests that GRU model performance is better when processing multi-feature large-scale data.
The root mean square error was reduced from 41.28 to 38.275 by introducing XGBoost training to feature importance and giving the GRU attention to the handling of sample features. In the process of predicting the fingers, the feature split times in training are the highest price, the lowest price, the rising and falling range, the opening price, the trading volume and the trading volume from high to low, namely the highest price and the lowest price occupy higher importance in the 6 important indexes, so that more attention is given to the following training. The average error rate of the XGBoost-GRU model is 38.275, and the accuracy is improved by about 8% compared with the general GRU and about 5% compared with the Attention mechanism model Attention-GRU which is popular.
The root mean square error of the WOA-XGB-GRU model is 36.012, and the accuracy is improved by about 6% on the basis of XGBoost-GRU. And introducing a WOA algorithm to optimize the number of neurons and the iteration times of the attention mechanism model, wherein the number of the locally optimal neurons after optimization is 388, and the optimal iteration times is 106. After the WOA algorithm is added, the iteration times are reduced from the initial 500 times to 106 times, the prediction accuracy is improved, the phenomenon of over-fitting is eliminated, and the model performance is improved greatly. By comparing the Attention-GRU model, the Attention mechanism based on XGBoost can calculate an accurate weight as a characteristic, so that the performance of the XGBoost-based Attention mechanism exceeds that of a common Attention mechanism, is more sensitive to change in prediction, can rapidly react at peaks and troughs, and can effectively avoid the occurrence of over-fitting or under-fitting by a WOA algorithm.
The training device of the time sequence prediction model fusing XGBoost to calculate the feature weight provided by the invention is described below, and the training device of the time sequence prediction model fusing XGBoost to calculate the feature weight described below and the training method of the time sequence prediction model fusing XGBoost to calculate the feature weight described above can be correspondingly referred to each other.
Fig. 4 is a schematic structural diagram of a training device of a time series prediction model fused with XGBoost computing feature weights. The training device for fusing XGBoost computing feature weight time sequence prediction model provided in the embodiment includes:
an acquisition module 710 for acquiring a plurality of types of feature information related to the predicted time series from a database;
a determining module 720, configured to obtain decision trees corresponding to each type of feature information according to the plurality of types of feature information and the gradient lifting model related to the predicted time sequence; the decision tree is used for determining the influence degree of each type of characteristic information on the predicted time sequence;
determining the weight of each piece of characteristic information corresponding to each decision tree according to the number of nodes in each decision tree;
the training module 730 is configured to train the time-series prediction model fused with XGBoost to calculate the feature weight according to the feature information of multiple types and the weights of the feature information of multiple types related to the predicted time-series, so as to obtain a trained time-series prediction model fused with XGBoost to calculate the feature weight; the time sequence prediction model fused with XGBoost to calculate the feature weight is constructed based on a cyclic neural network model GRU; and a time sequence prediction model fused with XGBoost to calculate feature weights is used for predicting expected results of the time sequence.
The device of the embodiment of the present invention is configured to perform the method of any of the foregoing method embodiments, and its implementation principle and technical effects are similar, and are not described in detail herein.
Fig. 5 illustrates a physical schematic diagram of an electronic device, which may include: processor 810, communication interface (Communications Interface) 820, memory 830, and communication bus 840, wherein processor 810, communication interface 820, memory 830 accomplish communication with each other through communication bus 840. Processor 810 may invoke logic instructions in memory 830 to perform a training method that fuses the XGBoost computational feature weight time series prediction model, the method comprising: acquiring a plurality of types of characteristic information related to the predicted time sequence variable from a database; according to the characteristic information of a plurality of types and the gradient lifting model XGBoost, each decision tree is learned; the decision tree is used for determining the influence degree of the characteristic information of each type on the predicted time sequence variable; determining the weight of each characteristic information according to the number of nodes in each decision tree and the splitting gain of the nodes; training the time sequence prediction model fused with the XGBoost calculation feature weights according to the feature information of the plurality of types and the weights of the feature information of the plurality of types to obtain a trained time sequence prediction model fused with the XGBoost calculation feature weights; the time sequence prediction model fused with XGBoost to calculate the feature weight is constructed based on a cyclic neural network model GRU; the XGBoost computing feature weight fusion time series prediction model is used for predicting the expected result of the time series variable.
Further, the logic instructions in the memory 830 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are capable of performing the training method of the time series prediction model incorporating XGBoost computing feature weights provided by the methods described above, the method comprising: acquiring a plurality of types of characteristic information related to the predicted time sequence variable from a database; according to the characteristic information of a plurality of types and the gradient lifting model XGBoost, each decision tree is learned; the decision tree is used for determining the influence degree of the characteristic information of each type on the predicted time sequence variable; determining the weight of each characteristic information according to the number of nodes in each decision tree and the splitting gain of the nodes; training the time sequence prediction model fused with the XGBoost calculation feature weights according to the feature information of the plurality of types and the weights of the feature information of the plurality of types to obtain a trained time sequence prediction model fused with the XGBoost calculation feature weights; the time sequence prediction model fused with XGBoost to calculate the feature weight is constructed based on a cyclic neural network model GRU; the XGBoost computing feature weight fusion time series prediction model is used for predicting the expected result of the time series variable.
In yet another aspect, the present invention further provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the above-provided training method of merging XGBoost computational feature weights time-series prediction models, the method comprising: acquiring a plurality of types of characteristic information related to the predicted time sequence variable from a database; according to the characteristic information of a plurality of types and the gradient lifting model XGBoost, each decision tree is learned; the decision tree is used for determining the influence degree of the characteristic information of each type on the predicted time sequence variable; determining the weight of each characteristic information according to the number of nodes in each decision tree and the splitting gain of the nodes; training the time sequence prediction model fused with the XGBoost calculation feature weights according to the feature information of the plurality of types and the weights of the feature information of the plurality of types to obtain a trained time sequence prediction model fused with the XGBoost calculation feature weights; the time sequence prediction model fused with XGBoost to calculate the feature weight is constructed based on a cyclic neural network model GRU; the XGBoost computing feature weight fusion time series prediction model is used for predicting the expected result of the time series variable.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (8)

1. A training method of a time series prediction model for calculating feature weights by fusing a gradient lifting model XGBoost is characterized by comprising the following steps:
Acquiring a plurality of types of characteristic information related to the predicted time sequence variable from a database;
according to the characteristic information of the multiple types and the gradient lifting model XGBoost, learning to obtain each decision tree; the decision tree is used for determining the influence degree of each type of characteristic information on the predicted time sequence variable;
determining the weight of each type of characteristic information according to the number of nodes in each decision tree and the splitting gain of the nodes;
training the time sequence prediction model fused with the XGBoost calculation feature weights according to the feature information of the multiple types and the weights of the feature information of the multiple types to obtain a trained time sequence prediction model fused with the XGBoost calculation feature weights; the XGBoost computing feature weight fusion time sequence prediction model is constructed based on a cyclic neural network model GRU; the XGBoost computing feature weight fusion time sequence prediction model is used for predicting the expected result of the time sequence variable; the time sequence prediction model is used for predicting traffic flow and weather change conditions in the traffic field;
the determining the weight of each type of characteristic information according to the number of nodes in each decision tree and the splitting gain of the nodes comprises the following steps:
The weights of the various types of feature information are determined based on the following formula:
；
wherein,representing characteristic information +.>Weights of (2); />Representing characteristic information +.>The number of nodes of the corresponding decision tree; t represents the number of all decision trees; n (t) represents the number of non-leaf nodes of the t-th tree; />A partition feature representing an ith non-leaf node of the nth tree; i () represents an indication function; />Representing the sum of the second derivatives of all samples falling on the ith non-leaf node of the nth tree; n represents the number of feature information.
2. The training method of a time series prediction model for computing feature weights by fusing a gradient lifting model XGBoost according to claim 1, further comprising:
determining whether the decision tree stops growing based on a target rule; the target rule includes at least one of:
the target gain before and after splitting of the decision tree nodes is smaller than or equal to a first threshold;
the number of samples contained in the decision tree node is smaller than or equal to a second threshold value;
the number of splitting layers of the decision tree reaches a third threshold.
3. The training method of a time series prediction model for computing feature weights by fusing a gradient lifting model XGBoost according to claim 2, further comprising:
Optimizing preset parameters of the time sequence prediction model fused with the XGBoost calculation feature weights based on a whale optimization algorithm WOA to obtain the optimized time sequence prediction model fused with the XGBoost calculation feature weights.
4. The training method of the time-series prediction model for calculating the feature weight by fusing the gradient lifting model XGBoost according to claim 3, wherein the optimizing the preset parameters of the time-series prediction model for calculating the feature weight by fusing the XGBoost based on the whale optimizing algorithm WOA to obtain the optimized time-series prediction model for calculating the feature weight by fusing the XGBoost comprises the following steps:
based on whale optimization algorithm WOA, the neuron number and training iteration number of the time sequence prediction model fused with XGBoost to calculate the feature weight are optimized, and the optimized time sequence prediction model fused with XGBoost to calculate the feature weight is obtained.
5. A time sequence prediction method for calculating feature weights by fusing a gradient lifting model XGBoost is characterized by comprising the following steps:
acquiring a plurality of types of characteristic information related to the predicted time sequence and weights of the plurality of types of characteristic information; the weight of the characteristic information is used for representing the influence degree of each type of characteristic information on the predicted time sequence;
Inputting the multiple types of feature information related to the predicted time sequence and the weights of the multiple types of feature information into a trained time sequence prediction model fused with XGBoost to calculate feature weights, so as to obtain a prediction result of the time sequence; the time sequence prediction model for fusing the XGBoost calculation feature weights is trained based on the training method for fusing the XGBoost calculation feature weights of the gradient lifting model according to any one of claims 1-4.
6. A training device for a time series prediction model for calculating feature weights by fusing a gradient lifting model XGBoost, comprising:
an acquisition module for acquiring a plurality of types of characteristic information related to the predicted time series variable from a database;
the determining module is used for learning to obtain each decision tree according to the characteristic information of the multiple types and the gradient lifting model XGBoost; the decision tree is used for determining the influence degree of each type of characteristic information on the predicted time sequence variable;
determining the weight of each type of characteristic information according to the number of nodes in each decision tree and the splitting gain of the nodes; the determining the weight of each type of characteristic information according to the number of nodes in each decision tree and the splitting gain of the nodes comprises the following steps:
The weights of the various types of feature information are determined based on the following formula:
；
wherein,representing characteristic information +.>Weights of (2); />Representing characteristic information +.>The number of nodes of the corresponding decision tree; t represents the number of all decision trees; n (t) represents the number of non-leaf nodes of the t-th tree; />Ith non-leaf representing the t-th treeDividing features of the child nodes; i () represents an indication function; />Representing the sum of the second derivatives of all samples falling on the ith non-leaf node of the nth tree; n represents the number of feature information;
the training module is used for training the time sequence prediction model fused with the XGBoost to calculate the characteristic weight according to the characteristic information of the plurality of types and the weight of the characteristic information of the plurality of types, so as to obtain the trained time sequence prediction model fused with the XGBoost to calculate the characteristic weight; the XGBoost computing feature weight fusion time sequence prediction model is constructed based on a cyclic neural network model GRU; the XGBoost computing feature weight fusion time sequence prediction model is used for predicting the expected result of the time sequence variable; the time sequence prediction model is used for predicting traffic flow and weather change conditions in the traffic field.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the training method of the time series prediction model of the fusion gradient boost model XGBoost computational feature weights of any one of claims 1 to 4 or the time series prediction method of the fusion gradient boost model XGBoost computational feature weights of claim 5 when executing the program.
8. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements a training method of a time-series prediction model of fusion gradient lifting model XGBoost computation feature weights according to any one of claims 1 to 4 or a time-series prediction method of fusion gradient lifting model XGBoost computation feature weights according to claim 5.
CN202311156329.2A 2023-09-08 2023-09-08 XGBoost-fused time sequence prediction method for calculating feature weights Active CN116881854B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311156329.2A CN116881854B (en) 2023-09-08 2023-09-08 XGBoost-fused time sequence prediction method for calculating feature weights

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311156329.2A CN116881854B (en) 2023-09-08 2023-09-08 XGBoost-fused time sequence prediction method for calculating feature weights

Publications (2)

Publication Number Publication Date
CN116881854A CN116881854A (en) 2023-10-13
CN116881854B true CN116881854B (en) 2023-12-22

Family

ID=88266697

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311156329.2A Active CN116881854B (en) 2023-09-08 2023-09-08 XGBoost-fused time sequence prediction method for calculating feature weights

Country Status (1)

Country Link
CN (1) CN116881854B (en)

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111340273A (en) * 2020-02-17 2020-06-26 南京邮电大学 Short-term load prediction method for power system based on GEP parameter optimization XGboost
CN111462485A (en) * 2020-03-31 2020-07-28 电子科技大学 Traffic intersection congestion prediction method based on machine learning
CN111738534A (en) * 2020-08-21 2020-10-02 支付宝(杭州)信息技术有限公司 Training of multi-task prediction model, and prediction method and device of event type
CN112581172A (en) * 2020-12-18 2021-03-30 四川中电启明星信息技术有限公司 Multi-model fusion electricity sales quantity prediction method based on empirical mode decomposition
CN113096388A (en) * 2021-03-22 2021-07-09 北京工业大学 Short-term traffic flow prediction method based on gradient lifting decision tree
CN113159389A (en) * 2021-03-25 2021-07-23 大连海事大学 Financial time sequence prediction method based on deep forest generation countermeasure network
CN114169110A (en) * 2022-02-10 2022-03-11 中国人民解放军海军工程大学 Motor bearing fault diagnosis method based on feature optimization and GWAA-XGboost
CN114358213A (en) * 2022-03-08 2022-04-15 湖南大学 Error ablation processing method, system and medium for nonlinear time series data prediction
CN115189416A (en) * 2022-06-27 2022-10-14 中国华能集团清洁能源技术研究院有限公司 Power generation system control method and system based on day-ahead electricity price grading prediction model
CN115510981A (en) * 2022-09-29 2022-12-23 中银金融科技(苏州)有限公司 Decision tree model feature importance calculation method and device and storage medium
CN115630711A (en) * 2022-12-19 2023-01-20 华控清交信息科技(北京)有限公司 XGboost model training method and multi-party security computing platform
CN115936773A (en) * 2022-08-26 2023-04-07 百维金科(上海)信息科技有限公司 Internet financial black product identification method and system
CN116010885A (en) * 2022-12-21 2023-04-25 重庆邮电大学 Method and system for detecting abnormal space-time data of vehicle under long-sequence condition
WO2023113729A1 (en) * 2021-11-24 2023-06-22 Kriptek Kripto Ve Bilişim Teknolojileri Sanayi Ve Ticaret Anonim Şirketi High performance machine learning system based on predictive error compensation network and the associated device
CN116304299A (en) * 2023-01-03 2023-06-23 电子科技大学 Personalized recommendation method integrating user interest evolution and gradient promotion algorithm

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10361926B2 (en) * 2017-03-03 2019-07-23 Nec Corporation Link prediction with spatial and temporal consistency in dynamic networks
US10783452B2 (en) * 2017-04-28 2020-09-22 International Business Machines Corporation Learning apparatus and method for learning a model corresponding to a function changing in time series

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111340273A (en) * 2020-02-17 2020-06-26 南京邮电大学 Short-term load prediction method for power system based on GEP parameter optimization XGboost
CN111462485A (en) * 2020-03-31 2020-07-28 电子科技大学 Traffic intersection congestion prediction method based on machine learning
CN111738534A (en) * 2020-08-21 2020-10-02 支付宝(杭州)信息技术有限公司 Training of multi-task prediction model, and prediction method and device of event type
CN112581172A (en) * 2020-12-18 2021-03-30 四川中电启明星信息技术有限公司 Multi-model fusion electricity sales quantity prediction method based on empirical mode decomposition
CN113096388A (en) * 2021-03-22 2021-07-09 北京工业大学 Short-term traffic flow prediction method based on gradient lifting decision tree
CN113159389A (en) * 2021-03-25 2021-07-23 大连海事大学 Financial time sequence prediction method based on deep forest generation countermeasure network
WO2023113729A1 (en) * 2021-11-24 2023-06-22 Kriptek Kripto Ve Bilişim Teknolojileri Sanayi Ve Ticaret Anonim Şirketi High performance machine learning system based on predictive error compensation network and the associated device
CN114169110A (en) * 2022-02-10 2022-03-11 中国人民解放军海军工程大学 Motor bearing fault diagnosis method based on feature optimization and GWAA-XGboost
CN114358213A (en) * 2022-03-08 2022-04-15 湖南大学 Error ablation processing method, system and medium for nonlinear time series data prediction
CN115189416A (en) * 2022-06-27 2022-10-14 中国华能集团清洁能源技术研究院有限公司 Power generation system control method and system based on day-ahead electricity price grading prediction model
CN115936773A (en) * 2022-08-26 2023-04-07 百维金科(上海)信息科技有限公司 Internet financial black product identification method and system
CN115510981A (en) * 2022-09-29 2022-12-23 中银金融科技(苏州)有限公司 Decision tree model feature importance calculation method and device and storage medium
CN115630711A (en) * 2022-12-19 2023-01-20 华控清交信息科技(北京)有限公司 XGboost model training method and multi-party security computing platform
CN116010885A (en) * 2022-12-21 2023-04-25 重庆邮电大学 Method and system for detecting abnormal space-time data of vehicle under long-sequence condition
CN116304299A (en) * 2023-01-03 2023-06-23 电子科技大学 Personalized recommendation method integrating user interest evolution and gradient promotion algorithm

Also Published As

Publication number Publication date
CN116881854A (en) 2023-10-13

Similar Documents

Publication Publication Date Title
CN110751318B (en) Ultra-short-term power load prediction method based on IPSO-LSTM
CN113190699B (en) Remote sensing image retrieval method and device based on category-level semantic hash
CN111898689B (en) Image classification method based on neural network architecture search
CN112529638B (en) Service demand dynamic prediction method and system based on user classification and deep learning
CN113839926B (en) Method, system and device for modeling intrusion detection system based on characteristic selection of wolf algorithm
CN114548591A (en) Time sequence data prediction method and system based on hybrid deep learning model and Stacking
CN116720620A (en) Grain storage ventilation temperature prediction method based on IPSO algorithm optimization CNN-BiGRU-Attention network model
CN114091650A (en) Searching method and application of deep convolutional neural network architecture
CN111275074A (en) Power CPS information attack identification method based on stack type self-coding network model
CN113822419A (en) Self-supervision graph representation learning operation method based on structural information
CN116245259B (en) Photovoltaic power generation prediction method and device based on depth feature selection and electronic equipment
CN116881854B (en) XGBoost-fused time sequence prediction method for calculating feature weights
CN112651499A (en) Structural model pruning method based on ant colony optimization algorithm and interlayer information
CN115713144A (en) Short-term wind speed multi-step prediction method based on combined CGRU model
CN116703607A (en) Financial time sequence prediction method and system based on diffusion model
CN112529637B (en) Service demand dynamic prediction method and system based on context awareness
CN112749211B (en) Novel tea yield prediction method based on electric power big data
CN115908909A (en) Evolutionary neural architecture searching method and system based on Bayes convolutional neural network
CN115240782A (en) Drug attribute prediction method, device, electronic device and storage medium
CN114120367A (en) Pedestrian re-identification method and system based on circle loss measurement under meta-learning framework
CN113837220A (en) Robot target identification method, system and equipment based on online continuous learning
CN114220164A (en) Gesture recognition method based on variational modal decomposition and support vector machine
CN113762591A (en) Short-term electric quantity prediction method and system based on GRU and multi-core SVM counterstudy
Jiang et al. A CTR prediction approach for advertising based on embedding model and deep learning
CN117668701B (en) AI artificial intelligence machine learning system and method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant