CN116777213A - Carbon transaction market risk early warning system and method based on big data - Google Patents
Carbon transaction market risk early warning system and method based on big data Download PDFInfo
- Publication number
- CN116777213A CN116777213A CN202310741848.9A CN202310741848A CN116777213A CN 116777213 A CN116777213 A CN 116777213A CN 202310741848 A CN202310741848 A CN 202310741848A CN 116777213 A CN116777213 A CN 116777213A
- Authority
- CN
- China
- Prior art keywords
- model
- module
- early warning
- risk assessment
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 title claims abstract description 51
- 229910052799 carbon Inorganic materials 0.000 title claims abstract description 51
- 238000012502 risk assessment Methods 0.000 claims abstract description 84
- 238000007637 random forest analysis Methods 0.000 claims abstract description 28
- 238000007781 pre-processing Methods 0.000 claims abstract description 25
- 238000003062 neural network model Methods 0.000 claims abstract description 20
- 230000015654 memory Effects 0.000 claims abstract description 16
- 238000003066 decision tree Methods 0.000 claims description 34
- 238000012549 training Methods 0.000 claims description 26
- 230000008569 process Effects 0.000 claims description 16
- 238000004422 calculation algorithm Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 13
- 238000005457 optimization Methods 0.000 claims description 12
- 238000012360 testing method Methods 0.000 claims description 12
- 238000012795 verification Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 9
- 238000004140 cleaning Methods 0.000 claims description 7
- 230000007787 long-term memory Effects 0.000 claims description 7
- 238000011156 evaluation Methods 0.000 claims description 6
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 5
- 230000002159 abnormal effect Effects 0.000 claims description 5
- 238000013079 data visualisation Methods 0.000 claims description 5
- 230000006403 short-term memory Effects 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000013480 data collection Methods 0.000 claims 1
- 230000008901 benefit Effects 0.000 abstract description 5
- 238000004364 calculation method Methods 0.000 description 5
- 230000003044 adaptive effect Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a carbon transaction market risk early warning system and method based on big data, wherein the system comprises a data acquisition module, a data preprocessing module, a risk assessment module, a model updating module and an early warning module; the data acquisition module is connected with the data preprocessing module, the data preprocessing module is connected with the risk assessment module, the risk assessment module is connected with the early warning module, and the model updating module is connected with the risk assessment module. The risk assessment module comprises a risk assessment model, wherein the risk assessment model adopts the combination of a random forest model and a long-period memory neural network model introducing residual connection, the advantages of the random forest model and the long-period memory neural network model are fully utilized, and the comprehensive risk prediction capability is obtained.
Description
Technical Field
The application relates to the technical field of big data analysis, in particular to a carbon transaction market risk early warning system and method based on big data.
Background
With the advent of the carbon trade market, more and more businesses or individuals participated in the carbon trade, the carbon trade market became more complex, and therefore, the carbon trade market required the necessary risk early warning.
The current carbon trade model mainly takes carbon emission reduction, emission reduction benefit, clean energy utilization rate and the like as indexes to measure results, and the current carbon trade model lacks prediction of the future of a carbon trade market.
In view of the foregoing, there is a great need for a carbon transaction market risk early warning system and method based on big data to solve the problems existing in the prior art.
Disclosure of Invention
The application aims to provide a carbon transaction market risk early warning system and method based on big data, and the specific technical scheme is as follows:
the carbon transaction market risk early warning system based on big data comprises a data acquisition module, a data preprocessing module, a risk assessment module, a model updating module and an early warning module; the data acquisition module is connected with the data preprocessing module, the data preprocessing module is connected with the risk assessment module, the risk assessment module is connected with the early warning module, and the model updating module is connected with the risk assessment module;
the data acquisition module is used for acquiring first transaction data;
the data preprocessing module is used for carrying out data cleaning on the first transaction data to obtain second transaction data;
the risk assessment module comprises a risk assessment model, and the risk assessment model is used for carrying out risk assessment on the second transaction data to obtain a risk assessment result;
the model updating module is used for updating and optimizing the risk assessment model;
and an adjustable risk threshold value is arranged in the early warning module, when the risk assessment result exceeds the risk threshold value, early warning is sent to a user, and otherwise, early warning is not sent.
Preferably, in the data acquisition module, the first transaction data includes transaction data in a carbon transaction market.
Preferably, in the data preprocessing module, the data cleansing includes a process missing value, a process abnormal value, and a process repeated value.
Preferably, in the risk assessment module, the risk assessment model includes a random forest model and a long-short-term memory neural network model with residual connection introduced, wherein the random forest model predicts the second transaction data and obtains a first prediction result, the long-short-term memory neural network model predicts the second transaction data and obtains a second prediction result, and the first prediction result and the second prediction result are weighted and averaged together to obtain the risk assessment result.
Preferably, the process of predicting the second transaction data by the random forest model is as follows:
step A1: the method comprises the steps of constructing a decision tree model, namely dividing second transaction data into a training set and a testing set, randomly extracting a certain number of samples from the training set in a put-back way to form a plurality of random subsets, randomly selecting the characteristics of each random subset to obtain a plurality of characteristic subsets, constructing the decision tree model on the characteristic subsets by adopting a classification and regression tree algorithm, wherein each characteristic subset corresponds to an independent decision tree model;
step A2: the method comprises the steps of constructing a random forest model, specifically, combining a plurality of constructed decision tree models to obtain a random forest model, inputting samples in a test set into each decision tree model to obtain a decision tree model prediction result, wherein the average value of all decision tree model prediction results is a first prediction result.
Preferably, in step A2, the method further includes a process of optimizing the random forest model, specifically: calculating root mean square error as an optimization index, and adjusting the depth and the minimum sample number of the decision tree model based on the optimization index, or adjusting the number of the decision tree models and the size of the feature subsets in the random forest model; the root mean square error expression is as follows:
wherein RMSE represents root mean square error, N represents the number of samples, y pred Representing the predicted value of the sample, y true Representing the true value of the sample.
Preferably, the process of predicting the second transaction data by the long-term and short-term memory neural network model is as follows:
step B1: the method comprises the steps of constructing a long-period memory neural network model, specifically, dividing second transaction data into a training set, a verification set and a test set, defining an input layer, a hidden layer, input layer nodes, hidden layer nodes and an activation function of the long-period memory neural network model, and adding residual connection between the hidden layers; training the long-term and short-term memory neural network model by adopting a mean square error as a loss function and adopting a back propagation algorithm and an ADam optimization algorithm to obtain a training model;
step B2: model verification, namely inputting a verification set into a training model to obtain a predicted value, and calculating an evaluation index according to the predicted value and a real label;
step B3: the model tuning, specifically, tuning the training model according to the evaluation index, wherein the tuning comprises adjusting an input layer, a hidden layer, an input layer node and a hidden layer node of the training model, and selecting the optimal combination of the input layer, the hidden layer, the input layer node and the hidden layer node by using a cross verification mode to obtain a prediction model;
step B4: and data prediction, namely inputting the test set into a prediction model, and calculating to obtain a second prediction result through the forward propagation process of the prediction model.
Preferably, in the early warning module, the early warning module continuously monitors feedback of the user, and if the user does not take any measures after receiving the early warning, the early warning module sends out the early warning to the user again.
Preferably, the carbon trade market risk early warning system further comprises a data visualization module, wherein the data visualization module is used for displaying the risk assessment result to a user in real time.
In addition, the application also discloses a carbon trade market risk early warning method based on big data, the method is applied to the carbon trade market risk early warning system to realize the carbon trade market risk early warning method, and the method comprises the following steps:
step S1: the data acquisition module acquires first transaction data and transmits the first transaction data to the data preprocessing module;
step S2: the data cleaning method comprises the steps of processing missing values, abnormal values and repeated values in first transaction data, then carrying out feature selection and normalization processing on the first transaction data to obtain second transaction data, and transmitting the second transaction data to a risk assessment module;
step S3: the risk assessment is specifically that second transaction data are input into a risk assessment model in a risk assessment module to obtain a risk assessment result, and the risk assessment result is transmitted to an early warning module;
step S4: and the risk early warning module detects whether a risk assessment result exceeds a preset risk threshold value, and when the risk assessment result exceeds the risk threshold value, the early warning module sends out early warning to a user, otherwise, the early warning module does not send out early warning.
The technical scheme of the application has the following beneficial effects:
the application discloses a carbon transaction market risk early warning system and method based on big data, wherein the system comprises a data acquisition module, a data preprocessing module, a risk assessment module, a model updating module and an early warning module, wherein the data acquisition module acquires first transaction data in a carbon transaction market, the data preprocessing module preprocesses the first transaction data to obtain second transaction data, the risk assessment module obtains a risk assessment result based on the second transaction data, prediction of the carbon transaction market risk is realized, and the early warning module can send early warning to a user according to the risk assessment result.
The data preprocessing module extracts the characteristics in the transaction data, evaluates and screens the characteristics through an information gain method, and selects the characteristics with higher information gain as the most important characteristics.
The risk assessment module comprises a risk assessment model, wherein the risk assessment model adopts a combination of a random forest model and a long-short-term memory neural network model (ResLSTM model) which introduces residual connection, and fully utilizes the advantages of the random forest model and the ResLSTM model to obtain more comprehensive risk prediction capability. The random forest model is good at processing the importance ranking of the structured data and the features, and can be used for feature selection and construction of a prediction model; the random forest model predicts through integrating a plurality of decision trees, has certain robustness to noise and abnormal values, can process complex relations among a plurality of characteristics, including nonlinear relations and interaction effects, and solves the problem that risk prediction in a carbon trade market is influenced by a plurality of factors. On the other hand, the application also introduces a residual connection LSTM model (ResLSTM model) to process long-term dependency relationship between time series data and capture sequences, and in the carbon trade market, the time series data has important significance, and risks and trends in the market often have time correlation. According to the application, by introducing residual connection, the gradient vanishing problem can be effectively relieved, so that the network can better capture long-term dependency, and meanwhile, the expression capacity, the robustness and the generalization capacity of the model are enhanced, and the convergence and the training efficiency of the model are promoted. According to the application, by fusing the random forest model and the ResLSTM model, the characteristics and the trend of the carbon trade market can be more comprehensively captured, uncertainty and data diversity can be better dealt with, and the accuracy and the stability of risk prediction are improved.
In addition to the objects, features and advantages described above, the present application has other objects, features and advantages. The present application will be described in further detail with reference to the drawings.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:
FIG. 1 is a system block diagram of a carbon trade market risk early warning system in accordance with a preferred embodiment of the present application;
FIG. 2 is a flow chart of the unit structure of the LSTM model in the preferred embodiment of the application;
FIG. 3 is a flow chart of steps of a carbon market risk early warning method in accordance with a preferred embodiment of the present application.
Detailed Description
Embodiments of the application are described in detail below with reference to the attached drawings, but the application can be implemented in a number of different ways, which are defined and covered by the claims.
Examples:
referring to fig. 1, the carbon transaction market risk early warning system based on big data comprises a data acquisition module, a data preprocessing module, a risk assessment module, a model updating module, an early warning module and a data visualization module; the data acquisition module is connected with the data preprocessing module, the data preprocessing module is connected with the risk assessment module, the risk assessment module is connected with the early warning module, and the model updating module is connected with the risk assessment module;
the data acquisition module is used for acquiring first transaction data; the first trading data includes trading data in a carbon trading market. In addition, the transaction data in the present implementation may be carbon emission data, carbon transaction price data, or industry data.
The data preprocessing module is used for carrying out data cleaning on the first transaction data to obtain second transaction data; specifically, the data cleansing includes processing missing values, processing outliers, and processing repeated values; the data cleansing in this embodiment may check the integrity and accuracy of the first transaction data, excluding unsatisfactory data. In addition, the data preprocessing module can unify the format of the data, and is beneficial to subsequent risk assessment. After data cleaning, the data preprocessing module extracts characteristic variables related to targets according to targets of carbon transaction market risk prediction, and evaluates and screens the characteristic variables by using an information gain method.
The risk assessment module comprises a risk assessment model, and the risk assessment model is used for carrying out risk assessment on the second transaction data to obtain a risk assessment result; the risk assessment model comprises a random forest model and a long-short-period memory neural network model with residual error connection introduced, wherein the random forest model predicts the second transaction data and obtains a first prediction result, the long-short-period memory neural network model predicts the second transaction data and obtains a second prediction result, and the first prediction result and the second prediction result are weighted, averaged and summed to obtain the risk assessment result.
The model updating module is used for updating and optimizing the risk assessment model; according to the method, the risk assessment model is updated and optimized through the model updating module so as to adapt to new data and service requirements, the prediction performance and generalization capability of the model are improved, the problem that the original model fails over time is solved, and the availability and accuracy of the model are guaranteed.
And an adjustable risk threshold value is arranged in the early warning module, when the risk assessment result exceeds the risk threshold value, early warning is sent to a user, and otherwise, early warning is not sent.
And the data visualization module is used for displaying the risk assessment result to the user in real time.
Further, in this embodiment, the process of predicting the second transaction data by the random forest model is as follows:
step A1: the method comprises the steps of constructing a decision tree model, namely dividing second transaction data into a training set and a testing set, randomly extracting a certain number of samples from the training set in a put-back way to form a plurality of random subsets, randomly selecting the characteristics of each random subset to obtain a plurality of characteristic subsets, constructing the decision tree model on the characteristic subsets by adopting a classification and regression tree algorithm, wherein each characteristic subset corresponds to an independent decision tree model;
step A2: the method comprises the steps of constructing a random forest model, specifically, combining a plurality of constructed decision tree models to obtain a random forest model, inputting samples in a test set into each decision tree model to obtain a decision tree model prediction result, wherein the average value of all decision tree model prediction results is a first prediction result.
Further, the concrete steps of constructing the decision tree model are as follows:
the first step: acquiring a data set according to the second transaction data, and selecting an initial node according to the characteristics and the labels of the data set;
and a second step of: selecting a feature for each node to divide, traversing all features, traversing all possible values of the feature for each feature, and dividing the data set into subsets;
and a third step of: calculating the degree of uncertainty of each subset, the classification and regression tree algorithm (CART algorithm) uses the square error as a measure, the square error being defined as follows:
wherein MSE represents square error, N is the number of samples, y i Is the true value of the sample i,is the predicted value for sample i. The smaller the square error, the more accurate the prediction result.
Fourth step: selecting the smallest square error, and determining the optimal dividing characteristic and dividing point;
fifthly, if the divided subsets meet the stopping condition, marking the nodes as leaf nodes, and representing a regression result of the decision tree model; if the divided subsets do not meet the stop condition, marking the node as an internal node, and recursively repeating the second to fourth steps for each subset until the divided subsets meet the stop condition; the stopping condition is the depth of a preset decision tree and the number of samples;
and sixthly, after a complete decision tree model is constructed, some nodes are cut off or some leaf nodes are combined to optimize the decision tree model through pruning operation (dropout), so that the complexity and generalization capability of the decision tree model are improved.
Further, in step A2, a process of optimizing the random forest model is further included, specifically: calculating root mean square error as an optimization index, and adjusting the depth and the minimum sample number of the decision tree model based on the optimization index, or adjusting the number of the decision tree models and the size of the feature subsets in the random forest model; the root mean square error expression is as follows:
wherein RMSE represents root mean square error, N represents the number of samples, y pred Representing the predicted value of the sample, y true Representing the true value of the sample.
Further, the process of predicting the second transaction data by the long-short term memory neural network (LSTM) model in this embodiment is as follows:
step B1: the method comprises the steps of constructing a long-period memory neural network model, specifically, dividing second transaction data into a training set, a verification set and a test set, defining an input layer, a hidden layer, input layer nodes, hidden layer nodes and an activation function of the long-period memory neural network model, and adding residual connection between the hidden layers; and training the long-term and short-term memory neural network model by adopting a mean square error as a loss function and adopting a back propagation algorithm and an ADam optimization algorithm to obtain a training model.
The residual connection is specifically that the output of the previous layer and the output of the current layer are added element by element to obtain the final residual connection output, and a specific calculation formula is as follows:
y res ＝y prev +y curr ；
wherein y is res Representing the residual connection output, y prev Representing the output of the previous layer, y curr Representing the output of the current layer.
Further, the specific calculation formula of the loss function is as follows:
wherein Loss is represented by Loss, N is the number of samples, y true,i For the true value of the sample, y pred,i Is a sample predictor.
The network is trained by adopting ADam (Adaptive moment estimation) optimization algorithm, which is an adaptive learning rate optimization algorithm, combines the characteristics of a momentum method and the adaptive learning rate, and dynamically adjusts the learning rate of each parameter by calculating first-order moment estimation and second-order moment estimation of gradient.
The specific algorithm formula of the ADam (Adaptive moment estimation) optimization algorithm is as follows:
m and v represent the first moment estimate and the second moment estimate, respectively
β 1 And beta 2 Exponential decay rates for the first and second moment estimates, respectively
Eta represents learning rate
Step B2: model verification, namely inputting a verification set into a training model to obtain a predicted value, and calculating an evaluation index according to the predicted value and a real label;
step B3: the model tuning, specifically, tuning the training model according to the evaluation index, wherein the tuning comprises adjusting an input layer, a hidden layer, an input layer node and a hidden layer node of the training model, and selecting the optimal combination of the input layer, the hidden layer, the input layer node and the hidden layer node by using a cross verification mode to obtain a prediction model;
step B4: and data prediction, namely inputting the test set into a prediction model, and calculating to obtain a second prediction result through the forward propagation process of the prediction model.
Further, the unit structure flow chart of the LSTM model shown in fig. 2:
(1) forgetting the door: accept a long-term memory C t-1 (output from last cell module) and decides to keep and forget C t-1 Is a part of the same. Input long-term memory at t-1 to C t-1 Multiplying the forgetting factor f t . The forgetting factor calculation formula is:
f t ＝σ(W f ·[h t-1 ,x t ]+b f )；
(2) an input door: determining how much of the input information is stored in the cell state C at the current time t t . The calculation formula is as follows:
i t ＝σ(W i ·[h t-1 ,x t ]+b i )
and cell state equation at time t:
(3) output door: control unit state C t How much is output to the LSTM current output value h t . The calculation formula is as follows:
o t ＝σ(W o ·[h t-1 ,x t ]+b o )
h t ＝o t ⊙tanh(C t )；
wherein x is t Representing the current input data, and consisting of training sets divided by the cross-validation method. W (W) 0 And b 0 Respectively representing a weight matrix and a bias term; sigma (·) represents a sigmoid function, and tanh (·) represents a hyperbolic tangent function.
Preferably, in the early warning module, the early warning module continuously monitors feedback of the user, and if the user does not take any measures after receiving the early warning, the early warning module sends out the early warning to the user again.
In addition, as shown in fig. 3, the embodiment also discloses a carbon trade market risk early warning method based on big data, the method adopts the carbon trade market risk early warning system to realize the carbon trade market risk early warning method, and the method comprises the following steps:
step S1: the data acquisition module acquires first transaction data and transmits the first transaction data to the data preprocessing module;
step S2: the data cleaning method comprises the steps of processing missing values, abnormal values and repeated values in first transaction data, then carrying out feature selection and normalization processing on the first transaction data to obtain second transaction data, and transmitting the second transaction data to a risk assessment module;
step S3: the risk assessment is specifically that second transaction data are input into a risk assessment model in a risk assessment module to obtain a risk assessment result, and the risk assessment result is transmitted to an early warning module;
step S4: and the risk early warning module detects whether a risk assessment result exceeds a preset risk threshold value, and when the risk assessment result exceeds the risk threshold value, the early warning module sends out early warning to a user, otherwise, the early warning module does not send out early warning.
Further, the feature selection is specifically to select feature variables related to risks according to targets of carbon trade market risk prediction, then evaluate and screen the features by adopting an information gain method, and determine the features with the most information quantity for predicting carbon trade market risk. The features are ordered according to the magnitude of the information gain, and the features with higher information gain are selected as the most important features, and the expression is as follows:
G(D,A)＝H(D)-H(D|A)
wherein D is a data set, the sample capacity is |D|, K is the number of classifications, |C k I represents class C k Is the number of samples; dividing D into n subsets D according to the value of the feature A 1 、D 2 、……、D n 。|D i I is sample D i Is the number of samples; g (D, A) is information gain, H (D) is empirical entropy of the data set D; h (D|A) is the empirical conditional entropy of feature A on data set D.
The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (10)
1. The carbon transaction market risk early warning system based on big data is characterized by comprising a data acquisition module, a data preprocessing module, a risk assessment module, a model updating module and an early warning module; the data acquisition module is connected with the data preprocessing module, the data preprocessing module is connected with the risk assessment module, the risk assessment module is connected with the early warning module, and the model updating module is connected with the risk assessment module;
the data acquisition module is used for acquiring first transaction data;
the data preprocessing module is used for carrying out data cleaning on the first transaction data to obtain second transaction data;
the risk assessment module comprises a risk assessment model, and the risk assessment model is used for carrying out risk assessment on the second transaction data to obtain a risk assessment result;
the model updating module is used for updating and optimizing the risk assessment model;
and an adjustable risk threshold value is arranged in the early warning module, when the risk assessment result exceeds the risk threshold value, early warning is sent to a user, and otherwise, early warning is not sent.
2. The carbon market risk early warning system of claim 1, wherein in the data collection module, the first transaction data comprises transaction data in a carbon market.
3. The carbon market risk early warning system of claim 1, wherein in the data preprocessing module, the data cleansing includes processing missing values, processing outliers, and processing duplicate values.
4. The carbon market risk early warning system of claim 1, wherein in the risk assessment module, the risk assessment model comprises a random forest model and a long-short-term memory neural network model introducing residual connection, wherein the random forest model predicts the second transaction data and obtains a first prediction result, the long-short-term memory neural network model predicts the second transaction data and obtains a second prediction result, and the weighted average summation of the first prediction result and the second prediction result obtains a risk assessment result.
5. The carbon market risk pre-warning system of claim 4, wherein the process of predicting the second transaction data by the random forest model is as follows:
step A1: the method comprises the steps of constructing a decision tree model, namely dividing second transaction data into a training set and a testing set, randomly extracting a certain number of samples from the training set in a put-back way to form a plurality of random subsets, randomly selecting the characteristics of each random subset to obtain a plurality of characteristic subsets, constructing the decision tree model on the characteristic subsets by adopting a classification and regression tree algorithm, wherein each characteristic subset corresponds to an independent decision tree model;
step A2: the method comprises the steps of constructing a random forest model, specifically, combining a plurality of constructed decision tree models to obtain a random forest model, inputting samples in a test set into each decision tree model to obtain a decision tree model prediction result, wherein the average value of all decision tree model prediction results is a first prediction result.
6. The carbon market risk pre-warning system according to claim 5, characterized in that in step A2, it further comprises a process of optimizing a random forest model, in particular: calculating root mean square error as an optimization index, and adjusting the depth and the minimum sample number of the decision tree model based on the optimization index, or adjusting the number of the decision tree models and the size of the feature subsets in the random forest model; the root mean square error expression is as follows:
wherein RMSE represents root mean square error, N represents the number of samples, y pred Representing the predicted value of the sample, y true Representing the true value of the sample.
7. The carbon market risk early warning system of claim 4, wherein the long-term memory neural network model predicts the second transaction data as follows:
step B1: the method comprises the steps of constructing a long-period memory neural network model, specifically, dividing second transaction data into a training set, a verification set and a test set, defining an input layer, a hidden layer, input layer nodes, hidden layer nodes and an activation function of the long-period memory neural network model, and adding residual connection between the hidden layers; training the long-term and short-term memory neural network model by adopting a mean square error as a loss function and adopting a back propagation algorithm and an ADam optimization algorithm to obtain a training model;
step B2: model verification, namely inputting a verification set into a training model to obtain a predicted value, and calculating an evaluation index according to the predicted value and a real label;
step B3: the model tuning, specifically, tuning the training model according to the evaluation index, wherein the tuning comprises adjusting an input layer, a hidden layer, an input layer node and a hidden layer node of the training model, and selecting the optimal combination of the input layer, the hidden layer, the input layer node and the hidden layer node by using a cross verification mode to obtain a prediction model;
step B4: and data prediction, namely inputting the test set into a prediction model, and calculating to obtain a second prediction result through the forward propagation process of the prediction model.
8. The carbon market risk early warning system of claim 1, wherein in the early warning module, the early warning module continuously monitors feedback of the user and if the user does not take any action after receiving the early warning, the early warning is sent to the user again.
9. The carbon market risk early warning system of claim 1, further comprising a data visualization module for presenting the risk assessment results to a user in real time.
10. A carbon market risk early warning method based on big data, characterized in that the carbon market risk early warning method is realized by applying the carbon market risk early warning system according to any one of claims 1-9, and the steps of the method are as follows:
step S1: the data acquisition module acquires first transaction data and transmits the first transaction data to the data preprocessing module;
step S2: the data cleaning method comprises the steps of processing missing values, abnormal values and repeated values in first transaction data, then carrying out feature selection and normalization processing on the first transaction data to obtain second transaction data, and transmitting the second transaction data to a risk assessment module;
step S3: the risk assessment is specifically that second transaction data are input into a risk assessment model in a risk assessment module to obtain a risk assessment result, and the risk assessment result is transmitted to an early warning module;
step S4: and the risk early warning module detects whether a risk assessment result exceeds a preset risk threshold value, and when the risk assessment result exceeds the risk threshold value, the early warning module sends out early warning to a user, otherwise, the early warning module does not send out early warning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310741848.9A CN116777213B (en) | 2023-06-21 | Carbon transaction market risk early warning system and method based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310741848.9A CN116777213B (en) | 2023-06-21 | Carbon transaction market risk early warning system and method based on big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116777213A true CN116777213A (en) | 2023-09-19 |
| CN116777213B CN116777213B (en) | 2024-11-08 |
Family
ID=
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117455245A (en) * | 2023-12-22 | 2024-01-26 | 赛飞特工程技术集团有限公司 | Intelligent risk assessment system for enterprise safety production |
| CN117689219A (en) * | 2024-02-04 | 2024-03-12 | 江西科技学院 | Sport equipment security evaluation system based on machine learning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110929926A (en) * | 2019-11-18 | 2020-03-27 | 西北工业大学 | Short-term explosion passenger flow prediction method based on long and short-term memory network and random forest |
| AU2020100709A4 (en) * | 2020-05-05 | 2020-06-11 | Bao, Yuhang Mr | A method of prediction model based on random forest algorithm |
| CN113159615A (en) * | 2021-05-10 | 2021-07-23 | 麦荣章 | Intelligent information security risk measuring system and method for industrial control system |
| CN113505983A (en) * | 2021-07-07 | 2021-10-15 | 广东电网有限责任公司 | Energy industry chain monitoring and early warning system |
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110929926A (en) * | 2019-11-18 | 2020-03-27 | 西北工业大学 | Short-term explosion passenger flow prediction method based on long and short-term memory network and random forest |
| AU2020100709A4 (en) * | 2020-05-05 | 2020-06-11 | Bao, Yuhang Mr | A method of prediction model based on random forest algorithm |
| CN113159615A (en) * | 2021-05-10 | 2021-07-23 | 麦荣章 | Intelligent information security risk measuring system and method for industrial control system |
| CN113505983A (en) * | 2021-07-07 | 2021-10-15 | 广东电网有限责任公司 | Energy industry chain monitoring and early warning system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117455245A (en) * | 2023-12-22 | 2024-01-26 | 赛飞特工程技术集团有限公司 | Intelligent risk assessment system for enterprise safety production |
| CN117689219A (en) * | 2024-02-04 | 2024-03-12 | 江西科技学院 | Sport equipment security evaluation system based on machine learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109977151B (en) | Data analysis method and system | |
| Barak et al. | Forecasting energy consumption using ensemble ARIMA–ANFIS hybrid algorithm | |
| CN110659744B (en) | Training event prediction model, and method and device for evaluating operation event | |
| CN111339433B (en) | Information recommendation method and device based on artificial intelligence and electronic equipment | |
| Efstratiadis et al. | One decade of multi-objective calibration approaches in hydrological modelling: a review | |
| US7702482B2 (en) | Dependency structure from temporal data | |
| Huber et al. | Should I stay or should I go? A latent threshold approach to large‐scale mixture innovation models | |
| CN110503531A (en) | The dynamic social activity scene recommended method of timing perception | |
| CN106327240A (en) | Recommendation method and recommendation system based on GRU neural network | |
| CN113536097B (en) | Recommendation method and device based on automatic feature grouping | |
| CN112651534B (en) | Method, device and storage medium for predicting resource supply chain demand | |
| CN111611488A (en) | Information recommendation method and device based on artificial intelligence and electronic equipment | |
| CN115249081A (en) | Object type prediction method and device, computer equipment and storage medium | |
| CN105491599A (en) | Novel regression system for predicting LTE network performance indexes | |
| CN105279692A (en) | Financial information technology system performance prediction method and apparatus | |
| CN115114128A (en) | Satellite health state evaluation system and evaluation method | |
| CN116777213B (en) | Carbon transaction market risk early warning system and method based on big data | |
| Wu et al. | Symphony in the latent space: Provably integrating high-dimensional techniques with non-linear machine learning models | |
| CN115333957B (en) | Service flow prediction method and system based on user behavior and enterprise service characteristics | |
| CN116777213A (en) | Carbon transaction market risk early warning system and method based on big data | |
| CN115510948A (en) | Block chain fishing detection method based on robust graph classification | |
| JP2009536412A (en) | Method and apparatus for performing real-time audience estimation and commercial selection suitable for targeted advertising | |
| CN114418701A (en) | Method and device for generating recommendation list, electronic equipment and storage medium | |
| Bidyuk et al. | The algorithm for predicting the cryptocurrency rate taking into account the influence of posts of a group of famous people in social networks | |
| Almohammadi | Quality prediction model based on artificial neural networks for mobile 3D video streaming |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |