CN114118562A - Flow prediction method, model training method and device and electronic equipment - Google Patents

Flow prediction method, model training method and device and electronic equipment Download PDF

Info

Publication number
CN114118562A
CN114118562A CN202111390340.6A CN202111390340A CN114118562A CN 114118562 A CN114118562 A CN 114118562A CN 202111390340 A CN202111390340 A CN 202111390340A CN 114118562 A CN114118562 A CN 114118562A
Authority
CN
China
Prior art keywords
data
model
detection point
flow
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111390340.6A
Other languages
Chinese (zh)
Inventor
张硕
田伦
杨敬
杨胜文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202111390340.6A priority Critical patent/CN114118562A/en
Publication of CN114118562A publication Critical patent/CN114118562A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/04Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06Energy or water supply

Landscapes

  • Business, Economics & Management (AREA)
  • Engineering & Computer Science (AREA)
  • Economics (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Marketing (AREA)
  • General Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • Tourism & Hospitality (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Water Supply & Treatment (AREA)
  • Development Economics (AREA)
  • Game Theory and Decision Science (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The disclosure provides a flow prediction method, a model training device and electronic equipment, and relates to the technical field of artificial intelligence, in particular to the fields of Internet of things, industrial big data, deep learning and the like. The specific implementation scheme is as follows: acquiring detection point data detected at different detection points; the detection point data correlated in time is used as input data and is respectively input to each intermediate model in the plurality of intermediate models, so that each intermediate model outputs a flow predicted value; and jointly inputting the flow predicted values output by each intermediate model into the fusion model so as to output flow predicted results by the fusion model.

Description

Flow prediction method, model training method and device and electronic equipment
Technical Field
The disclosure relates to the technical field of artificial intelligence, in particular to the fields of internet of things, industrial big data, deep learning and the like, and specifically relates to a flow prediction method, a model training method, a device and electronic equipment.
Background
For a hydropower station, water is a production raw material, power generation is a main economic benefit, the warehousing flow is accurately predicted, and flood prevention and reasonable arrangement of planned scheduling of the hydropower station can be facilitated.
Disclosure of Invention
The disclosure provides a flow prediction method, a model training method, a device and electronic equipment.
According to an aspect of the present disclosure, there is provided a traffic prediction method, including: acquiring detection point data detected at different detection points; the detection point data correlated in time is used as input data and is respectively input to each intermediate model in a plurality of intermediate models, so that each intermediate model outputs a flow predicted value; and jointly inputting the flow prediction values output by each intermediate model into a fusion model so that the fusion model outputs a flow prediction result.
According to another aspect of the present disclosure, there is provided a model training method, including: acquiring sample data, wherein the sample data comprises detection point data which are obtained by detecting at different detection points and are related in time; respectively inputting the sample data into each intermediate model of a plurality of intermediate models to be trained so that each intermediate model outputs a flow predicted value corresponding to the sample data; adjusting the model parameters of each intermediate model according to the flow predicted value corresponding to the sample data; jointly inputting the flow predicted values output by each intermediate model into a fusion model to be trained so that the fusion model to be trained outputs a flow predicted result corresponding to the sample data; adjusting the model parameters of the fusion model according to the flow prediction result corresponding to the sample data; the trained intermediate model and the trained fusion model are applied to the flow prediction method.
According to another aspect of the present disclosure, there is provided a flow prediction apparatus including: the first acquisition module is used for acquiring detection point data detected at different detection points; the first prediction module is used for inputting the detection point data correlated in time as input data to each intermediate model in a plurality of intermediate models so that each intermediate model outputs a flow prediction value; and the second prediction module is used for inputting the flow prediction value output by each intermediate model into the fusion model together so that the fusion model outputs a flow prediction result.
According to another aspect of the present disclosure, there is provided a model training apparatus including: the second acquisition module is used for acquiring sample data, wherein the sample data comprises detection point data which are obtained by detection at different detection points and are related in time; the third prediction module is used for respectively inputting the sample data into each of a plurality of intermediate models to be trained so that each intermediate model outputs a flow prediction value corresponding to the sample data; the first adjusting module is used for adjusting the model parameters of each intermediate model according to the flow predicted value corresponding to the sample data; the fourth prediction module is used for jointly inputting the flow prediction value output by each intermediate model into a fusion model to be trained so that the fusion model to be trained outputs a flow prediction result corresponding to the sample data; the second adjusting module is used for adjusting the model parameters of the fusion model according to the flow prediction result corresponding to the sample data; the trained intermediate model and the trained fusion model are applied to the flow prediction method.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a flow prediction method or a model training method as described above.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the flow prediction method or the model training method as described above.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a flow prediction method or a model training method as described above.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 schematically illustrates an exemplary system architecture to which a traffic prediction method or model training method and apparatus may be applied, according to an embodiment of the disclosure;
FIG. 2 schematically illustrates a flow diagram of a traffic prediction method according to an embodiment of the disclosure;
FIG. 3 schematically illustrates a flow chart of a model training method according to an embodiment of the present disclosure;
FIG. 4 schematically illustrates a schematic diagram of traffic prediction based on a trained intermediate model and a trained fusion model according to an embodiment of the disclosure;
FIG. 5 schematically illustrates a block diagram of a flow prediction device according to an embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of a model training apparatus according to an embodiment of the present disclosure; and
FIG. 7 illustrates a schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all in accordance with the regulations of related laws and regulations, necessary security measures are taken, and the customs of the public order is not violated.
The water flow prediction method comprises the following steps: and estimating the warehousing flow for a period of time in the future based on manual experience. And predicting the warehousing flow based on an autoregressive and stepwise regression method. And predicting the warehousing traffic based on a single machine learning model or a deep learning model, and the like.
The inventor finds that the prediction method of the warehousing flow of most hydropower stations adopts principal component analysis on the detected point data and uses a stepwise regression, autoregressive and time series trend analysis method or a single neural network model for prediction in the process of realizing the concept disclosed by the invention. The method models can obtain better prediction results under the conditions of strong regularity and strong periodicity, but are difficult to effectively predict in the environment with rapid climate change in extreme weather, and have weak sensitivity to environmental change and poor generalization capability.
Fig. 1 schematically illustrates an exemplary system architecture to which a traffic prediction method or model training method and apparatus may be applied, according to an embodiment of the present disclosure.
It should be noted that fig. 1 is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios. For example, in another embodiment, an exemplary system architecture to which the traffic prediction method and apparatus or the model training method and apparatus may be applied may include a terminal device, but the terminal device may implement the traffic prediction method and apparatus or the model training method and apparatus provided in the embodiments of the present disclosure without interacting with a server.
As shown in fig. 1, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired and/or wireless communication links, and so forth.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have installed thereon various communication client applications, such as a knowledge reading application, a web browser application, a search application, an instant messaging tool, a mailbox client, and/or social platform software, etc. (by way of example only).
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (for example only) providing support for content browsed by the user using the terminal devices 101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device. The Server may be a cloud Server, which is also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service extensibility in a traditional physical host and a VPS service ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server incorporating a blockchain.
It should be noted that the traffic prediction method or the model training method provided by the embodiments of the present disclosure may be generally executed by the terminal device 101, 102, or 103. Accordingly, the traffic prediction device or the model training device provided by the embodiment of the present disclosure may also be disposed in the terminal device 101, 102, or 103.
Alternatively, the traffic prediction method or the model training method provided by the embodiments of the present disclosure may also be generally performed by the server 105. Accordingly, the traffic prediction device or the model training device provided by the embodiments of the present disclosure may be generally disposed in the server 105. The traffic prediction method or the model training method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the traffic prediction apparatus or the model training apparatus provided in the embodiments of the present disclosure may also be disposed in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
For example, when traffic prediction is required, the terminal devices 101, 102, and 103 may obtain detection point data detected at different detection points, and then send the obtained detection point data to the server 105, and the server 105 inputs the detection point data correlated in time as input data to each of the plurality of intermediate models, so that each of the intermediate models outputs a traffic prediction value, and inputs the traffic prediction values output by each of the intermediate models into the fusion model together, so that the fusion model outputs a traffic prediction result. Or the server or server cluster capable of communicating with the terminal devices 101, 102, 103 and/or the server 105 analyzes the detection point data and implements the fusion model to output the flow prediction result.
For example, when model training is required, the terminal devices 101, 102, and 103 may acquire sample data including temporally related detection point data detected at different detection points. Then, the acquired sample data is sent to the server 105, the server 105 inputs the sample data into each of the plurality of intermediate models to be trained respectively, so that each intermediate model outputs a flow predicted value corresponding to the sample data, model parameters of each intermediate model are adjusted according to the flow predicted value corresponding to the sample data, the flow predicted values output by each intermediate model are input into the fusion model to be trained together, so that the fusion model to be trained outputs a flow predicted result corresponding to the sample data, and the model parameters of the fusion model are adjusted according to the flow predicted result corresponding to the sample data. Or by a server or server cluster capable of communicating with the terminal devices 101, 102, 103 and/or the server 105, analyzes the sample data and implements model training.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Fig. 2 schematically shows a flow chart of a traffic prediction method according to an embodiment of the present disclosure.
As shown in fig. 2, the method includes operations S210 to S230.
In operation S210, detected checkpoint data detected at different checkpoint is acquired.
In operation S220, the temporally associated detection point data is input as input data to each of the plurality of intermediate models, respectively, so that each intermediate model outputs a flow prediction value.
In operation S230, the flow prediction values output by each intermediate model are collectively input into the fusion model so that the fusion model outputs a flow prediction result.
According to embodiments of the present disclosure, the flow may include water flow, snow flow, and the like. The detection point may include at least one of a telemetry station, a warehousing traffic detection point, an environmental observation station, and the like, and may not be limited thereto. The environmental observatory can comprise, for example, a weather station. The telemetry station may include at least one telemetry station disposed at a distance from the hydroelectric power station within a first predetermined range. The telemetry station may detect rainfall data in the vicinity of the hydroelectric power station. For example, by having telemetry stations distributed over a plurality of areas in the vicinity of a hydroelectric power station, it is possible to record the rainfall at different areas of the vicinity every 1 hour for 1 time. The warehousing flow detection point may comprise a detection point provided in at least one water flow warehousing area of the hydropower station. The warehousing flow detection point can detect warehousing flow entering the hydropower station. For example, the warehousing flow detection point may record warehousing flows entering the hydropower station 1 time every 1 hour. The environment observation stations can comprise at least one environment observation station which is arranged in a first preset range of distance from the hydropower station, at least one remotely arranged environment observation station and the like. The environment observation platform can detect the environment observation data of a preset area near the hydropower station. For example, through the environment observation station, the environment observation data of a plurality of measurement sections such as real-time daily air temperature, wind direction and wind speed of a preset area near the hydropower station can be detected and recorded. The weather station may include at least one weather station disposed at a distance within a first predetermined range from the hydroelectric power station, or may include at least one weather station disposed remotely, or the like. The weather station can detect weather forecast data of rainfall, snowfall and the like in a preset area near the hydropower station. For example, weather forecast data for the current day in a preset area near the hydropower station may be acquired from weather stations each day. Accordingly, the detection point data can be obtained to include at least one of rainfall, warehousing traffic, environmental observation data, meteorological prediction data and the like.
According to an embodiment of the present disclosure, the temporally associated detection point data may include, for example, at least two of rainfall, warehousing traffic, environmental observation data, and the like detected at the same time or within the same time period.
According to an embodiment of the present disclosure, the plurality of intermediate models may include at least two of xgboost (extreme Gradient boosting), LightGBM, LSTM (Long Short Term Memory Network, Long-Short Term Memory Network), LSTNet (Long-and-Short Term Temporal Patterns with Deep Neural Networks), Prophet, LR (Logistic Regression), and the like, and is not limited thereto. XGboost is an optimized distributed gradient enhancement library, LightGBM is a rapid, distributed and high-performance gradient enhancement framework based on a decision tree algorithm, and Prophet is an open-source time sequence prediction algorithm. The fusion model may include a DNN (Deep Neural Network) model and the like, which are not limited herein.
According to an embodiment of the present disclosure, temporally correlated data detected at different detection points may be input into different intermediate models, respectively. Then, the output results of the intermediate models, i.e., the predicted flow values predicted for the actually associated prediction data, are used as the input of the fusion model. And predicting through a fusion model to obtain a flow prediction result.
Through the embodiment of the disclosure, the detection point data acquired at different acquisition points is predicted according to the time correlation, so that the method can cope with various extreme and rapidly-changing environments, and has strong normalization capability in application. And the flow result prediction obtained by combining the plurality of intermediate models and the fusion model prediction has higher accuracy.
The method shown in fig. 2 is further described below with reference to specific embodiments.
According to the embodiment of the disclosure, different detection points are respectively used for detecting and obtaining different types of detection point data. The traffic prediction method may further include: and carrying out feature extraction on the detection point data of different types to obtain one or more feature data. The input of the temporally associated detection point data as input data to each of the plurality of intermediate models, respectively, may include: the temporally correlated detection point data and one or more characteristic data are input as input data to each intermediate model, respectively.
According to the embodiment of the disclosure, after rainfall data of different regions are obtained according to detection of remote stations of different regions, environment observation data of different regions are obtained according to detection of environment observation stations of different regions, weather forecast data of different regions are detected according to weather stations, historical warehousing flow data of different detection points are obtained according to detection of different warehousing flow detection points, and the like, data characteristics can be fully mined for the data, for example, characteristics such as weather data, environment data, runoff data and the like can be obtained. At least one of rainfall data, warehousing traffic data, weather data, environmental data, runoff data and the like which are related in time can be reasonably processed and presumed, and the traffic predicted value used as the input of the fusion model is obtained by predicting by using intermediate models such as XGboost, LightGBM, Prophet, LR, LSTNet and the like. Then, the flow prediction values output by the intermediate models can be weighted and the like by combining with the CNN and other fusion models to obtain flow prediction results, so that the purpose of predicting the inflow flow of the hydropower station is achieved.
Through the embodiment of the disclosure, the combination of the detection point data and the characteristic data is used as the input of the intermediate model, and the obtained flow prediction value can have higher accuracy and reference value. The method has great significance for realizing water and electricity coordination and ensuring reasonable power grid dispatching.
According to an embodiment of the present disclosure, the different detection points may include at least two of: a telemetry station, a warehouse entry flow detection point, an environment observation station, but not limited thereto.
Through the embodiment of the disclosure, the detection points aiming at different detection data can be set in different regions, and the data detected based on the detection points is used as the input of the model, so that the richness of the input characteristics of the model is favorably improved, and the accuracy of the prediction result of the model can be effectively improved.
According to the embodiment of the disclosure, the detection point data detected at the remote station comprises rainfall in different areas; detecting point data obtained by detecting at the warehousing flow detecting point comprises warehousing flows at different time periods; the detection point data detected at the environment observation station comprises at least one of the following data: meteorological forecast data, air temperature observation data, wind direction observation data, wind speed observation data and day and night time data.
Through the embodiment of the disclosure, different types of data obtained by detection can be detected based on different detection points, so that the richness of the input features of the model is favorably improved, and the accuracy of the model prediction result can be effectively improved.
According to an embodiment of the present disclosure, performing feature extraction on different types of detection point data, and obtaining one or more feature data may include: and extracting convergence characteristic data according to the detection point data obtained by detection at the remote sensing station and the detection point data obtained by detection at the warehousing flow detection point. The confluence characteristic data is used for representing the relation between the warehousing flow and the rainfall.
According to the embodiment of the disclosure, by analyzing rainfall, warehousing flow, environmental observation data, meteorological prediction data and the like, a flood process can be formed after each rainfall, and the warehousing flow in the whole process can present an analysis result of a bimodal wave pattern. The analysis result can represent that the period and the peak value of the water flow waveform of the flood warehousing have a certain relation with the rainfall size and the duration. Based on the data, the convergence characteristic data used for representing the relation between the warehousing flow and the rainfall can be calculated and obtained by using an LR model and the like according to the detection point data obtained by detection at the remote station and the detection point data obtained by detection at the warehousing flow detection point.
Through the embodiment of the disclosure, the convergence characteristic data of different dimensions can be extracted according to the detection point data, and the convergence characteristic data is introduced, so that the richness of the input characteristics of the model is favorably improved, and the accuracy of the model prediction result can be effectively improved.
According to an embodiment of the present disclosure, performing feature extraction on different types of detection point data, and obtaining one or more feature data may include: and extracting statistical characteristic data of different time periods according to detection point data obtained by detecting the warehousing flow detection points. The statistical characteristic data is used for representing the historical change situation of the data.
According to an embodiment of the present disclosure, the statistical data may include at least one of a mean, a variance, a most significant, a median, a most significant ratio, an increment, a kurtosis, a skewness, etc. of the warehoused water flows over different window times counted from the hydropower station warehoused flow data. The statistical characteristics may reflect historical changes in the flow data being put into the warehouse.
Through the embodiment of the disclosure, the statistical characteristic data corresponding to the data at different time periods can be extracted according to the warehousing flow data, the statistical characteristic data is introduced, the richness of the input characteristics of the model is favorably improved, and the accuracy of the model prediction result can be effectively improved.
According to an embodiment of the present disclosure, performing feature extraction on different types of detection point data, and obtaining one or more feature data may include: and fitting the detection point data obtained by detecting the warehousing flow detection points by using a time sequence model, and extracting time sequence characteristic data. The statistical characteristic data is used for representing the historical change situation of the data.
According to embodiments of the present disclosure, the timing model may include a Prophet timing model. The warehousing traffic can be fitted according to the Prophet time sequence model, the obtained time sequence characteristics are predicted, and the time sequence rule of the warehousing traffic data can be captured.
Through the embodiment of the disclosure, corresponding time sequence data can be obtained according to the warehouse entry flow data, and the time sequence characteristic data is introduced, so that the richness of the input characteristics of the model is favorably improved, and the accuracy of the model prediction result can be effectively improved.
According to an embodiment of the present disclosure, performing feature extraction on different types of detection point data, and obtaining one or more feature data may include: and extracting the day time characteristic and the night time characteristic according to day and night time data detected by the environment observation platform.
According to the embodiment of the disclosure, the long characteristics such as day, night and the like, and the characteristics such as sunset time, sunrise time and the like can be acquired from the natural seasonal law.
By the embodiment of the disclosure, long characteristics such as day and night are introduced, the warehousing flow can be calculated according to different tide attractive forces in the day and the night, the richness of the input characteristics of the model is favorably improved, and the accuracy of the model prediction result can be effectively improved.
FIG. 3 schematically shows a flow chart of a model training method according to an embodiment of the present disclosure.
As shown in fig. 3, the method includes operations S310 to S350, by which the trained intermediate model and the trained fusion model may be applied to the above-described traffic prediction method.
In operation S310, sample data is acquired, where the sample data includes temporally related detection point data detected at different detection points.
In operation S320, sample data is respectively input to each of the plurality of intermediate models to be trained, so that each intermediate model outputs a predicted flow value corresponding to the sample data.
In operation S330, model parameters of each intermediate model are adjusted according to the traffic prediction value corresponding to the sample data.
In operation S340, the flow prediction values output by each intermediate model are commonly input into the fusion model to be trained, so that the fusion model to be trained outputs a flow prediction result corresponding to the sample data.
In operation S350, model parameters of the fusion model are adjusted according to the traffic prediction result corresponding to the sample data.
According to an embodiment of the present disclosure, XGBoost, LightGBM, LSTM, LSTNet, etc. may be used as intermediate models to model detection point data associated at all times, respectively. Then, SHAP (SHAPLey Additive empirical analysis, a model after-exPlanation method, which can explain complex machine learning models) algorithm can be used for analyzing and sequencing the importance of different models, screening more important data characteristics, and continuously iterating the models. After each model converges, stacking model fusion can be performed on all models, that is, the predicted values of different models can be used as data characteristics, a DNN neural network is trained, and the prediction result of the DNN is used as a flow prediction result. Stacking is a method of ensemble learning.
According to the embodiment of the disclosure, the trained intermediate model and the trained fusion model can be deployed in a system of the hydropower station inflow, and the hydropower station inflow can be predicted in real time based on the deployed models. For example, the hydropower station inflow rate can be predicted in real time for a future week.
Through the embodiment of the disclosure, various AI models of different types are used for fusion, so that the generalization capability and the prediction effect of the models are greatly improved, and the accuracy of the prediction result of the models is improved.
Fig. 4 schematically illustrates a schematic diagram of traffic prediction based on a trained intermediate model and a trained fusion model according to an embodiment of the disclosure.
As shown in fig. 4, checkpoint data 410 may include rainfall 411, warehousing traffic 412, and environmental observation data 413. The feature data 420 may include confluence feature data 421, statistical feature data 422, timing feature data 423, and circadian time data 424. The intermediate models 430 may include an XGBoost model 431, a LightGBM model 432, an LSTM model 433, and an LSTNet model 434. By taking the detection point data 410 and the feature data 420 as inputs to the respective models in the intermediate model 430, corresponding flow prediction values can be obtained. The flow prediction values output by the models in the intermediate model 430 are used as the input of the CNN 440, and a flow prediction result 450 can be obtained.
According to the embodiment of the disclosure, for the model deployed in the system of the hydropower station inflow, the feature importance analysis can be performed by adjusting the parameters of the equipment device and recording in real time. When the model predicts a reliable result, the value of the characteristic value can be adjusted, and the influence of the change of different characteristic values on the prediction result can be observed, so that the model is favorable for guiding the data collection work. For example, rainfall data in different regions have different importance on the prediction result, and analysis can find that the rainfall data in which regions are more effective, so that more telemetry stations can be arranged in the regions for data collection, and invalid regions can reduce resource and manual release.
Through the embodiment of the disclosure, various AI models of different types are fused, so that the generalization capability and the prediction effect of the models are greatly improved, data are fully mined in characteristics, and the data value is exerted as much as possible. In addition, the method can not only accurately predict the water inflow of the hydropower station, but also perform attribution analysis and guidance on data information collected by different devices in different regions, so that data more useful for a prediction result can be collected nearby the hydropower station.
Fig. 5 schematically illustrates a block diagram of a flow prediction apparatus according to an embodiment of the present disclosure.
As shown in fig. 5, the traffic prediction apparatus 500 includes a first obtaining module 510, a first prediction module 520, and a second prediction module 530.
The first obtaining module 510 is configured to obtain detection point data obtained by detecting at different detection points.
A first prediction module 520, configured to input the temporally associated detection point data as input data to each of the plurality of intermediate models, respectively, so that each intermediate model outputs a flow prediction value.
And a second prediction module 530, configured to jointly input the flow prediction values output by each intermediate model into the fusion model, so that the fusion model outputs a flow prediction result.
According to the embodiment of the disclosure, different detection points are respectively used for detecting and obtaining different types of detection point data. The flow rate prediction apparatus further includes: and the characteristic extraction module is used for extracting the characteristics of the detection point data of different types to obtain one or more characteristic data. The first prediction module includes: and the input unit is used for respectively inputting the detection point data and one or more kinds of characteristic data which are correlated in time into each intermediate model as input data.
According to an embodiment of the present disclosure, the different detection points include at least two of: a remote sensing station, a warehousing flow detection point and an environment observation platform.
According to the embodiment of the disclosure, the detection point data detected at the remote station comprises rainfall in different areas; detecting point data obtained by detecting at the warehousing flow detecting point comprises warehousing flows at different time periods; the detection point data detected at the environment observation station comprises at least one of the following data: meteorological forecast data, air temperature observation data, wind direction observation data, wind speed observation data and day and night time data.
According to an embodiment of the present disclosure, the feature extraction module includes a first extraction unit.
The first extraction unit is used for extracting convergence characteristic data according to detection point data obtained by detection at a remote sensing station and detection point data obtained by detection at a warehousing flow detection point, wherein the convergence characteristic data is used for representing the relation between warehousing flow and rainfall.
According to an embodiment of the present disclosure, the feature extraction module includes a second extraction unit.
And the second extraction unit is used for extracting statistical characteristic data of different time periods according to detection point data obtained by detecting the warehousing flow detection points, wherein the statistical characteristic data are used for representing the historical change condition of the data.
According to an embodiment of the present disclosure, the feature extraction module includes a third extraction unit.
And the third extraction unit is used for fitting the detection point data obtained by detecting the warehousing flow detection points by using the time sequence model and extracting time sequence characteristic data, wherein the statistical characteristic data is used for representing the historical change condition of the data.
According to an embodiment of the present disclosure, the feature extraction module includes a fourth extraction unit.
And the fourth extraction unit is used for extracting the day time characteristic and the night time characteristic according to the day and night time data detected by the environment observation platform.
FIG. 6 schematically shows a block diagram of a model training apparatus according to an embodiment of the present disclosure.
As shown in fig. 6, the model training apparatus includes a second obtaining module 610, a third predicting module 620, a first adjusting module 630, a fourth predicting module 640, and a second adjusting module 650.
The second obtaining module 610 is configured to obtain sample data, where the sample data includes detection point data associated in time and detected at different detection points.
The third prediction module 620 is configured to input the sample data to each of the plurality of intermediate models to be trained, so that each intermediate model outputs a predicted flow value corresponding to the sample data.
A first adjusting module 630, configured to adjust the model parameter of each intermediate model according to the predicted flow value corresponding to the sample data.
The fourth prediction module 640 is configured to input the flow prediction value output by each intermediate model into the fusion model to be trained together, so that the fusion model to be trained outputs a flow prediction result corresponding to the sample data.
And a second adjusting module 650, configured to adjust the model parameters of the fusion model according to the traffic prediction result corresponding to the sample data.
The trained intermediate model and the trained fusion model are applied to the flow prediction method.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a flow prediction method or a model training method as described above.
According to an embodiment of the present disclosure, a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to execute a flow prediction method or a model training method as described above.
According to an embodiment of the present disclosure, a computer program product comprising a computer program which, when executed by a processor, implements a flow prediction method or a model training method as described above.
FIG. 7 illustrates a schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 7, the device 700 comprises a computing unit 701, which may perform various suitable actions and processes according to a computer program stored in a Read Only Memory (ROM)702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
Various components in the device 700 are connected to the I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, or the like; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
Computing unit 701 may be a variety of general purpose and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 701 performs the respective methods and processes described above, such as the flow prediction method or the model training method. For example, in some embodiments, the flow prediction method or the model training method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 700 via ROM 702 and/or communications unit 709. When loaded into RAM 703 and executed by the computing unit 701, may perform one or more steps of the flow prediction method or the model training method described above. Alternatively, in other embodiments, the computing unit 701 may be configured to perform a traffic prediction method or a model training method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (21)

1. A traffic prediction method, comprising:
acquiring detection point data detected at different detection points;
the detection point data correlated in time is used as input data and is respectively input to each intermediate model in a plurality of intermediate models, so that each intermediate model outputs a flow predicted value; and
and jointly inputting the flow predicted values output by each intermediate model into a fusion model so that the fusion model outputs flow predicted results.
2. The method according to claim 1, wherein the different detection points are respectively used for detecting and obtaining different types of detection point data;
the method further comprises the following steps: carrying out feature extraction on the detection point data of different types to obtain one or more feature data;
wherein the respectively inputting the temporally associated detection point data as input data to each of the plurality of intermediate models comprises:
and respectively inputting the detection point data and the one or more characteristic data which are correlated in time into each intermediate model as input data.
3. The method according to claim 1 or 2, wherein the different detection points comprise at least two of: a remote sensing station, a warehousing flow detection point and an environment observation platform.
4. The method of claim 3, wherein the detected checkpoint data at the telemetry station includes rainfall in different regions; detecting point data obtained by detecting at the warehousing flow detecting points comprises warehousing flows at different time periods; the detection point data detected at the environment observation station comprises at least one of the following data: meteorological forecast data, air temperature observation data, wind direction observation data, wind speed observation data and day and night time data.
5. The method of claim 3, wherein the feature extraction of the different types of detection point data, obtaining one or more feature data comprises:
and extracting convergence characteristic data according to the detection point data obtained by detection at the remote sensing station and the detection point data obtained by detection at the warehousing flow detection point, wherein the convergence characteristic data is used for representing the relation between the warehousing flow and the rainfall.
6. The method of claim 3, wherein the feature extraction of the different types of detection point data, obtaining one or more feature data comprises:
and extracting statistical characteristic data of different time periods according to detection point data obtained by detecting the warehousing flow detection points, wherein the statistical characteristic data are used for representing historical data change conditions.
7. The method of claim 3, wherein the feature extraction of the different types of detection point data, obtaining one or more feature data comprises:
and fitting the detection point data obtained by detecting the warehousing flow detection points by using a time sequence model, and extracting time sequence characteristic data, wherein the statistical characteristic data is used for representing the historical change condition of the data.
8. The method of claim 3, wherein the feature extraction of the different types of detection point data, obtaining one or more feature data comprises:
and extracting day time characteristics and night time characteristics according to day and night time data detected by the environment observation platform.
9. A model training method, comprising:
acquiring sample data, wherein the sample data comprises detection point data which are obtained by detecting at different detection points and are related in time;
respectively inputting the sample data into each intermediate model of a plurality of intermediate models to be trained so that each intermediate model outputs a flow predicted value corresponding to the sample data;
adjusting the model parameters of each intermediate model according to the flow predicted value corresponding to the sample data;
jointly inputting the flow predicted values output by each intermediate model into a fusion model to be trained so that the fusion model to be trained outputs a flow predicted result corresponding to the sample data; and
adjusting the model parameters of the fusion model according to the flow prediction result corresponding to the sample data;
wherein the trained intermediate model and the trained fusion model are applied to the traffic prediction method according to any one of claims 1 to 8.
10. A flow prediction device comprising:
the first acquisition module is used for acquiring detection point data detected at different detection points;
the first prediction module is used for inputting the detection point data correlated in time as input data to each intermediate model in a plurality of intermediate models so that each intermediate model outputs a flow prediction value; and
and the second prediction module is used for inputting the flow prediction value output by each intermediate model into the fusion model together so that the fusion model can output a flow prediction result.
11. The apparatus of claim 10, wherein the different detection points are respectively used for detecting and obtaining different types of detection point data;
the device further comprises: the characteristic extraction module is used for extracting the characteristics of the detection point data of different types to obtain one or more characteristic data;
wherein the first prediction module comprises:
and the input unit is used for respectively inputting the detection point data and the one or more types of feature data which are correlated in time into each intermediate model as input data.
12. The apparatus of claim 10 or 11, wherein the different detection points comprise at least two of: a remote sensing station, a warehousing flow detection point and an environment observation platform.
13. The apparatus of claim 12, wherein the detected checkpoint data at the telemetry station includes rainfall at different regions; detecting point data obtained by detecting at the warehousing flow detecting points comprises warehousing flows at different time periods; the detection point data detected at the environment observation station comprises at least one of the following data: meteorological forecast data, air temperature observation data, wind direction observation data, wind speed observation data and day and night time data.
14. The apparatus of claim 12, wherein the feature extraction module comprises:
and the first extraction unit is used for extracting convergence characteristic data according to the detection point data obtained by detection at the remote sensing station and the detection point data obtained by detection at the warehousing flow detection point, wherein the convergence characteristic data is used for representing the relation between the warehousing flow and the rainfall.
15. The apparatus of claim 12, wherein the feature extraction module comprises:
and the second extraction unit is used for extracting statistical characteristic data of different time periods according to detection point data obtained by detecting the warehousing flow detection points, wherein the statistical characteristic data are used for representing the historical change condition of the data.
16. The apparatus of claim 12, wherein the feature extraction module comprises:
and the third extraction unit is used for fitting the detection point data obtained by detecting the warehousing flow detection points by using a time sequence model and extracting time sequence characteristic data, wherein the statistical characteristic data is used for representing the historical change condition of the data.
17. The apparatus of claim 12, wherein the feature extraction module comprises:
and the fourth extraction unit is used for extracting the time characteristics of day time and night time according to the day and night time data detected by the environment observation platform.
18. A model training apparatus comprising:
the second acquisition module is used for acquiring sample data, wherein the sample data comprises detection point data which are obtained by detection at different detection points and are related in time;
the third prediction module is used for respectively inputting the sample data into each of a plurality of intermediate models to be trained so that each intermediate model outputs a flow prediction value corresponding to the sample data;
the first adjusting module is used for adjusting the model parameters of each intermediate model according to the flow predicted value corresponding to the sample data;
the fourth prediction module is used for jointly inputting the flow prediction value output by each intermediate model into a fusion model to be trained so that the fusion model to be trained outputs a flow prediction result corresponding to the sample data; and
the second adjusting module is used for adjusting the model parameters of the fusion model according to the flow prediction result corresponding to the sample data;
wherein the trained intermediate model and the trained fusion model are applied to the traffic prediction method according to any one of claims 1 to 8.
19. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8 or 9.
20. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any of claims 1-8 or 9.
21. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the method according to any one of claims 1-8 or claim 9.
CN202111390340.6A 2021-11-22 2021-11-22 Flow prediction method, model training method and device and electronic equipment Pending CN114118562A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111390340.6A CN114118562A (en) 2021-11-22 2021-11-22 Flow prediction method, model training method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111390340.6A CN114118562A (en) 2021-11-22 2021-11-22 Flow prediction method, model training method and device and electronic equipment

Publications (1)

Publication Number Publication Date
CN114118562A true CN114118562A (en) 2022-03-01

Family

ID=80439439

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111390340.6A Pending CN114118562A (en) 2021-11-22 2021-11-22 Flow prediction method, model training method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN114118562A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114781766A (en) * 2022-06-22 2022-07-22 长江水利委员会长江科学院 Hydrological information prediction method, device, equipment and storage medium for hydrological site

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114781766A (en) * 2022-06-22 2022-07-22 长江水利委员会长江科学院 Hydrological information prediction method, device, equipment and storage medium for hydrological site
CN114781766B (en) * 2022-06-22 2022-09-13 长江水利委员会长江科学院 Hydrological information prediction method, device, equipment and storage medium for hydrological site

Similar Documents

Publication Publication Date Title
CN109471783B (en) Method and device for predicting task operation parameters
CN107642347B (en) Shale gas fracturing underground accident prediction and early warning method and device
CN113283630A (en) Air quality prediction method, device, equipment and computer readable storage medium
CN112668773A (en) Method and device for predicting warehousing traffic and electronic equipment
CN115390161A (en) Precipitation prediction method and device based on artificial intelligence
CN113657467A (en) Model pre-training method and device, electronic equipment and storage medium
CN114494776A (en) Model training method, device, equipment and storage medium
CN114418093B (en) Method and device for training path characterization model and outputting information
CN114118562A (en) Flow prediction method, model training method and device and electronic equipment
CN114090601B (en) Data screening method, device, equipment and storage medium
CN114037059A (en) Pre-training model, model generation method, data processing method and data processing device
CN113723607A (en) Training method, device and equipment of space-time data processing model and storage medium
CN117332896A (en) New energy small time scale power prediction method and system for multilayer integrated learning
CN116937645A (en) Charging station cluster regulation potential evaluation method, device, equipment and medium
CN115641360A (en) Battery detection method and device based on artificial intelligence and electronic equipment
CN115169731A (en) Smart campus energy consumption prediction method, device, equipment and medium
CN115757075A (en) Task abnormity detection method and device, computer equipment and storage medium
CN115482436A (en) Training method and device for image screening model and image screening method
Chen et al. Machine learning-based anomaly detection of ganglia monitoring data in HEP Data Center
CN114692987A (en) Time sequence data analysis method, device, equipment and storage medium
CN112560987A (en) Image sample processing method, device, equipment, storage medium and program product
CN114860616B (en) Code detection method, device, equipment and storage medium
CN115294536B (en) Violation detection method, device, equipment and storage medium based on artificial intelligence
CN116796620A (en) Method, device, equipment and storage medium for predicting oil temperature of pumped storage unit
US20220392199A1 (en) Method and apparatus for training classification model and data classification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination