CN112990558B - Meteorological temperature and illumination prediction method based on deep migration learning - Google Patents

Meteorological temperature and illumination prediction method based on deep migration learning Download PDF

Info

Publication number
CN112990558B
CN112990558B CN202110220175.3A CN202110220175A CN112990558B CN 112990558 B CN112990558 B CN 112990558B CN 202110220175 A CN202110220175 A CN 202110220175A CN 112990558 B CN112990558 B CN 112990558B
Authority
CN
China
Prior art keywords
data
average
weather
meteorological
network module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110220175.3A
Other languages
Chinese (zh)
Other versions
CN112990558A (en
Inventor
毛永芳
王腾
魏铨
郭铭磊
柴毅
国祎晴
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University
Original Assignee
Chongqing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University filed Critical Chongqing University
Priority to CN202110220175.3A priority Critical patent/CN112990558B/en
Publication of CN112990558A publication Critical patent/CN112990558A/en
Application granted granted Critical
Publication of CN112990558B publication Critical patent/CN112990558B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/04Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/906Clustering; Classification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/048Activation functions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10Services
    • G06Q50/26Government or public services
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Human Resources & Organizations (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Tourism & Hospitality (AREA)
  • Economics (AREA)
  • Strategic Management (AREA)
  • Evolutionary Computation (AREA)
  • Marketing (AREA)
  • Development Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Databases & Information Systems (AREA)
  • Game Theory and Decision Science (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Educational Administration (AREA)
  • Primary Health Care (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application provides a weather temperature and illumination prediction method based on deep migration learning, which comprises the following specific steps: acquiring historical data recorded by each meteorological site in a target area, and performing data preprocessing on the historical data to obtain preprocessed data; preprocessing data by a clustering algorithm based on density peaks to classify weather detection points, and calculating a clustering center point of each class; training a respective spatial prediction neural network module by using each type of data, and training a respective prediction model for the type data obtained by each cluster according to the optimized spatial prediction neural network module; according to the fact that data samples exist in the target prediction area, different input and model structures are used for predicting the temperature and illumination in the target area, and the method is suitable for predicting the temperature and illumination when a small amount of sample data or zero sample data exist in the target prediction area.

Description

Meteorological temperature and illumination prediction method based on deep migration learning
Technical Field
The application relates to the field of weather prediction, in particular to a weather temperature and illumination prediction method based on deep migration learning.
Background
The changes of weather temperature and illumination have close relations to human life, agricultural production, social activities, academic research and the like, and people mainly take a professional weather station as a center for research aiming at weather prediction. With the continuous progress of scientific technology, meteorological data are gradually enriched, and higher requirements are put on space-time resolution and accuracy of weather prediction data. If weather factors greatly influence the efficiency of a photovoltaic power station, due to the limitation of the existing energy storage equipment, the photovoltaic power generation mainly adopts a mode of instant power generation and instant use, and in order to prevent the stability of a power grid from being influenced by power grid fluctuation caused by weather random change, accurate prediction of weather is needed to prevent power grid impact caused by weather change and improve energy management capability to bring higher energy efficiency. With the development and popularization of photovoltaic power generation, small or independent power stations will become an important energy source in the future, but for example, due to the difference of the distribution of detection points of meteorological stations and the characteristics of regional climatic environment, the current data density cannot meet the requirements, and how to predict weather in some regions where long-term and accurate meteorological data cannot be obtained is one of the important points of research in the field.
The rise of computer data mining technology provides a new thought for the research of weather prediction methods. The traditional mainstream method is generally divided into an optical flow method and a modeling method, wherein the optical flow method and the modeling method are used for predicting by analyzing weather map feature data and utilizing time domain changes in image data and structure and motion relations of space objects; the latter is more focused on the physical and chemical statistics, and parameter prediction is performed by constructing a physical motion model based on differential equations. Because of more influence factors of meteorological changes, the method is complex, has high experience dependence, is complex in calculation, long in calculation time and expensive in calculation cost. Meanwhile, because the artificial data processing can not well conduct predictive modeling on the missing part of the data, the space-time resolution is lower in a large-scale application scene under the condition of lower existing data density.
And the deep learning is used as a new modeling method which is emerging in the big data age, so that a large amount of data is utilized in a high-efficiency and deep manner. Features in big data can be automatically mined, and complex object models can be automatically learned in a data-driven mode. Compared with the traditional method, the method has high calculation efficiency, can quickly combine multimode information, and simultaneously has the capability of predicting and deducing the incompletely uncertain information, so that the defect of insufficient data density in the prior art can be overcome. Deep learning has achieved attention in various fields such as image processing, speech recognition, text classification, information security, etc., and applications in the field of weather prediction have also been growing. In the foreseeable upcoming internet of things era, multimedia data fusion has become an essential ring in the development of important weather data analysis in the future.
Disclosure of Invention
The application aims to provide a weather temperature and illumination prediction method based on deep migration learning, which can be used for predicting the temperature and sunlight when a small amount of sample data or zero sample data exists in a target prediction area.
The application aims at realizing the technical scheme by the following specific steps:
1) And (3) data acquisition: acquiring historical data recorded by each meteorological site in a target area, and performing data preprocessing on the historical data to obtain preprocessed data;
2) Data classification: classifying the meteorological detection points through a clustering algorithm based on density peaks and the preprocessing data of each meteorological site in the step 1), and calculating a clustering center point of each class;
3) And (3) constructing a model: training a respective spatial prediction neural network module by using each type of data, optimizing the spatial prediction neural network module by adopting an RMSprop algorithm, and training a respective prediction model for the type data obtained by each cluster according to the optimized spatial prediction neural network module;
4) Predicted temperature and illumination: if the predicted target area has sample data, matching types and a prediction model according to the sample data, performing Fine tuning of the prediction model, performing prediction on temperature and sunlight by an isolated spatial prediction network module, if the predicted target area has no sample data, selecting the type of a weather station and the prediction model which are nearest to the predicted target area, performing field adaptation on a prediction model construction field discriminator, and performing prediction by using the spatial network module.
Further, the specific steps of data acquisition in step 1) are as follows:
1-1) collecting average, maximum and minimum air temperatures recorded by each weather station in a target area, precipitation at 20-20 hours, average wind speed, sunshine hours, average relative humidity, average air pressure, longitude and latitude of the weather station and data collection time by using daily data units, and obtaining historical data;
1-2) replacing the missing value or the abnormal value of the average, the maximum and the minimum air temperature, the precipitation amount at 20-20 days, the average air speed, the sunshine hours, the average relative humidity and the average air pressure in the historical data with the average value of the missing value or the two values before and after the abnormal value or with the average value in the time range of 3-5 days;
1-3) adding a weather value to the historical data, wherein the weather value is 1 when the precipitation is more than 0, and 0 when the precipitation is equal to 0;
1-4) transforming longitude and latitude data of the weather station:
in the formula (1), x i And (3) withRespectively represent longitude, y before and after transformation i And->Respectively representing the latitude before and after transformation;
1-5) transforming the temperature data T:
in the formula (2), whereinIs the temperature after transformation;
1-6) date data D value range 0-365, expressed as:
in the formula (3), D isin ,D icos For the date data after processing, due to D i Is limited in value range and all data are shared, D is directly used in calculation for simplifying calculation isin ,D icos Replacement D i
1-7) carrying out normalization processing on the data of precipitation, average wind speed, sunshine hours, average relative humidity and average air pressure at 20-20 hours in the historical data;
1-8) intercepting the data obtained in the step 1-7) in a year unit to be a time sequence, and selecting the maximum value, the minimum value, the average value, the variance and the time corresponding to the maximum value and the time data corresponding to the minimum value of the temperature, the rainfall and the air pressure data as data points X of each meteorological station i
Further, the specific steps of data classification in step 2) are as follows:
2-1) calculating the cluster center weight gamma of each meteorological site through a clustering algorithm based on density peak values and the preprocessing data of each meteorological site in the step 1) i Log gamma i And log gamma is plotted in descending order i Arranging;
2-2) calculation of ordered log y i Difference vector D between two adjacent numbers of sequence i According to the difference vector D i Acquiring clustering center points of meteorological site data, and clustering non-clustering center points;
2-3) calculating data X for each meteorological site in each class i Data X with other meteorological detection points j Euclidean distance d of (2) ij And summing to obtain the total distance value D between the ith meteorological site and other meteorological sites i And find D i The smallest weather station is the cluster center point of the class.
Further, calculating a cluster center weight y of each meteorological site in the step 2-1) i Log gamma i And log gamma is plotted in descending order i The specific steps for the arrangement are as follows:
2-1-1) use of European distance meterCalculating distance matrix d of each meteorological site ij
2-1-2) calculating the cluster center weight gamma of the ith weather site i
γ i =ρ i δ i (4)
In formula (4), ρ i For the local density, delta, corresponding to the ith weather site i Is the relative distance;
relative distance delta i
In the formula (5), d ij The Euclidean distance between the ith meteorological site and other meteorological sites;
local density ρ corresponding to the ith weather site i
In the formula (6), d ij For Euclidean distance between the ith meteorological site and other meteorological sites, d c To intercept the distance, wherein:
determining an optimal truncation distance d using an adaptive approach based on coefficient minimization c Continuously adjust the cutting distance d c The optimal cut-off distance d corresponds to the minimum value of the coefficient G c The formula of G is:
in the formula (7), G represents the magnitude of the coefficient value of the foundation of the data set of the weather station, and Z represents the magnitude of the cluster center weight of the data set of the weather station.
2-1-3)Calculation of log gamma i And arranged in descending order.
Further, the specific steps of calculating the clustering center points of the meteorological site data in the step 2-2) and clustering the non-clustering center points are as follows:
2-2-1) calculation of ordered log gamma i Difference vector D between two adjacent numbers of sequence i Find out the position D with the largest difference change m+1 D is the position with the maximum difference value change m+1 The previous m points are all set as the center points of the initial clusters;
2-2-2) calculating the average of the first m difference vectors from the difference vector D m+1 Firstly, sequentially comparing whether the difference vector is larger than the average value of m values before the vector D, and if so, classifying the point as a clustering center point;
2-2-3) taking the clustering center points marked in the steps 2-2-1) and 2-2-2) as final clustering center points;
2-2-4) clustering non-clustered center points.
Further, the specific steps of calculating the clustering center points of each type of meteorological stations in the step 2-3) are as follows:
2-3-1) calculating data X for each meteorological site in each class i Data X with other meteorological detection points j Euclidean distance d of (2) ij And summing to obtain the total distance value D between the ith meteorological site and other meteorological sites i
In the formula (8), k is the maximum number of weather site data in a single data set, and k is C;
2-3-2) find D in each class i The smallest weather station is the cluster center point of each class. Further, in step 3), each type of data is used to train a respective spatial prediction neural network module, and the specific steps of optimizing the spatial prediction neural network module by adopting the RMSprop algorithm are as follows:
3-1-1) all weather station counts for each classSelecting date, longitude and latitude, weather value, average temperature and average sunlight 5 kinds of data as input data X i Average temperature, average insolation as output data Y i
3-1-2) randomly ordering all the data in the step 3-1-1), then randomly selecting tags of 30% of the data, adding a deviation coefficient mu, and calculating the deviation coefficient mu when the deviation coefficient 0 of the rest 70% of the data is:
in the formula (9), T i k As the average temperature over the k days,for average insolation over k days, k { k|k.epsilon.2, 10],k∈Z},T i 1 For the average temperature of the day, +.>Average insolation for the day;
3-1-3) randomly sequencing the data added with the deviation coefficient in the step 3-2), and selecting 70% of the data as a training set and the rest 30% of the data as a verification set;
3-1-4) constructing a 5-layer fully-connected feedforward network, wherein each layer of neuron parameters are 64, 128, 64 and 3 respectively, obtaining a spatial prediction neural network module, and selecting a secondary loss function MSE as a loss function L MSE
And optimizing by adopting an RMSprop algorithm, so that the spatial prediction neural network module is stable and the verification centralized loss function value is minimum. Further, in step 3), the specific steps of training the respective prediction model for the type data obtained by each cluster according to the optimized spatial prediction neural network module are as follows:
3-2-1) sorting the data of step 1-1) in time sequence, randomly intercepting 30 consecutive data as a group of data, namely D i ={x t ,x t+1 ,...,x t+29 Average, maximum and minimum air temperature of 7 data after the sequence is taken, and the sunshine hours are taken as training labelsInput matrix dimension (1,11,30), output and label dimensions (4, 7);
3-2-2) randomly selecting 80% of the data as a training set, and the remaining 20% as a verification set;
3-2-3) constructing a prediction model consisting of a convolution layer, a BN layer, an average pooling layer and a full connection layer;
3-2-4) connecting 3 outputs of the spatial prediction neural network module trained in the step 3) to a first full connection layer of a prediction model;
3-2-5) setting the parameter learning rate of the spatial prediction neural network module to be 1e-3, adding the loss function to the loss function of the output of the spatial prediction neural network module and the output of the overall model, training the overall model by using the data of the step 3-2-1), removing the spatial prediction neural network module, replacing the deviation coefficient with 0 by the average temperature on the day and the average sunlight on the day in the output of the spatial prediction neural network module, continuing training the model for a plurality of times until the loss of the verification set is stable.
Further, the specific steps of predicting the temperature and illumination of the target area in the step 4) are as follows:
4-1) if the target area has sample data, determining the category of the predicted target area by calculating the data and the Euclidean distance of the cluster center point obtained in the step 2), finding out the corresponding type and the prediction model, performing Fine tuning of the Fine-tuning on the prediction model, fixing all layer parameters before the first fully-connected layer during Fine tuning, and setting the learning rate to be 1e-3; if no data exists, selecting the type of the weather station closest to the target area, and performing field adaptation on a model construction field discriminator by using the data of the weather station closest to the prediction area, wherein the input of the field discriminator is the output of a first full-connection layer, and the parameters of all layers outside the first full-connection layer are fixed and used for adjusting the weight of the first full-connection layer so that the model characteristic data distribution situation is closer to the target area;
4-2) according to the optimization model obtained in the step 4-1), if the target area has sample data, disconnecting the output of the spatial prediction neural network module, replacing the output with the current day average temperature and the average sunlight of the target area after data processing, and inputting a deviation coefficient of 0; if the target area has no sample data, using longitude and latitude and current date of the target area as spatial prediction neural network module input, inputting weather value (0-1) according to weather observation, then inputting data of the nearest area into a model for prediction, and outputting 1 x 28 vectors which are respectively prediction results of the model on average, maximum, minimum air temperature and sunshine hours of 1 to 7 days in the future.
Due to the adoption of the technical scheme, the application has the following advantages:
1. according to the application, clustering modeling is carried out by utilizing different weather features among various weather tables, so that the accuracy of weather prediction is improved; 2. the method utilizes the nonlinear inference capability of the neural network to improve the spatial resolution of meteorological data based on latitude and longitude data and known data information; 3. the method is suitable for predicting the temperature and sunlight when a small amount of sample data or zero sample data exists in the target prediction area, and has wide application range; 4. according to the application, the historical data of each region is utilized to generate the prediction module, the prediction of the temperature and the sunlight when a small amount of sample data or zero sample data exists in the target prediction region is realized through the prediction module and the deep transfer learning method, and the prediction precision is high.
Additional advantages, objects, and features of the application will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof.
Drawings
The drawings of the present application are described below.
FIG. 1 is a flow chart of the deep transfer learning prediction of the present application.
Fig. 2 is a schematic diagram of the neural network structure of the present application.
Detailed Description
The application is further described below with reference to the drawings and examples.
A weather temperature and illumination prediction method based on deep migration learning is characterized by comprising the following specific steps:
1) And (3) data acquisition: the method comprises the following specific steps of obtaining historical data recorded by each meteorological site in a target area, and carrying out data preprocessing on the historical data to obtain preprocessed data:
1-1) collecting average, maximum and minimum air temperatures recorded by each weather station in a target area, precipitation at 20-20 hours, average wind speed, sunshine hours, average relative humidity, average air pressure, longitude and latitude of the weather station and data collection time by using daily data units, and obtaining historical data;
1-2) replacing the missing value or the abnormal value of the average, the maximum and the minimum air temperature, the precipitation amount at 20-20 days, the average air speed, the sunshine hours, the average relative humidity and the average air pressure in the historical data with the average value of the missing value or the two values before and after the abnormal value or with the average value in the time range of 3-5 days;
1-3) adding a weather value to the historical data, wherein the weather value is 1 when the precipitation is more than 0, and 0 when the precipitation is equal to 0;
1-4) the method is mainly oriented to China, and the latitude and longitude range of China is approximately as follows: latitude 3.86 to 53.55, longitude 73.66 to 135.05, thus transforming the weather station longitude and latitude data:
in the formula (1), x i And (3) withRespectively represent longitude, y before and after transformation i And->Respectively representing the latitude before and after transformation;
1-5) since the temperature data are concentrated in a large amount in the range of 0-30, uneven distribution on the value range can have a great influence on the neural network learning, and the problems of reduced prediction accuracy, difficult convergence and the like are caused, so the temperature data T are converted into:
in the formula (2), whereinIs the temperature after treatment;
1-6) date data D value range 0-365, which is expressed as:
in the formula (3), D isin ,D icos For the date data after processing, due to D i Is limited in value range and all data are shared, D is directly used in calculation for simplifying calculation isin ,D icos Replacement D i
1-7) carrying out normalization processing on the data of precipitation, average wind speed, sunshine hours, average relative humidity and average air pressure at 20-20 hours in the historical data;
1-8) intercepting the data obtained in the step 1-7) in a year unit to be a time sequence, and selecting the maximum value, the minimum value, the average value, the variance and the time corresponding to the maximum value and the time data corresponding to the minimum value of the temperature, the rainfall and the air pressure data as data points X of each meteorological station i
2) Data classification: classifying the meteorological detection points through a clustering algorithm based on density peaks and the preprocessing data of each meteorological site in the step 1), and calculating a clustering center point of each class, wherein the specific steps are as follows:
2-1) calculating the cluster center weight gamma of each meteorological site through a clustering algorithm based on density peaks and the preprocessing data of each meteorological site in the step 1) i Log gamma i And log gamma is plotted in descending order i The method comprises the following specific steps of:
2-1-1) calculating a distance matrix d for each meteorological site using Euclidean distances ij
2-1-2) calculating the cluster center weight gamma of the ith weather site i
γ i =ρ i δ i (4)
In formula (4), ρ i For the local density, delta, corresponding to the ith weather site i Is the relative distance;
relative distance delta i
In the formula (5), d ij The Euclidean distance between the ith meteorological site and other meteorological sites;
local density ρ corresponding to the ith weather site i
In the formula (6), d ij For Euclidean distance between the ith meteorological site and other meteorological sites, d c To intercept the distance, wherein:
determining an optimal truncation distance d using an adaptive approach based on coefficient minimization c Continuously adjust the cutting distance d c When the coefficient of Kerning G is takenThe minimum value corresponds to the optimal cut-off distance d c The formula of G is:
in the formula (7), G represents the magnitude of the coefficient value of the foundation of the data set of the weather station, and Z represents the magnitude of the cluster center weight of the data set of the weather station.
2-1-3) calculation of log gamma i And arranged in descending order.
2-2) calculation of ordered log gamma i Difference vector D between two adjacent numbers of sequence i According to the difference vector D i The method comprises the specific steps of obtaining clustering center points of meteorological site data, and clustering non-clustering center points, wherein the specific steps are as follows:
2-2-1) calculation of ordered log gamma i Difference vector D between two adjacent numbers of sequence i Find out the position D with the largest difference change m+1 D is the position with the maximum difference value change m+1 The previous m points are all set as the center points of the initial clusters;
2-2-2) calculating the average of the first m difference vectors from the difference vector D m+1 Firstly, sequentially comparing whether the difference vector is larger than the average value of m values before the vector D, and if so, classifying the point as a clustering center point;
2-2-3) taking the clustering center points marked in the steps 2-2-1) and 2-2-2) as final clustering center points;
2-2-4) clustering non-clustered center points.
2-3) calculating data X for each meteorological site in each class i Data X with other meteorological detection points j Euclidean distance d of (2) ij And summing to obtain the total distance value D between the ith meteorological site and other meteorological sites i And find D i The minimum meteorological site is the clustering center point of the class, and the specific steps are as follows:
2-3-1) calculating data X for each meteorological site in each class i Data X with other meteorological detection points j Euclidean distance d of (2) ij And summing to obtain the total distance value D between the ith meteorological site and other meteorological sites i
In the formula (8), k is the maximum number of weather site data in a single data set, and k is C;
2-3-2) find D in each class i The smallest weather station is the cluster center point of each class.
3) And (3) constructing a model: training a respective spatial prediction neural network module by using each type of data, optimizing the spatial prediction neural network module by adopting an RMSprop algorithm, and training a respective prediction model for the type data obtained by each cluster according to the optimized spatial prediction neural network module;
in the step 3), each type of data is used for training a respective spatial prediction neural network module, and the specific steps of optimizing the spatial prediction neural network module by adopting an RMSprop algorithm are as follows:
3-1-1) selecting date, longitude and latitude, weather value, average temperature, average sunlight 5 kinds of data for all weather station data of each class in daily unit, wherein the date, longitude and latitude, weather value is used as input data X i Average temperature, average insolation as output data Y i
3-1-2) randomly ordering all the data in the step 3-1-1), then randomly selecting tags of 30% of the data, adding a deviation coefficient mu, and calculating the deviation coefficient mu when the deviation coefficient 0 of the rest 70% of the data is:
in the formula (9), T i k As the average temperature over the k days,for average insolation over k days, k { k|k.epsilon.2, 10],k∈Z},T i 1 For the average temperature of the day, +.>Average insolation for the day;
3-1-3) randomly sequencing the data added with the deviation coefficient in the step 3-2), and selecting 70% of the data as a training set and the rest 30% of the data as a verification set;
3-1-4) constructing a 5-layer fully-connected feedforward network, wherein each layer of neuron parameters are 64, 128, 64 and 3 respectively, obtaining a spatial prediction neural network module, and selecting a secondary loss function MSE as a loss function L MSE
The first 4 layers are optimized by adopting an RMSprop algorithm, and an LRelu activation function (alpha=0.2) is adopted, so that the spatial prediction neural network module is stable and the verification centralized loss function value is minimum.
In the step 3), the specific steps of training the respective prediction model for the type data obtained by each cluster according to the optimized spatial prediction neural network module are as follows:
3-2-1) sorting the data of step 1-1) in time sequence, randomly intercepting 30 consecutive data as a group of data, namely D i ={x t ,x t+1 ,...,x t+29 Average, maximum and minimum air temperature of 7 data after the sequence is taken, and the sunshine hours are taken as training labelsInput matrix dimension (1,11,30), output and label dimensions (4, 7);
3-2-2) randomly selecting 80% of the data as a training set, and the remaining 20% as a verification set;
3-2-3) constructing a prediction model consisting of a convolution layer, a BN layer, an average pooling layer and a full connection layer; 1D convolutional layer 1 (128, k3, s 1), 1D convolutional layer 2 (64, k4, s 1), 1D hole convolutional layer 1 (32, k4, s1, dr 2), BN layer, 1D hole convolutional layer 2 (32, k4, s1, dr 4), BN layer, average pooling layer (k 2), full connection layer (512, activation=relu, dropout=0.3), full connection layer (256, activation=relu, dropout=0.2), full connection layer (28);
3-2-4) accessing the 3 outputs of the spatial prediction neural network module trained in the step 3) to a full connection layer (512, activation=relu, dropout=0.3) of the prediction model;
3-2-5) setting the parameter learning rate of the spatial prediction neural network module to be 1e-3, adding the loss function to the loss function of the output of the spatial prediction neural network module and the output of the overall model, training the overall model by using the data of the step 3-2-1), removing the spatial prediction neural network module, replacing the deviation coefficient with 0 by the average temperature on the day and the average sunlight on the day in the output of the spatial prediction neural network module, continuing training the model for a plurality of times until the loss of the verification set is stable.
4) Predicted temperature and illumination: if the predicted target area has sample data, matching the type and the prediction model according to the sample data, finely adjusting the prediction model, predicting the temperature and the sunlight by an isolated space prediction network module, if the predicted target area has no sample data, selecting the type of a weather station and the prediction model which are nearest to the predicted target area, performing field self-adaption on a prediction model construction field discriminator, and predicting by using the space network module, wherein the specific steps are as follows:
4-1) if the target area has sample data, determining the category of the predicted target area by calculating the data and the Euclidean distance of the cluster center point obtained in the step 2), finding out the corresponding type and the prediction model, performing Fine tuning of the Fine-tuning of the prediction model, fixing all layer parameters before the full-connection layer (512, activation=inl, dropout=0.3) during Fine tuning of the Fine-tuning, and setting the learning rate to be 1e-3; if no data exists, selecting the type of the weather station closest to the target area, and performing field adaptation on a model construction field discriminator by using the data of the weather station closest to the prediction area, wherein the field discriminator inputs the output of a full-connection layer (512, activation=relu, dropoout=0.3), and fixing the parameters of all layers outside the full-connection layer (512, activation=relu, dropoout=0.3) to adjust the weight of the first full-connection layer so that the model characteristic data distribution situation is closer to the target area;
4-2) according to the optimization model obtained in the step 4-1), if the target area has sample data, disconnecting the output of the spatial prediction neural network module, replacing the output with the current day average temperature and the average sunlight of the target area after data processing, and inputting a deviation coefficient of 0; if the target area has no sample data, using longitude and latitude and current date of the target area as spatial prediction neural network module input, inputting weather value (0-1) according to weather observation, then inputting data of the nearest area into a model for prediction, and outputting 1 x 28 vectors which are respectively prediction results of the model on average, maximum, minimum air temperature and sunshine hours of 1 to 7 days in the future.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical aspects of the present application and not for limiting the same, and although the present application has been described in detail with reference to the above embodiments, it should be understood by those of ordinary skill in the art that: modifications and equivalents may be made to the specific embodiments of the application without departing from the spirit and scope of the application, which is intended to be covered by the claims.

Claims (1)

1. A weather temperature and illumination prediction method based on deep migration learning is characterized by comprising the following specific steps:
1) And (3) data acquisition: acquiring historical data recorded by each meteorological site in a target area, and performing data preprocessing on the historical data to obtain preprocessed data;
2) Data classification: classifying the meteorological detection points through a clustering algorithm based on density peaks and the preprocessing data of each meteorological site in the step 1), and calculating a clustering center point of each class;
3) And (3) constructing a model: training a respective spatial prediction neural network module by using each type of data, optimizing the spatial prediction neural network module by adopting an RMSprop algorithm, and training a respective prediction model for the type data obtained by each cluster according to the optimized spatial prediction neural network module;
4) Predicted temperature and illumination: if the predicted target area has sample data, matching types and a prediction model according to the sample data, performing Fine tuning of the prediction model, performing prediction on temperature and sunlight by an isolated spatial prediction network module, if the predicted target area has no sample data, selecting a weather station type and a prediction model which are nearest to the predicted target area, and performing field adaptation on a prediction model construction field discriminator, wherein the prediction is performed by using the spatial network module;
the specific steps of data acquisition in step 1) are as follows:
1-1) collecting average, maximum and minimum air temperatures recorded by each weather station in a target area, precipitation at 20-20 hours, average wind speed, sunshine hours, average relative humidity, average air pressure, longitude and latitude of the weather station and data collection time by using daily data units, and obtaining historical data;
1-2) replacing the missing values or abnormal values of the average, maximum and minimum air temperatures, precipitation at 20-20 days, average wind speed, sunshine hours, average relative humidity and average air pressure in the historical data with the average value of the missing values or the values before and after the abnormal values or with the average value of the time range of 3-5 days;
1-3) adding a weather value to the historical data, wherein the weather value is 1 when the precipitation is more than 0, and 0 when the precipitation is equal to 0;
1-4) transforming longitude and latitude data of the weather station:
in the formula (1), x i And (3) withRespectively represent longitude, y before and after transformation i And->Respectively representing the latitude before and after transformation;
1-5) transforming the temperature data T:
in the formula (2), whereinIs the temperature after transformation;
1-6) date data D value range 0-365, expressed as:
in the formula (3), D isin ,D icos For the date data after processing, due to D i Is limited in value range and all data are shared, D is directly used in calculation for simplifying calculation isin ,D icos Replacement D i
1-7) carrying out normalization processing on the data of precipitation, average wind speed, sunshine hours, average relative humidity and average air pressure at 20-20 hours in the historical data;
1-8) intercepting the data obtained in the step 1-7) in a year unit to be a time sequence, and selecting the maximum value, the minimum value, the average value, the variance, the time corresponding to the maximum value and the time data corresponding to the minimum value of the temperature, the rainfall and the air pressure data as data points X of each meteorological station i
The specific steps of data classification in step 2) are as follows:
2-1) calculating the cluster center weight gamma of each meteorological site through a clustering algorithm based on density peak values and the preprocessing data of each meteorological site in the step 1) i Log gamma i And log gamma is calculated in descending order i Arranging;
2-2) calculation of ordered log y i Difference vector D between two adjacent numbers of sequence i According to the difference vector D i Acquiring clustering center points of meteorological site data, and clustering non-clustering center points;
2-3) calculating data X for each meteorological site in each class i Data X with other meteorological detection points j Euclidean distance d of (2) ij And summing to obtain the total distance value D between the ith meteorological site and other meteorological sites i And find D i The minimum weather station is the cluster center point of the class;
step 2-1) calculating a cluster center weight y of each meteorological site i Log gamma i And log gamma is calculated in descending order i The specific steps for the arrangement are as follows:
2-1-1) calculating a distance matrix d for each meteorological site using Euclidean distances ij
2-1-2) calculating a cluster center weight y of the ith weather site i
Υ i =ρ i δ i (4)
In formula (4), ρ i For the local density, delta, corresponding to the ith weather site i Is the relative distance;
relative distance delta i
In the formula (5), d ij The Euclidean distance between the ith meteorological site and other meteorological sites;
local density ρ corresponding to the ith weather site i
In the formula (6), d ij For the Euclidean distance between the ith meteorological site and other meteorological sitesSeparation, d c To intercept the distance, wherein:
determining an optimal truncation distance d using an adaptive approach based on coefficient minimization c Continuously adjust the cutting distance d c The optimal cut-off distance d corresponds to the minimum value of the coefficient G c The formula of G is:
in the formula (7), G represents the magnitude of a coefficient value of a foundation of a data set of the weather station, and Z represents the magnitude of a cluster center weight of the data set of the weather station;
2-1-3) calculation of log y i And arranged in descending order;
the specific steps of calculating the clustering center points of the meteorological site data and clustering the non-clustering center points in the step 2-2) are as follows:
2-2-1) calculation of ordered log gamma i Difference vector D between two adjacent numbers of sequence i Find out the position D with the largest difference change m+1 D is the position with the maximum difference value change m+1 The previous m points are all set as the center points of the initial clusters;
2-2-2) calculating the average of the first m difference vectors from the difference vector D m+1 Firstly, sequentially comparing whether the difference vector is larger than the average value of m values before the vector D, and if so, classifying the point as a clustering center point;
2-2-3) taking the clustering center points marked in the steps 2-2-1) and 2-2-2) as final clustering center points;
2-2-4) clustering the non-clustered center points;
the specific steps for calculating the clustering center point of each type of meteorological site in the step 2-3) are as follows:
2-3-1) calculating data X for each meteorological site in each class i Among other thingsData X of meteorological detection point j Euclidean distance d of (2) ij And summing to obtain the total distance value D between the ith meteorological site and other meteorological sites i
In the formula (8), k is the maximum number of weather site data in a single data set, and k is C;
2-3-2) find D in each class i The minimum weather station is the cluster center point of each class;
in the step 3), each type of data is used for training a respective spatial prediction neural network module, and the specific steps of optimizing the spatial prediction neural network module by adopting an RMSprop algorithm are as follows:
3-1-1) selecting date, longitude and latitude, weather value, average temperature, average sunlight 5 kinds of data for all weather station data of each class in daily unit, wherein the date, longitude and latitude, weather value is used as input data X i Average temperature and average insolation as output data Y i
3-1-2) randomly ordering all the data in the step 3-1-1), then randomly selecting tags of 30% of the data, adding a deviation coefficient mu, and calculating the deviation coefficient mu when the deviation coefficient 0 of the rest 70% of the data is:
in the formula (9), the amino acid sequence of the compound,for average temperature in k days H i k For average insolation over k days, k { k|k.epsilon.2, 10],k∈Z},/>Is the same as the dayAverage temperature,/->Average insolation for the day;
3-1-3) randomly sequencing the data added with the deviation coefficient in the step 3-2), and selecting 70% of the data as a training set and the rest 30% of the data as a verification set;
3-1-4) constructing a 5-layer fully-connected feedforward network, wherein each layer of neuron parameters are 64, 128, 64 and 3 respectively, obtaining a spatial prediction neural network module, and selecting a secondary loss function MSE as a loss function L MSE
Optimizing by adopting an RMSprop algorithm, so that the spatial prediction neural network module is stable and the verification centralized loss function value is minimum;
in the step 3), the specific steps of training the respective prediction model for the type data obtained by each cluster according to the optimized spatial prediction neural network module are as follows:
3-2-1) sorting the data of step 1-1) in time sequence, randomly intercepting 30 consecutive data as a group of data, namely D i ={x t ,x t+1 ,...,x t+29 Average, maximum and minimum air temperature of 7 data after the sequence is taken, and the sunshine hours are taken as training labelsInput matrix dimension (1,11,30), output and label dimensions (4, 7);
3-2-2) randomly selecting 80% of the data as a training set, and the remaining 20% as a verification set;
3-2-3) constructing a prediction model consisting of a convolution layer, a BN layer, an average pooling layer and a full connection layer;
3-2-4) connecting 3 outputs of the spatial prediction neural network module trained in the step 3) to a first full connection layer of a prediction model;
3-2-5) setting the parameter learning rate of the spatial prediction neural network module as 1e-3, adding the loss function which is the loss function of the output of the spatial prediction neural network module and the output of the overall model, training the overall model by using the data of the step 3-2-1), removing the spatial prediction neural network module, replacing the deviation coefficient with 0 by using the average temperature on the same day and the average sunlight on the same day in the output of the spatial prediction neural network module, continuing training the model for a plurality of times until the loss of the verification set is stable;
the specific steps of predicting the temperature and illumination of the target area in the step 4) are as follows:
4-1) if the target area has sample data, determining the category of the predicted target area by calculating the data and the Euclidean distance of the cluster center point obtained in the step 2), finding out the corresponding type and the prediction model, performing Fine tuning of the Fine-tuning on the prediction model, fixing all layer parameters before the first fully-connected layer during Fine tuning, and setting the learning rate to be 1e-3; if no data exists, selecting the type of the weather station closest to the target area, and performing field adaptation on a model construction field discriminator by using the data of the weather station closest to the prediction area, wherein the input of the field discriminator is the output of a first full-connection layer, and the parameters of all layers outside the first full-connection layer are fixed and used for adjusting the weight of the first full-connection layer so that the model characteristic data distribution situation is closer to the target area;
4-2) according to the optimization model obtained in the step 4-1), if the target area has sample data, disconnecting the output of the spatial prediction neural network module, replacing the output with the current day average temperature and the average sunlight of the target area after data processing, and inputting a deviation coefficient of 0; if the target area has no sample data, using longitude and latitude and current date of the target area as spatial prediction neural network module input, inputting weather value (0-1) according to weather observation, then inputting data of the nearest area into a model for prediction, and outputting 1 x 28 vectors which are respectively prediction results of the model on average, maximum, minimum air temperature and sunshine hours of 1 to 7 days in the future.
CN202110220175.3A 2021-02-26 2021-02-26 Meteorological temperature and illumination prediction method based on deep migration learning Active CN112990558B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110220175.3A CN112990558B (en) 2021-02-26 2021-02-26 Meteorological temperature and illumination prediction method based on deep migration learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110220175.3A CN112990558B (en) 2021-02-26 2021-02-26 Meteorological temperature and illumination prediction method based on deep migration learning

Publications (2)

Publication Number Publication Date
CN112990558A CN112990558A (en) 2021-06-18
CN112990558B true CN112990558B (en) 2023-08-29

Family

ID=76351258

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110220175.3A Active CN112990558B (en) 2021-02-26 2021-02-26 Meteorological temperature and illumination prediction method based on deep migration learning

Country Status (1)

Country Link
CN (1) CN112990558B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116028838B (en) * 2023-01-09 2023-09-19 广东电网有限责任公司 Clustering algorithm-based energy data processing method and device and terminal equipment
CN115796564B (en) * 2023-02-13 2023-04-25 山东济矿鲁能煤电股份有限公司阳城煤矿 Colliery work management system based on meteorological supervision
CN116108761B (en) * 2023-04-12 2023-06-27 中国科学院地理科学与资源研究所 Regional climate simulation method and system for coupling deep learning and HASM
CN117057416B (en) * 2023-10-11 2024-02-09 中国科学技术大学 Sub-solar photovoltaic power generation prediction method and system
CN117422182A (en) * 2023-12-18 2024-01-19 保大坊科技有限公司 Data prediction method, device and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711620A (en) * 2018-12-26 2019-05-03 浙江大学 A kind of Short-Term Load Forecasting Method based on GRU neural network and transfer learning
CN110458844A (en) * 2019-07-22 2019-11-15 大连理工大学 A kind of semantic segmentation method of low illumination scene
CN111260117A (en) * 2020-01-10 2020-06-09 燕山大学 CA-NARX water quality prediction method based on meteorological factors

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10871594B2 (en) * 2019-04-30 2020-12-22 ClimateAI, Inc. Methods and systems for climate forecasting using artificial neural networks

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711620A (en) * 2018-12-26 2019-05-03 浙江大学 A kind of Short-Term Load Forecasting Method based on GRU neural network and transfer learning
CN110458844A (en) * 2019-07-22 2019-11-15 大连理工大学 A kind of semantic segmentation method of low illumination scene
CN111260117A (en) * 2020-01-10 2020-06-09 燕山大学 CA-NARX water quality prediction method based on meteorological factors

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于时空优化LSTM 深度学习网络的气温预测;杨耘等;《徐州工程学院学报( 自然科学版)》;第35卷(第2期);第44-49页 *

Also Published As

Publication number Publication date
CN112990558A (en) 2021-06-18

Similar Documents

Publication Publication Date Title
CN112990558B (en) Meteorological temperature and illumination prediction method based on deep migration learning
CN109002915B (en) Photovoltaic power station short-term power prediction method based on Kmeans-GRA-Elman model
CN107766990B (en) Method for predicting power generation power of photovoltaic power station
Ghaderi et al. Deep forecast: Deep learning-based spatio-temporal forecasting
CN106529719B (en) Wind power prediction method based on particle swarm optimization algorithm wind speed fusion
CN105701572B (en) Photovoltaic short-term output prediction method based on improved Gaussian process regression
CN106022538B (en) Photovoltaic power generation prediction method based on K-means clustering improved generalized weather
CN112215428B (en) Photovoltaic power generation power prediction method and system based on error correction and fuzzy logic
CN112465256A (en) Building power consumption prediction method and system based on Stacking model fusion
Feng et al. Short-term global horizontal irradiance forecasting based on sky imaging and pattern recognition
CN116128150B (en) Photovoltaic intelligent output prediction method based on two-stage optimization
CN114118596A (en) Photovoltaic power generation capacity prediction method and device
Chen et al. Research on wind power prediction method based on convolutional neural network and genetic algorithm
CN112927097A (en) Photovoltaic power generation short-term prediction method based on GRA-ABC-Elman model
CN114387121A (en) Collaborative optimization scheduling method and system for power storage material supply
CN115912502A (en) Intelligent power grid operation optimization method and device
CN114882373A (en) Multi-feature fusion sandstorm prediction method based on deep neural network
CN110852492A (en) Photovoltaic power ultra-short-term prediction method for finding similarity based on Mahalanobis distance
CN114240687A (en) Energy hosting efficiency analysis method suitable for comprehensive energy system
CN112688306A (en) Photovoltaic cycle output scene prediction method based on weather forecast and historical power data combined drive
CN114880754B (en) BIM-based building energy consumption management method and system
CN116029440A (en) Ultra-short-term power prediction method and device for photovoltaic power station
CN111062509A (en) Wind power medium-short term prediction method and system based on improved neural network
CN116484998A (en) Distributed photovoltaic power station power prediction method and system based on meteorological similar day
CN109034453A (en) A kind of Short-Term Load Forecasting Method based on multiple labeling neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant