CN114564513A - Sea fog prediction method, device, equipment and storage medium based on neural network - Google Patents
Sea fog prediction method, device, equipment and storage medium based on neural network Download PDFInfo
- Publication number
- CN114564513A CN114564513A CN202210169616.6A CN202210169616A CN114564513A CN 114564513 A CN114564513 A CN 114564513A CN 202210169616 A CN202210169616 A CN 202210169616A CN 114564513 A CN114564513 A CN 114564513A
- Authority
- CN
- China
- Prior art keywords
- observation data
- sea fog
- data set
- prediction model
- fog prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 35
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000007781 pre-processing Methods 0.000 claims abstract description 27
- 238000012549 training Methods 0.000 claims description 57
- 238000004590 computer program Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 9
- 238000013480 data collection Methods 0.000 claims description 6
- 238000003062 neural network model Methods 0.000 description 22
- 230000006870 function Effects 0.000 description 14
- 238000012937 correction Methods 0.000 description 8
- 238000012360 testing method Methods 0.000 description 8
- 210000002569 neuron Anatomy 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000003066 decision tree Methods 0.000 description 5
- 230000002159 abnormal effect Effects 0.000 description 4
- 230000004913 activation Effects 0.000 description 4
- 230000003247 decreasing effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000005856 abnormality Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000012417 linear regression Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 208000003443 Unconsciousness Diseases 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2477—Temporal data queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Tourism & Hospitality (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Business, Economics & Management (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Marketing (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Development Economics (AREA)
- Fuzzy Systems (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Entrepreneurship & Innovation (AREA)
- Game Theory and Decision Science (AREA)
- Primary Health Care (AREA)
- Educational Administration (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention is suitable for the technical field of sea fog forecast, and provides a sea fog prediction method, a sea fog prediction device, sea fog prediction equipment and a storage medium based on a neural network, wherein the sea fog prediction method comprises the following steps: the method comprises the steps of collecting ground and high-altitude live observation data sets in a forecast area, preprocessing the live observation data sets, and forecasting sea fog in the forecast area at a future preset time by using a pre-trained sea fog forecasting model according to the preprocessed live observation data sets, so that the accuracy and the timeliness of sea fog forecasting are improved.
Description
Technical Field
The invention belongs to the technical field of sea fog prediction, and particularly relates to a sea fog prediction method, a sea fog prediction device, sea fog prediction equipment and a storage medium based on a neural network.
Background
Sea fog is a phenomenon occurring at sea or in coastal areas, and when the sea fog occurs, the visibility at sea level is low, which can seriously affect the production life of people along the shore and the safety of offshore operation. At present, the research on the intelligent short-term sea fog forecast is less, the influence factors of the sea fog are numerous, the forming mechanism is complex, the establishment of a numerical forecast model is difficult, extreme weather is more and more frequent along with the increasing abundance and bulkiness of meteorological data and the change of climate environment, the traditional intelligent forecast model established by depending on experience and numerical value is often unconscious and is not suitable for use more and more in the presence of large meteorological data, and therefore the forecast accuracy is difficult to meet the requirement of daily forecast business.
At present, the classification of the existence of sea fog is commonly carried out by utilizing a decision tree model or linear regression based on the basic theory cognition of the sea fog. The decision tree is a prediction discrimination model, which represents a mapping relationship between meteorological observation data and weather phenomena, each node in the tree represents a certain object, each branch path represents a certain possible observation value, and each leaf node corresponds to the weather phenomena represented by the paths from the root node to the leaf node, however, a small change in the meteorological data may cause a completely different decision tree generation, that is, the decision tree may be unstable, and especially in the context of frequent extreme weather in recent years, the instability of the decision tree may challenge the accuracy of sea fog prediction. The linear regression model is based on the premise that the relation between the weather observation data and a specific weather phenomenon is determined to be linear, and then existing data is used for fitting to find the mapping relation between the observation data and the weather phenomenon, however, the sea fog generation influence factors are numerous, and the sea fog and the observation data are not necessarily in a linear relation, so that the sea fog prediction result is inaccurate.
Disclosure of Invention
The embodiment of the invention provides a sea fog prediction method based on a neural network, and aims to solve the problems of low accuracy and poor timeliness of sea fog prediction in the prior art.
The embodiment of the invention is realized in such a way that the sea fog prediction method based on the neural network comprises the following steps:
collecting ground and high-altitude live observation data sets in a forecast area;
pre-processing the live observation data set;
and predicting the sea fog in the forecast area at the future preset time by using a pre-trained sea fog prediction model according to the pre-processed live observation data set.
Still further, before the step of collecting live observation data sets of ground and high altitude in the forecast area, the method further comprises:
collecting historical observation data sets of the ground and the high altitude in the forecast area;
preprocessing the historical observation data set;
establishing the sea fog prediction model;
and training the sea fog prediction model according to the preprocessed historical observation data set.
Still further, the step of preprocessing the historical observation data set includes:
correcting the historical observation data set;
and carrying out normalization processing on the corrected historical observation data set.
Further, the step of training the sea fog prediction model according to the preprocessed historical observation data set includes:
and adjusting the model structure of the sea fog prediction model according to the training result.
The embodiment of the invention also provides a sea fog prediction device based on the neural network, and the device comprises:
the first data collection unit is used for collecting ground and high-altitude live observation data sets in the forecast area;
a first pre-processing unit for pre-processing the live observation data set;
and the sea fog prediction unit is used for predicting the sea fog in the future preset time in the forecast area by utilizing a pre-trained sea fog prediction model according to the preprocessed live observation data set.
Still further, the apparatus further comprises:
the second data collection unit is used for collecting historical observation data sets of the ground and the high altitude in the forecast area;
the second preprocessing unit is used for preprocessing the historical observation data set;
the prediction model establishing unit is used for establishing the sea fog prediction model;
and the network model training unit is used for training the sea fog prediction model according to the preprocessed historical observation data set.
Still further, the second pre-processing unit includes:
the data correcting unit is used for correcting the historical observation data set;
and the normalizing processing unit is used for carrying out normalization processing on the corrected historical observation data set.
Still further, the network model training unit includes:
and the model structure adjusting unit is used for adjusting the model structure of the sea fog prediction model according to the training result.
Embodiments of the present invention further provide a computing device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the method for sea fog prediction based on a neural network are implemented.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the steps of the method for predicting sea fog based on a neural network are implemented.
The invention provides a sea fog prediction method, a sea fog prediction device, sea fog prediction equipment and a storage medium based on a neural network. According to the sea fog prediction method, the sea fog prediction model trained in advance is adopted to predict the sea fog at the future time according to the preprocessed live observation data set, so that the accuracy and the timeliness of sea fog prediction are improved.
Drawings
Fig. 1 is a flowchart of an implementation of a sea fog prediction method based on a neural network according to an embodiment of the present invention;
FIG. 2 is a flow chart of an implementation of sea fog prediction model training provided by an embodiment of the present invention;
FIG. 3 is a flow chart of an implementation of pre-processing a historical observation data set according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a sea fog prediction device based on a neural network according to an embodiment of the present invention;
fig. 5 is another schematic structural diagram of a sea fog prediction apparatus based on a neural network according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a preferred sea fog prediction device based on a neural network according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a computing device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
According to the sea fog prediction method, the sea fog prediction model trained in advance is adopted to predict the sea fog at the future time according to the preprocessed live observation data set, so that the accuracy and the timeliness of sea fog prediction are improved.
Example one
Fig. 1 shows an implementation flow of a sea fog prediction method based on a neural network according to a first embodiment of the present invention, and for convenience of description, only the relevant parts related to the embodiment of the present invention are shown, which are detailed as follows:
in step S101, live observation data sets of the ground and the high altitude within the forecast area are collected.
In the embodiment of the invention, various numerical values of ground meteorological observation data items related to sea fog are obtained in real time through meteorological observation sites in a forecast area, various numerical values of high-altitude meteorological observation data items related to sea fog are obtained at the same time, and the obtained ground numerical values and the obtained high-altitude numerical values form a ground and high-altitude live observation data set, wherein the ground meteorological observation data items comprise: air temperature, dew point, wind, visibility and relative humidity, wherein the high-altitude meteorological observation data items comprise air temperature, humidity and wind, and all values of the high-altitude meteorological observation data items are observation data of 900 hectopascal (hPa) at high altitude.
In step S102, a live observation data set is preprocessed.
In the embodiment of the invention, the meteorological observation data may have an abnormality, and at this time, the numerical values corresponding to the data items in the live observation data set are preprocessed to ensure that the numerical values corresponding to the data items are usable.
In step S103, according to the pre-processed live observation data set, a pre-trained sea fog prediction model is used to predict sea fog in the forecast area at a future preset time.
In the embodiment of the invention, the numerical values corresponding to the data items in the preprocessed live observation data set are input into a sea fog prediction model trained in advance, the sea fog prediction model predicts the sea fog with the future preset time according to the received numerical values, the prediction result is output, according to the prediction result, meteorological staff can know whether the sea fog will occur in the future preset time, the prediction result can be a numerical value representing truth and falsity, for example, if the prediction result is 1, the sea fog will occur, if the prediction result is 0, the sea fog will not occur, and the sea fog prediction result can be the sea fog prediction result in the time of 12 hours, 24 hours or 48 hours in the future.
In the embodiment of the invention, the ground and high altitude live observation data sets in the forecast area are collected, the live observation data sets are preprocessed, and sea fog in the forecast area at the future preset time is forecasted by utilizing the pre-trained sea fog forecasting model according to the preprocessed live observation data sets, so that the accuracy and the timeliness of sea fog forecasting are improved.
Example two
As an embodiment of the present invention, the pre-processing of the live observation data set in step S102 of embodiment one is implemented by:
(1) the live observation data set is corrected.
In the embodiment of the invention, the meteorological observation data may have abnormal conditions of data missing or data errors, so that the data items with missing values in the live observation data set are filled first, and the data items with wrong values are corrected.
In the embodiment of the invention, manual correction can be carried out by adopting a manual mode, and automatic correction can also be carried out by utilizing a neural network model. When the neural network model is used for automatic setting, the neural network model is firstly established, then the historical value of a single data item (such as air temperature) is learned by the neural network model, the value-taking rule of the single data item is established by learning the historical value, finally the neural network model automatically fills the missing value in the corresponding data item according to the learned value-taking rule, and the error value is corrected.
(2) And carrying out normalization processing on the corrected live observation data set.
In the embodiment of the invention, because the values of different data items are greatly different, in order to facilitate the sea fog prediction by the sea fog prediction model, the values of different data items are uniformly normalized to the range of [ -1,1] by utilizing the normalized variance, so that the data is smoother.
In the embodiment of the invention, the preprocessing of the live observation data set is realized through the steps (1) to (2), so that the smoothness of each numerical value in the live observation data set is improved.
EXAMPLE III
Fig. 2 shows an implementation flow of a sea fog prediction method based on a neural network according to a third embodiment of the present invention, and for convenience of description, only the relevant parts related to the third embodiment of the present invention are shown, which are detailed as follows:
in step S201, historical observation data sets of the ground and the high altitude in the forecast area are collected.
In an embodiment of the present invention, historical meteorological observation data of the last decade (for example, 40 years) in a forecast area is acquired from meteorological observation sites in the forecast area, historical values of ground meteorological observation data items related to sea fog are extracted from the historical meteorological observation data, and historical values of high altitude meteorological observation data items related to sea fog are acquired at the same time, and the acquired ground historical values and high altitude historical values form a ground and high altitude historical observation data set, where the ground meteorological observation data items include: air temperature, dew point, wind, visibility and relative humidity, wherein the high-altitude meteorological observation data items comprise air temperature, humidity and wind, and all values of the high-altitude meteorological observation data items are observation data of 900 hectopascal (hPa) at high altitude.
In step S202, the historical observation data set is preprocessed.
In the embodiment of the invention, the numerical values corresponding to the data items in the historical observation data set are preprocessed, so that the numerical values corresponding to the data items are ensured to be available.
In step S203, a sea fog prediction model is established.
In the embodiment of the invention, the sea fog prediction model is established as a Back Propagation (BP) neural network model, the BP neural network comprises an input layer, a hidden layer and an output layer, and the BP neural network is a multi-layer feedforward neural network trained according to an error Back Propagation algorithm. And setting model parameters of the sea fog prediction model, wherein the model parameters comprise but are not limited to the node number of an input layer, the node number of an output layer, the number of layers of hidden layers, the number of neurons of each layer of hidden layer, an activation function and a loss function, specifically, the node number of the input layer of the sea fog prediction model is determined according to data items in the historical observation data set, the node number of the output layer of the sea fog prediction model is set to be one, and meanwhile, related parameters such as the number of layers of hidden layers, the number of neurons of each layer, the activation function and the loss function of the sea fog prediction model are initially set.
In step S204, a sea fog prediction model is trained from the preprocessed historical observation data set.
In the embodiment of the invention, training parameters of the sea fog prediction model are set firstly, the training parameters include but are not limited to the division ratio of the training set and the test set (namely, the preprocessed historical observation data set is divided into the training set and the test set according to the preset ratio), the training times, the feeding data amount and the learning rate, and then the sea fog prediction model is trained according to the set training parameters and the preprocessed historical observation data set.
In the embodiment of the invention, in the training process of the sea fog prediction model, the training parameters are adjusted according to the training results, for example, under the condition that the loss value is not converged or the fluctuation is large, the feeding data amount can be increased or the learning rate is increased or decreased, then the sea fog prediction model is continuously trained according to the adjusted training parameters, and the steps are repeated until the loss value and the prediction accuracy of the loss function tend to be stably converged, so that the training efficiency and the test accuracy are improved, wherein the prediction accuracy is the matching probability of the sea fog prediction model on the prediction result of sea fog and the actual real result according to the historical observation data set.
In the embodiment of the invention, the sea fog prediction model is trained according to the preprocessed historical observation data set, and the training parameters of the sea fog prediction model are adjusted according to the training result in the training process, so that the training efficiency and the testing accuracy are improved.
Example four
As an embodiment of the present invention, when the over-fitting or under-fitting phenomenon in the sea fog prediction model training process and/or poor convergence of the loss value cannot be solved by adjusting the training parameters of the sea fog prediction model, the model structure of the sea fog prediction model is adjusted according to the training result, for example, the number of hidden layers and/or the number of neurons is increased/decreased, thereby improving the prediction accuracy and learning efficiency of the model.
EXAMPLE five
Fig. 3 shows an implementation flow of preprocessing a historical observation data set in a sea fog prediction method based on a neural network according to a fifth embodiment of the present invention, and for convenience of description, only the relevant parts of the embodiment of the present invention are shown, which is detailed as follows:
in step S301, the historical observation data set is corrected.
In the embodiment of the invention, the meteorological observation data may have abnormal conditions of data missing or data errors, so that data items with missing values in the historical observation data set are filled first, and data items with wrong values are corrected.
In the embodiment of the invention, manual correction can be carried out by adopting a manual mode, and automatic correction can also be carried out by utilizing a neural network model. When the neural network model is used for automatic ordering, the neural network model is firstly established, then the neural network model is used for learning the historical value of a single data item (such as air temperature), the value-taking rule of the single data item is established through the learning of the historical value, finally, the neural network model is used for automatically filling the missing value in the corresponding data item according to the learned value-taking rule, and correcting the error value.
In step S302, the normalized historical observation data set is normalized.
In the embodiment of the invention, because the values of different data items are greatly different, in order to facilitate the training of the sea fog prediction model, the values of different data items are uniformly normalized to the range of [ -1,1] by utilizing the normalized variance, so that the data is smoother.
In the embodiment of the invention, the smoothness of each numerical value in the historical observation data set is improved by correcting the historical observation data set and normalizing the corrected historical observation data set.
EXAMPLE six
Fig. 4 shows a structure of a sea fog prediction apparatus based on a neural network according to a sixth embodiment of the present invention, and for convenience of explanation, only the parts related to the embodiment of the present invention are shown, which include:
a first data collection unit 41 for collecting live observation data sets of the ground and the high altitude in the forecast area.
In the embodiment of the invention, various numerical values of ground meteorological observation data items related to sea fog are obtained in real time through meteorological observation sites in a forecast area, various numerical values of high-altitude meteorological observation data items related to sea fog are obtained at the same time, and the obtained ground numerical values and the obtained high-altitude numerical values form a ground and high-altitude live observation data set, wherein the ground meteorological observation data items comprise: air temperature, dew point, wind, visibility and relative humidity, wherein the high-altitude meteorological observation data items comprise air temperature, humidity and wind, and all values of the high-altitude meteorological observation data items are observation data of 900 hectopascal (hPa) at high altitude.
A first pre-processing unit 42 for pre-processing the live observation data set.
In the embodiment of the invention, the meteorological observation data may have an abnormality, and at this time, the numerical value corresponding to each data item in the live observation data set is preprocessed to ensure that the numerical value corresponding to each data item is available.
And the sea fog prediction unit 43 is configured to predict sea fog in the forecast area at a future preset time by using a pre-trained sea fog prediction model according to the pre-processed live observation data set.
In the embodiment of the invention, the numerical values corresponding to the data items in the preprocessed live observation data set are input into a sea fog prediction model trained in advance, the sea fog prediction model predicts the sea fog with the future preset time according to the received numerical values, the prediction result is output, according to the prediction result, meteorological staff can know whether the sea fog will occur in the future preset time, the prediction result can be a numerical value representing truth and falsity, for example, if the prediction result is 1, the sea fog will occur, if the prediction result is 0, the sea fog will not occur, and the sea fog prediction result can be the sea fog prediction result in the time of 12 hours, 24 hours or 48 hours in the future.
In the embodiment of the present invention, each unit of the sea fog prediction apparatus based on the neural network may be implemented by corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein.
EXAMPLE seven
As an embodiment of the present invention, the first preprocessing unit 42 includes:
the first data correcting unit is used for correcting the live observation data set.
In the embodiment of the invention, the meteorological observation data may have abnormal conditions of data missing or data errors, so that the data items with missing values in the live observation data set are filled first, and the data items with wrong values are corrected.
In the embodiment of the invention, manual correction can be carried out by adopting a manual mode, and automatic correction can also be carried out by utilizing a neural network model. When the neural network model is used for automatic ordering, the neural network model is firstly established, then the neural network model is used for learning the historical value of a single data item (such as air temperature), the value-taking rule of the single data item is established through the learning of the historical value, finally, the neural network model is used for automatically filling the missing value in the corresponding data item according to the learned value-taking rule, and correcting the error value.
And the first data processing unit is used for carrying out normalization processing on the corrected live observation data set.
In the embodiment of the invention, because the values of different data items are greatly different, in order to facilitate the sea fog prediction by the sea fog prediction model, the values of different data items are uniformly normalized to the range of [ -1,1] by utilizing the normalized variance, so that the data is smoother.
In the embodiment of the present invention, each unit of the sea fog prediction apparatus based on the neural network may be implemented by corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein.
Example eight
Fig. 5 shows a structure of a sea fog prediction apparatus based on a neural network according to an eighth embodiment of the present invention, and for convenience of explanation, only the parts related to the embodiment of the present invention are shown, where the parts include:
and the second data collection unit 51 is used for collecting historical observation data sets of the ground and the high altitude in the forecast area.
In an embodiment of the present invention, historical meteorological observation data of the last decade (for example, 40 years) in a forecast area are acquired from meteorological observation sites in the forecast area, historical values of ground meteorological observation data items related to sea fog are extracted from the historical meteorological observation data, historical values of high-altitude meteorological observation data items related to sea fog are acquired at the same time, and the acquired ground historical values and high-altitude historical values form a ground and high-altitude historical observation data set, wherein the ground meteorological observation data items include: air temperature, dew point, wind, visibility and relative humidity, wherein the high-altitude meteorological observation data items comprise air temperature, humidity and wind, and all values of the high-altitude meteorological observation data items are observation data of 900 hectopascal (hPa) at high altitude.
A second preprocessing unit 52 for preprocessing the historical observation data set.
In the embodiment of the invention, the numerical values corresponding to the data items in the historical observation data set are preprocessed, so that the numerical values corresponding to the data items are ensured to be available.
And the prediction model establishing unit 53 is used for establishing a sea fog prediction model.
In the embodiment of the invention, the sea fog prediction model is established as a Back Propagation (BP) neural network model, the BP neural network comprises an input layer, a hidden layer and an output layer, and the BP neural network is a multi-layer feedforward neural network trained according to an error Back Propagation algorithm. And setting model parameters of the sea fog prediction model, wherein the model parameters comprise but are not limited to the node number of an input layer, the node number of an output layer, the number of layers of hidden layers, the number of neurons of each layer of hidden layer, an activation function and a loss function, specifically, the node number of the input layer of the sea fog prediction model is determined according to data items in the historical observation data set, the node number of the output layer of the sea fog prediction model is set to be one, and meanwhile, related parameters such as the number of layers of hidden layers, the number of neurons of each layer, the activation function and the loss function of the sea fog prediction model are initially set.
And the network model training unit 54 is used for training the sea fog prediction model according to the preprocessed historical observation data set.
In the embodiment of the invention, training parameters of the sea fog prediction model are set firstly, the training parameters include but are not limited to the division ratio of the training set and the test set (namely, the preprocessed historical observation data set is divided into the training set and the test set according to the preset ratio), the training times, the feeding data amount and the learning rate, and then the sea fog prediction model is trained according to the set training parameters and the preprocessed historical observation data set.
In the embodiment of the invention, in the training process of the sea fog prediction model, the training parameters are adjusted according to the training results, for example, under the condition that the loss value is not converged or the fluctuation is large, the feeding data amount can be increased or the learning rate is increased or decreased, then the sea fog prediction model is continuously trained according to the adjusted training parameters, and the steps are repeated until the loss value and the prediction accuracy of the loss function tend to be stably converged, so that the training efficiency and the test accuracy are improved, wherein the prediction accuracy is the matching probability of the sea fog prediction model on the prediction result of sea fog and the actual real result according to the historical observation data set.
In the embodiment of the invention, the sea fog prediction model is trained according to the preprocessed historical observation data set, and the training parameters of the sea fog prediction model are adjusted according to the training result in the training process, so that the training efficiency and the testing accuracy are improved.
In the embodiment of the present invention, each unit of the sea fog prediction apparatus based on the neural network may be implemented by corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein.
Example nine
Referring to fig. 6, as an embodiment of the present invention, the second preprocessing unit 52 includes:
a data correcting unit 521, configured to correct the historical observation data set.
In the embodiment of the invention, the meteorological observation data may have abnormal conditions of data missing or data errors, so that the data items with missing values in the historical observation data set are filled first, and the data items with wrong values are corrected.
In the embodiment of the invention, manual correction can be carried out by adopting a manual mode, and automatic correction can also be carried out by utilizing a neural network model. When the neural network model is used for automatic ordering, the neural network model is firstly established, then the neural network model is used for learning the historical value of a single data item (such as air temperature), the value-taking rule of the single data item is established through the learning of the historical value, finally, the neural network model is used for automatically filling the missing value in the corresponding data item according to the learned value-taking rule, and correcting the error value.
A normalization processing unit 522, configured to perform normalization processing on the corrected historical observation data set.
In the embodiment of the invention, because the values of different data items are greatly different, in order to facilitate the training of the sea fog prediction model, the values of different data items are uniformly normalized to the range of [ -1,1] by utilizing the normalized variance, so that the data is smoother.
In the embodiment of the invention, the smoothness of each numerical value in the historical observation data set is improved by correcting the historical observation data set and normalizing the corrected historical observation data set.
In the embodiment of the present invention, each unit of the sea fog prediction apparatus based on the neural network may be implemented by corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein.
EXAMPLE ten
Referring to fig. 6, as an embodiment of the present invention, the network model training unit 54 includes:
and a model structure adjusting unit 541, configured to adjust a model structure of the sea fog prediction model according to the training result.
In the embodiment of the invention, when the over-fitting or under-fitting phenomenon in the sea fog prediction model training process and/or poor convergence of the loss value cannot be solved by adjusting the training parameters of the sea fog prediction model, the model structure of the sea fog prediction model is adjusted according to the training result, for example, the number of hidden layers and/or the number of neurons is increased/decreased, so that the prediction accuracy and the learning efficiency of the model are improved.
In the embodiment of the present invention, each unit of the sea fog prediction apparatus based on the neural network may be implemented by corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein.
EXAMPLE eleven
Fig. 7 shows a structure of a computing device according to an eleventh embodiment of the present invention, and only a part related to the embodiment of the present invention is shown for convenience of explanation.
The computing device 7 of an embodiment of the invention comprises a processor 70, a memory 71 and a computer program 72 stored in the memory 71 and executable on the processor 70. The processor 70, when executing the computer program 72, implements the steps of one embodiment of the method for sea fog prediction based on neural network, such as the steps S101 to S103 shown in fig. 1. Alternatively, the processor 70, when executing the computer program 72, implements the functions of the units in the above-described device embodiments, such as the functions of the units 41 to 43 shown in fig. 4.
In the embodiment of the invention, the ground and high-altitude live observation data sets in the forecast area are collected, the live observation data sets are preprocessed, and the sea fog in the forecast area within the future preset time is forecasted by utilizing the pre-trained sea fog forecasting model according to the preprocessed live observation data sets, so that the accuracy and timeliness of sea fog forecasting are improved.
The computing device of the embodiment of the invention is a platform, a device or a terminal with large data processing capacity, such as a personal computer, a server and the like. The steps of the method for predicting sea fog based on neural network implemented by the processor 70 in the computing device 7 when executing the computer program 72 can refer to the description of the foregoing method embodiments, and are not repeated herein.
Example twelve
In an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program, which when executed by a processor implements the steps in an embodiment of the method for predicting sea fog based on a neural network, for example, the steps S101 to S103 shown in fig. 1. Alternatively, the computer program may be adapted to perform the functions of the units of the above-described device embodiments, such as the functions of the units 41 to 43 shown in fig. 4, when executed by the processor.
In the embodiment of the invention, the ground and high altitude live observation data sets in the forecast area are collected, the live observation data sets are preprocessed, and sea fog in the forecast area at the future preset time is forecasted by utilizing the pre-trained sea fog forecasting model according to the preprocessed live observation data sets, so that the accuracy and the timeliness of sea fog forecasting are improved.
The computer readable storage medium of the embodiments of the present invention may include any entity or device capable of carrying computer program code, a recording medium, such as a ROM/RAM, a magnetic disk, an optical disk, a flash memory, or the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. A sea fog prediction method based on a neural network is characterized by comprising the following steps:
collecting ground and high-altitude live observation data sets in a forecast area;
pre-processing the live observation data set;
and predicting the sea fog in the forecast area at the future preset time by using a pre-trained sea fog prediction model according to the pre-processed live observation data set.
2. The method of claim 1, wherein the step of collecting live observation data sets of ground and high altitude within a forecast area is preceded by the method further comprising:
collecting historical observation data sets of the ground and the high altitude in the forecast area;
preprocessing the historical observation data set;
establishing the sea fog prediction model;
and training the sea fog prediction model according to the preprocessed historical observation data set.
3. The method of claim 2, wherein the step of preprocessing the historical observation data set comprises:
correcting the historical observation data set;
and carrying out normalization processing on the corrected historical observation data set.
4. The method of claim 2, wherein the step of training the sea fog prediction model based on the pre-processed historical observation data set comprises:
and adjusting the model structure of the sea fog prediction model according to the training result.
5. An apparatus for sea fog prediction based on neural networks, the apparatus comprising:
the first data collection unit is used for collecting ground and high-altitude live observation data sets in the forecast area;
a first pre-processing unit for pre-processing the live observation data set;
and the sea fog prediction unit is used for predicting the sea fog in the future preset time in the forecast area by utilizing a pre-trained sea fog prediction model according to the preprocessed live observation data set.
6. The apparatus of claim 5, wherein the apparatus further comprises:
the second data collection unit is used for collecting historical observation data sets of the ground and the high altitude in the forecast area;
the second preprocessing unit is used for preprocessing the historical observation data set;
the prediction model establishing unit is used for establishing the sea fog prediction model;
and the network model training unit is used for training the sea fog prediction model according to the preprocessed historical observation data set.
7. The apparatus of claim 6, wherein the second pre-processing unit comprises:
the data correcting unit is used for correcting the historical observation data set;
and the normalization processing unit is used for carrying out normalization processing on the corrected historical observation data set.
8. The apparatus of claim 6, wherein the network model training unit comprises:
and the model structure adjusting unit is used for adjusting the model structure of the sea fog prediction model according to the training result.
9. A computing device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 4 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210169616.6A CN114564513A (en) | 2022-02-23 | 2022-02-23 | Sea fog prediction method, device, equipment and storage medium based on neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210169616.6A CN114564513A (en) | 2022-02-23 | 2022-02-23 | Sea fog prediction method, device, equipment and storage medium based on neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114564513A true CN114564513A (en) | 2022-05-31 |
Family
ID=81713169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210169616.6A Pending CN114564513A (en) | 2022-02-23 | 2022-02-23 | Sea fog prediction method, device, equipment and storage medium based on neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114564513A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115097548A (en) * | 2022-08-08 | 2022-09-23 | 广东省气象公共服务中心(广东气象影视宣传中心) | Sea fog classification early warning method, device, equipment and medium based on intelligent prediction |
| CN116401932A (en) * | 2023-06-08 | 2023-07-07 | 成都远望探测技术有限公司 | Sea fog dissipation time estimation method based on laser radar and millimeter wave radar |
-
2022
- 2022-02-23 CN CN202210169616.6A patent/CN114564513A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115097548A (en) * | 2022-08-08 | 2022-09-23 | 广东省气象公共服务中心(广东气象影视宣传中心) | Sea fog classification early warning method, device, equipment and medium based on intelligent prediction |
| CN116401932A (en) * | 2023-06-08 | 2023-07-07 | 成都远望探测技术有限公司 | Sea fog dissipation time estimation method based on laser radar and millimeter wave radar |
| CN116401932B (en) * | 2023-06-08 | 2023-08-15 | 成都远望探测技术有限公司 | Sea fog dissipation time estimation method based on laser radar and millimeter wave radar |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109214592B (en) | Multi-model-fused deep learning air quality prediction method | |
| CN110929918B (en) | 10kV feeder fault prediction method based on CNN and LightGBM | |
| CN110766212B (en) | Ultra-short-term photovoltaic power prediction method for historical data missing electric field | |
| CN114564513A (en) | Sea fog prediction method, device, equipment and storage medium based on neural network | |
| CN108921339B (en) | Quantile regression-based photovoltaic power interval prediction method for genetic support vector machine | |
| CN108898251A (en) | Consider the marine wind electric field power forecasting method of meteorological similitude and power swing | |
| CN111369045A (en) | Method for predicting short-term photovoltaic power generation power | |
| CN116090821A (en) | Power distribution network line security risk assessment method considering extreme weather | |
| CN115936177A (en) | Photovoltaic output power prediction method and system based on neural network | |
| CN114330934A (en) | Model parameter self-adaptive GRU new energy short-term power generation power prediction method | |
| CN116842337A (en) | Transformer fault diagnosis method based on LightGBM (gallium nitride based) optimal characteristics and COA-CNN (chip on board) model | |
| CN115758151A (en) | Combined diagnosis model establishing method and photovoltaic module fault diagnosis method | |
| CN115859099A (en) | Sample generation method and device, electronic equipment and storage medium | |
| AU2019100631A4 (en) | Self-correcting multi-model numerical rainfall ensemble forecasting method | |
| CN111697560B (en) | Method and system for predicting load of power system based on LSTM | |
| CN113095547A (en) | Short-term wind power prediction method based on GRA-LSTM-ICE model | |
| CN117574776A (en) | Task planning-oriented model self-learning optimization method | |
| CN117096871A (en) | Wind power probability density prediction method based on space-time distribution | |
| CN116663393A (en) | Random forest-based power distribution network continuous high-temperature fault risk level prediction method | |
| CN116565840A (en) | High-precision wind speed soft measurement method for wind power prediction of wind power plant | |
| CN115271242A (en) | Training method, prediction method and device of photovoltaic power generation power prediction model | |
| CN111626465B (en) | New energy power short-term interval prediction method and system | |
| CN114936640A (en) | Online training method for new energy power generation intelligent prediction model | |
| CN114897204A (en) | Method and device for predicting short-term wind speed of offshore wind farm | |
| CN113723670A (en) | Photovoltaic power generation power short-term prediction method with variable time window |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |