CN116861347A - Magnetic force abnormal data calculation method based on deep learning model - Google Patents
Magnetic force abnormal data calculation method based on deep learning model Download PDFInfo
- Publication number
- CN116861347A CN116861347A CN202310581807.8A CN202310581807A CN116861347A CN 116861347 A CN116861347 A CN 116861347A CN 202310581807 A CN202310581807 A CN 202310581807A CN 116861347 A CN116861347 A CN 116861347A
- Authority
- CN
- China
- Prior art keywords
- data
- magnetic force
- deep learning
- learning model
- magnetic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 112
- 238000013136 deep learning model Methods 0.000 title claims abstract description 76
- 238000004364 calculation method Methods 0.000 title claims abstract description 18
- 230000005484 gravity Effects 0.000 claims abstract description 11
- 238000001556 precipitation Methods 0.000 claims abstract description 11
- 238000000034 method Methods 0.000 claims description 30
- 238000012360 testing method Methods 0.000 claims description 29
- 238000012549 training Methods 0.000 claims description 27
- 238000013528 artificial neural network Methods 0.000 claims description 21
- 230000008439 repair process Effects 0.000 claims description 14
- 230000000694 effects Effects 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 10
- 238000012952 Resampling Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 238000012886 linear function Methods 0.000 claims description 5
- 238000011160 research Methods 0.000 claims description 5
- 210000002569 neuron Anatomy 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 claims description 2
- 238000013507 mapping Methods 0.000 claims description 2
- 238000005070 sampling Methods 0.000 claims description 2
- 238000005259 measurement Methods 0.000 abstract description 12
- 238000012545 processing Methods 0.000 abstract description 7
- 230000005358 geomagnetic field Effects 0.000 abstract description 4
- 239000000463 material Substances 0.000 abstract description 4
- 230000004888 barrier function Effects 0.000 abstract description 2
- 230000008859 change Effects 0.000 abstract description 2
- 230000005856 abnormality Effects 0.000 abstract 1
- 230000000306 recurrent effect Effects 0.000 description 7
- 238000011176 pooling Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 5
- 125000004122 cyclic group Chemical group 0.000 description 3
- 238000011282 treatment Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 229910052500 inorganic mineral Inorganic materials 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000002844 melting Methods 0.000 description 1
- 230000008018 melting Effects 0.000 description 1
- 239000011707 mineral Substances 0.000 description 1
- 230000000704 physical effect Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Measuring Magnetic Variables (AREA)
Abstract
The invention discloses a magnetic force abnormal data calculation method based on a deep learning model, which is characterized in that the magnetic force abnormal data which cannot be obtained through field measurement, such as remote geographic positions, severe environments, military regulations, cost limits and the like, are effectively filled by constructing the deep learning model, based on vegetation, elevation, precipitation, outcrop lithology, gravity abnormality and magnetic force abnormal data and calling the deep learning model to predict and fit magnetic force abnormal information of a missing region; the scheme does not need to be obtained through field measurement, does not need to carry out related works such as assembly and processing of field measurement data, saves a large amount of manpower and material resources, and has very important significance and guiding value for searching underwater magnetic barriers, investigating marine geomagnetism, researching marine geomagnetic field distribution and change and the like.
Description
Technical Field
The invention belongs to the field of geomagnetism and ocean magnetic force measurement, and particularly relates to a magnetic force abnormal data calculation method based on a deep learning model.
Background
The ocean magnetic anomaly provides extremely important evidence for ocean evolution, can reflect detailed information of plate construction, and has great significance for researching ocean shell structures, geomagnetic field changes, related earth internal dynamic processes and ocean mineral resource distribution. For example, the magnetic anomalies obtained by measuring in the middle of the ocean in the ridge area, the banding features of which clearly record the history of the expansion of the ocean floor. However, in most areas, marine survey work is only done with sparsely distributed survey lines, and the large gaps between the survey lines are likely to cause ambiguity in the knowledge of magnetic anomaly characteristics, particularly the stretch characteristics of the magnetic stripe. In view of the above, it is an urgent need to create a magnetic anomaly map for the whole region.
The data drawn on the magnetic anomaly graphs at present are generally obtained by field measurement such as satellite magnetic measurement, ocean magnetic measurement, aviation magnetic measurement and the like, so that a large amount of manpower and material resources are consumed; the data processor consumes a great deal of time and energy to perform a series of treatments such as eliminating distortion points, gridding, stitching, electrode melting and the like so as to form gridding magnetic force abnormal data on the unified ground level surface; in addition, a large amount of manpower and material resources are required for field investigation, and in some areas, due to remote geographic positions, severe environments, military regulations or cost limitation, effective magnetic anomaly data cannot be obtained by actual measurement, so that a plurality of magnetic anomaly data blank areas exist in the areas.
Disclosure of Invention
The invention provides a magnetic force abnormal data calculation method based on a deep learning model, which aims to solve the defects of difficult acquisition of magnetic force abnormal data and the like in the prior art, and predicts and fits magnetic force abnormal information of a missing region by calling the deep learning model.
The invention is realized by adopting the following technical scheme: a magnetic force abnormal data calculation method based on a deep learning model comprises the following steps:
step A, constructing a training data set: resampling and classifying the abnormal magnetic force data and the data sets related to the same magnetic force, and manufacturing a data set to be repaired, a test data set and a training data set, wherein the data sets related to the same magnetic force comprise elevation data, precipitation data, vegetation data, outcrop lithology and abnormal gravity data;
step B, constructing a deep learning model: training according to parameters to be set and the selected neural network, wherein the parameters to be set comprise iteration times, the number of layers of the neural network, an activation function and the number of neurons of each layer of the neural network; the deep learning model structure and the principle are as follows:
(1) The deep learning model is constructed by combining a gate-controlled recurrent neural network (GRU) with full connection, and the gate-controlled recurrent neural network (GRU) solves the long-term dependence problem of the traditional time Recurrent Neural Network (RNN) and comprises N GRU layers, a Flatten layer, a full connection layer and an output layer, wherein N is more than or equal to 2 and less than or equal to 5;
(2) The method comprises the steps of taking a three-dimensional array of N multiplied by d, which is formed by magnetic anomaly related data in an N multiplied by N pixel window, as model input, acquiring magnetic force information of a data window through forward propagation learning of N layers of GRU network units, unidimensionally outputting the magnetic force information through a flat layer, mapping a magnetic force information feature space acquired by the GRU network to a sample mark space through a full connection layer, and finally outputting a repair value of a central pixel of the window through an output layer, thereby completing one-time repair of the central pixel value; traversing the whole scene data in a window sliding mode, and executing the operation on each window to repair the magnetic abnormal data in the missing area of the research area.
Step C, based on a trained deep learning model, repairing magnetic force abnormal data of a missing area of a research area is achieved;
further, in the step B: the calculation formula used by the gated recurrent neural network (GRU) is as follows:
update door: z t ＝σ(W z ·[h t-1 ,x t ])；
Reset gate: r is (r) t ＝σ(W r ·[h t-1 ,x t ])
Candidate hidden state:
the hidden state passed to the next moment:
wherein x is t Indicating the input information at the current moment, h t-1 Represents the hidden state of the last moment, h t Indicating a hidden state to be passed on to the next instant,representing candidate hidden states, r t Indicating reset gate, z t Representing an update gate, tanh represents tanh function, W represents a convolution calculation mode, W represents a weight matrix, and σ represents sigmoid activation function.
Further, in the step a, the resampling and classifying operations specifically adopt the following modes:
step A1, sampling a data set related to the same magnetic force data and a magnetic force abnormal data set to the same resolution through resampling operation, wherein all data are in the same longitude and latitude;
and A2, dividing all data into a data set to be repaired and a data set with magnetic force abnormal data according to whether the magnetic force abnormal data exist or not, wherein the data set to be repaired is the data set of the abnormal missing part of the magnetic force data, and then dividing the data set with the magnetic force abnormal data into a test data set and a training data set.
In the step B, the deep learning model based on the repair of the magnetic abnormal data is composed of a convolution layer, a pooling layer, a flat layer, a full connection layer and an output layer. The convolution layer has 64 convolution kernels, the dimension is N multiplied by N, the moving step length is 1, the filling mode selects the Same filling, then the relu activation function is adopted for activation, the pooling mode of the pooling layer adopts the maximum pooling, and the convolution layer and the pooling layer together form N GRU layers. In the embodiment, the gated cyclic neural network GRU of one of the LSTM long-short-term memory neural network variants is used for constructing a deep learning model on a Keras framework, and the three gates in the LSTM are different from each other, and only two gates in the GRU are reset gates and update gates respectively, so that the dependency relationship on a sequence with a long time step distance can be better captured, overfitting is not easy to occur, and the GRU has the advantages that the effect of the LSTM is maintained, the structure is simpler, a larger network is easier to create, and only two gates are used, and the model is faster in calculation performance and convenient to expand.
Further, in the step B, when model training is performed, a training data set is preprocessed by adopting a linear function normalization formula as follows:
to achieve equal scaling of the original data, where X norm For normalized data, X is the original data, X max 、X min Respectively, the maximum and minimum of the data in the original dataset.
Further, in the step B, for the effect of model training, a root mean square error is introduced to evaluate the effect of data fitting to analyze the proximity degree between the real data of the magnetic anomaly data and the fitting data, and the formula is as follows:
wherein, RMSE is the root mean square error, N is the number of data to be analyzed, model t Known is a test for true magnetic anomaly data for a dataset t Magnetic anomaly data calculated by a deep learning model for the test dataset.
Further, in the step B, the original magnetic force abnormal data output by the deep learning model is reversely normalized, and the actual magnetic force abnormal data output by the deep learning model is calculated:
X endout ＝X out ×(X max -X min )+X min
wherein X is endout X is the actually output magnetic force abnormal data out Normalized magnetic anomaly data, X, output for a deep learning model max To train maximum value of abnormal magnetic force data in data set, X min Is the minimum value of the magnetic force abnormal data in the training data set.
Further, in the step C, the magnetic force abnormal data fitted by the deep learning model is further processed to match with the region where the magnetic force abnormal data is missing, specifically: firstly, summarizing the fitted magnetic force abnormal data and the original magnetic force abnormal data, then, orderly arranging and outputting the processed magnetic force abnormal data according to the longitude and latitude sequence, and finally, using IDL software programming to process the file, and outputting a complete magnetic force abnormal data image.
Compared with the prior art, the invention has the advantages and positive effects that:
the scheme is based on vegetation, elevation, precipitation, outcrop lithology, gravity anomaly and magnetic anomaly data and invokes a deep learning model to predict and fit magnetic anomaly information of a missing region, so that magnetic anomaly data which cannot be obtained through field measurement, such as remote geographic positions, severe environments, military regulations, cost limits and the like, can be effectively filled; and the method does not need to be obtained through field measurement, does not need to carry out related works such as compilation and processing of field measurement data, saves a great deal of manpower and material resources, and has very important significance and guiding value for searching underwater magnetic barriers, investigating marine geomagnetism, researching marine geomagnetic field distribution and change and the like.
Drawings
FIG. 1 is a schematic flow chart of a method for calculating magnetic anomaly data according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a deep learning model dataset processing procedure according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a deep learning model establishment process according to an embodiment of the present invention;
FIG. 4 is a graph of dot density of test data and real data for a deep learning model according to an embodiment of the present invention;
FIG. 5 is an absolute value diagram of the error between the test data and the real data of the deep learning model according to the embodiment of the present invention;
FIG. 6 is a diagram of magnetic anomaly data for southeast Asia region according to an embodiment of the present invention, (a) is a complete magnetic anomaly data diagram; (b) a magnetic abnormal data deletion map; (c) a complete magnetic anomaly data final result graph.
Detailed Description
In order that the above objects, features and advantages of the invention will be more readily understood, a further description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced otherwise than as described herein, and therefore the present invention is not limited to the specific embodiments disclosed below.
The magnetic anomaly is a direct reflection of the geomagnetic field on the earth's surface and is mainly related to the composition, physical properties and the like of the rock. In the regional geological structure research, the invention creatively utilizes 5 data sets of space gravity anomaly data, elevation data, outcrop lithology data, precipitation data and vegetation data and magnetic anomaly data available around a data blank area, and utilizes a neural network to build a deep learning model for repairing the magnetic anomaly data on a Keras frame so as to calculate and obtain the magnetic anomaly data of the data blank area.
As shown in fig. 1, this embodiment proposes a method for predicting and fitting magnetic anomaly information of a missing area based on vegetation, elevation, precipitation, gravity anomaly and magnetic anomaly data and invoking a deep learning model, and in order to understand this patent scheme more clearly, this embodiment is described in detail by taking magnetic correlation data of a southeast asia correlation area as an example, and specifically includes the following steps:
1. building training data sets
Step 1, data acquisition: firstly, magnetic force abnormal data of a southeast Asia related area are downloaded, and then elevation data, precipitation data, vegetation data, outcrop lithology data and gravity abnormal data of the southeast Asia related area are selected as data sets related to the same magnetic force data. And classifying according to the data sources to provide sources for the data set of the magnetic abnormal data repair deep learning model.
Step 2, data processing: after the data is downloaded, the data is required to be subjected to resampling, classifying and other treatments, a high-quality test set and a training set are provided for the subsequent repair of the magnetic abnormal data, and the specific flow is shown in fig. 2, and the process is as follows:
(1) and (3) after the data in the step (1) are downloaded and classified, resampling the data set related to the magnetic force data and the magnetic force abnormal data set, and resampling the magnetic force abnormal data map and the elevation data map, the precipitation data map, the vegetation data map and the gravity abnormal data map to the same resolution.
(2) Extracting data, wherein all the data are in the same longitude and latitude, and classifying the data as follows: firstly, classifying the data sets into two main types according to whether magnetic abnormal data exist or not, wherein the first type is the data set of the abnormal missing part of the magnetic data and is marked as the data set to be repaired, and the second type is the data set with the magnetic abnormal data.
(3) Dividing the data set of the second type of magnetic abnormal data, numbering all the data sets with magnetic abnormal data according to the longitude and latitude emission sequence, randomly selecting a number position between zero and ten, extracting all the data at the position of the number in the data set, taking the data as a test data set of a deep learning model, and marking the rest data as a training data set required by the deep learning model.
Step 3: pretreatment of training sets: the data set is reasonably divided, precipitation data (average value in 2012 to average value in 2017), vegetation data (average value in 2012 to average value in 2017), elevation data, outcrop lithology data and gravity anomaly data are used as input data of a deep learning model, and magnetic anomaly data of an existing region are used as output data of the deep learning model.
Because the evaluation criteria of the sample data are different, the dimensionalization of the sample data is needed, and the neuron output saturation phenomenon which is easy to cause by the large unified evaluation criteria is realized. In order to avoid numerical problems, the network is converged quickly and the small numerical value in the output data is ensured not to be swallowed. The patent adopts linear function normalization (Min-Max scaling) to preprocess data, and the linear function normalization formula is as follows:
the method achieves an equal scale of the original data, wherein X norm For normalized data, X is the original data, X max 、X min Respectively, the maximum and minimum of the data in the original dataset.
2. Construction of deep learning model and call thereof
Step 4: building a model and training: in the embodiment, a variant of an LSTM neural network, namely a gating cyclic neural network GRU (Gated Recurrent Unit), is adopted to build a deep learning model based on magnetic abnormal data repair on a Keras framework; the calculation efficiency and the precision of the models containing different GRU layers are compared, the Root Mean Square Error (RMSE) between the predicted value and the true value of each model is trained to meet the precision requirement, and the deep learning model structure with the optimal calculation efficiency and precision and the relevant super parameters are saved as the optimal values and used as the model call of the magnetic force abnormal data of the subsequent repair designated area.
In this embodiment, when the RMSE tends to stabilize in the training process, the super-parameters in this training are determined as the optimal model parameters. In this example, three-dimensional array composed of magnetic force anomaly data, elevation data, precipitation data, vegetation data, outcrop lithology data and gravity anomaly data is used as input, and after calculation by N GRU layers, magnetic force anomaly data of a missing region is output by an output layer.
In the embodiment, a variant of an LSTM neural network, namely a gating cyclic neural network GRU (Gated Recurrent Unit), is adopted to build a deep learning model based on magnetic abnormal data repair on a Keras framework; the network of the model is composed of an N GRU layer, a Flatten layer, a full connection layer and an output layer. The model takes magnetic force abnormal data, elevation data, precipitation data, vegetation data, outcrop lithology data and gravity abnormal data of a 15 multiplied by 15 data window as input, magnetic force abnormal information of the data window is extracted through forward propagation learning of N layers of GRU network units, original magnetic force abnormal data is mapped to a hidden layer characteristic space, then multidimensional output of a convolution layer and a pooling layer is unidimensionally carried out through a flat layer, then the characteristic space obtained through calculation in front of a full-connection layer is mapped to a sample mark space, finally magnetic force abnormal data of a central pixel of the window is output through an output layer as a repair result, and one-time repair of a missing area in a single pixel is completed. The magnetic force abnormal data of the whole southeast Asia missing area can be repaired by traversing the process for all the pixels.
Further, in the step B: the calculation formulas used by the gated recurrent neural network (GRU) are shown in formulas (2) to (5):
update door:
z t ＝σ(W z ·[h t-1 ,x t ]) (2)
reset gate:
r t ＝σ(W r ·[h t-1 ,x t ])
(3)
candidate hidden state:
the hidden state passed to the next moment:
wherein x is t Indicating the input information at the current moment, h t-1 Represents the hidden state of the last moment, h t Indicating a hidden state to be passed on to the next instant,representing candidate hidden states, r t Indicating reset gate, z t Representing an update gate, tanh represents tanh function, W represents a convolution calculation mode, W represents a weight matrix, and σ represents sigmoid activation function.
In order to find out the structure and relevant super parameters of the optimal prediction model, the embodiment adjusts the number of GRU layers in the model from 2 to 5 when training the model, compares the speed and the precision of the model when the Root Mean Square Error (RMSE) of each model reaches the precision requirement under the GRU states with 2, 3, 4 and 5 layers, and designates the LSTM layer number and the relevant super parameters of the optimal model under the condition that the model prediction efficiency is highest. The best LSTM model for predicting the transparency and the sea surface temperature of the sea is determined by training the model, and comprises 3 LSTM layers. This step determines the calculation formula for RMSE used for the best model structure and associated hyper-parameters as follows:
the method can well analyze the closeness degree between the real data and the fitting data of the magnetic force abnormal data. Wherein, RMSE is the root mean square error, N is the number of data to be analyzed, model t True magnetic force abnormal constant for test data setAccording to the above, known t Magnetic anomaly data calculated by a deep learning model for the test dataset.
And according to the result of the error formula, adjusting each parameter of the deep learning model to enable the model to fit magnetic force abnormal data meeting the requirements.
Step 5: and calling the trained optimal deep learning model to a designated area, and inputting a data set to be repaired in the deep learning model to fill magnetic abnormal data in the area.
At this time, the output magnetic abnormal data is normalized data, so that the original normalized magnetic abnormal data output by the deep learning model needs to be reversely normalized, and reverse normalization reasoning is performed according to the linear function normalization of the input data, that is, the reverse normalization formula is obtained by reasoning in the formula (1) as follows:
X endout ＝X out ×(X max -X min )+X min (7)
the actual magnetic force abnormal data output by the deep learning model can be calculated by the formula, wherein X endout X is the actually output magnetic force abnormal data out Normalized magnetic anomaly data, X, output for a deep learning model max To train maximum value of abnormal magnetic force data in data set, X min Is the minimum value of the magnetic force abnormal data in the training data set.
After the above work, the deep learning model based on the magnetic force abnormal data repair is established, and the repaired magnetic force abnormal data can be obtained, and the establishment process of the deep learning model based on the magnetic force abnormal data is shown in fig. 3.
3. Deep learning model evaluation and model output magnetic force abnormal data processing
Step 6, evaluating a deep learning model: after a deep learning model based on magnetic anomaly data is established, the model needs to be evaluated. The present embodiment selects root mean square error (Root Mean Squared Error, RMSE) to evaluate the effect of fitting the data to the deep learning model. The evaluation formula of the deep learning model is shown in formula (6).
Wherein, RMSE is the root mean square error, N is the number of data to be analyzed, model t Known is a test for true magnetic anomaly data for a dataset t Magnetic anomaly data calculated by a deep learning model for the test dataset.
Model evaluation is performed by inputting a test data set into an established deep learning model, and fitting a set of magnetic anomaly data. The data was analyzed by dot density map with the true magnetometric anomaly data of the test dataset as shown in FIG. 4. The effect of the deep learning model fitting the magnetic force abnormal data can be intuitively seen through the dot density map. Wherein slop represents the slope, R 2 The degree of fitting of the magnetic force data of the test set and the real magnetic force data is represented, and n is the number of the magnetic force data of the test set. As can be seen from fig. 4, the magnetic force abnormal data of the test data set has higher fitting degree with the real magnetic force abnormal data, which indicates that the fitting effect of the deep learning model is better.
In order to better verify the effect of the invention, the effect of fitting the deep learning model is emphasized and analyzed, 41191 data are selected as test sets, the number of the data sets accounts for ten percent of the total data sets (comprising a training set and a test set), and the test sets can effectively reflect the model effect. The specific method comprises the following steps: the magnetic force abnormal data trained by the deep learning model is subtracted from the real magnetic force abnormal data of the test data set, and the data is subjected to drawing analysis after taking an absolute value, as shown in fig. 5.
As can be seen from FIG. 5, the absolute value of the error between the magnetic force anomaly data fitted by the deep learning model and the real magnetic force anomaly data is basically between 0nT and 50 nT. Wherein, the data between 0 and 50nT account for 97.55% of the total test data, and the data with the absolute value of the error of the magnetic force abnormal data fitted by the model above 50nT is considered as abnormal data. Because the abnormal data is less and only accounts for 2.45% of the total test data, the magnetic abnormal data fitted by the model basically meets the requirements.
As can be seen intuitively by analyzing fig. 5, the ratio of the data of the absolute value of the error data between 0 and 50nT to the total data is very large. As can be seen from the examination data, the magnetic force abnormal data with the absolute value of the error of 10nT or less is better. The absolute value of the error is between 0 and 10nT, which is 30000, accounting for 72.83% of the total test data set. The magnetic force abnormal data above 10nT are relatively large in error, and the data volume of the magnetic force abnormal data is 27.17% of the total test data set. Therefore, the total error of the magnetic force abnormal data fitted by the deep learning model is within 30%, and the magnetic force abnormal data meets the precision requirement.
Step 7, the subsequent processing of magnetic force fitting data: the magnetic force abnormal data fitted by the deep learning model also needs to be further processed so as to be matched with the region where the magnetic force abnormal data is missing. The specific treatment process is as follows: firstly, summarizing the fitted magnetic force abnormal data and the original magnetic force abnormal data, then, arranging the processed magnetic force abnormal data in order according to longitude and latitude sequences, outputting a TXT text file, finally, using IDL software programming to process the file, and outputting a TIF format image of the magnetic force abnormal data of the national land areas such as the southeast Asian Burmese, laos, cambodia, indonesia, philippines and the like, wherein the TIF format image is shown in fig. 6 (a). The TIF format image of the magnetic anomaly data of the untouched regions of the land areas of China such as the southeast Asian Burma, laos, cambodia, indonesia and Philippines is shown in FIG. 6 (b). And finally outputting a TIF format image of the complete magnetic force abnormal data of the southeast Asia region, and processing the TIF format image by the Arcmap software as shown in fig. 6 (c).
The present invention is not limited to the above-mentioned embodiments, and any equivalent embodiments which can be changed or modified by the technical content disclosed above can be applied to other fields, but any simple modification, equivalent changes and modification made to the above-mentioned embodiments according to the technical substance of the present invention without departing from the technical content of the present invention still belong to the protection scope of the technical solution of the present invention.
Claims (7)
1. The magnetic force abnormal data calculation method based on the deep learning model is characterized by comprising the following steps of:
step A, constructing a training data set: resampling and classifying the abnormal magnetic force data and the data sets related to the same magnetic force, and manufacturing a data set to be repaired, a test data set and a training data set, wherein the data sets related to the same magnetic force comprise elevation data, precipitation data, vegetation data, outcrop lithology and gravity abnormal data;
step B, constructing a deep learning model: training according to parameters to be set and the selected neural network, wherein the parameters to be set comprise iteration times, the number of layers of the neural network, an activation function and the number of neurons of each layer of the neural network; the deep learning model is constructed on a Keras framework by adopting a gated circulating neural network and comprises an N-layer LSTM layer, a layer of Flatten layer, a layer of full-connection layer and a layer of output layer;
and C, based on a deep learning model, repairing magnetic force abnormal data of a research area is realized, and the principle is as follows:
the method comprises the steps of inputting a three-dimensional array of N multiplied by d, which is formed by magnetic anomaly related data in an N multiplied by N pixel window, serving as a deep learning model, acquiring magnetic force information of the data window through forward propagation learning of N layers of GRU neural network units, unidimensionally outputting the magnetic force information through a layer of flame, mapping a magnetic force information characteristic space output before to a sample mark space through a full connection layer, and finally outputting values of a missing area of the N multiplied by N data window through an output layer, so as to finish one-time repair of a central pixel value; traversing the whole scene data in a window sliding mode, and executing the operation on each window to realize calculation of magnetic abnormal data of a research area.
2. The method for computing magnetic anomaly data based on the deep learning model according to claim 1, wherein the method comprises the following steps: in the step a, the resampling and classifying operations specifically adopt the following modes:
step A1, sampling a data set related to the same magnetic force data and a magnetic force abnormal data set to the same resolution through resampling operation, wherein all data are in the same longitude and latitude;
and A2, dividing all data into a data set to be repaired and a data set with magnetic force abnormal data according to whether the magnetic force abnormal data exist or not, wherein the data set to be repaired is the data set of the abnormal missing part of the magnetic force data, and then dividing the data set with the magnetic force abnormal data into a test data set and a training data set.
3. The method for computing magnetic anomaly data based on the deep learning model according to claim 1, wherein the method comprises the following steps: in the step B, a deep learning model based on magnetic abnormal data repair is built on a Keras frame by adopting a gated circulating neural network GRU; the GRU comprises a reset gate and an update gate; the updating door is formed by combining a forgetting door and an input door and is used for controlling the degree to which the state information at the previous moment is brought into the current state, and the larger the value of the updating door is, the more the state information at the previous moment is brought into; the reset gate is used to control the degree to which state information at a previous time is ignored, a smaller value of the reset gate indicating more is ignored.
4. The method for computing magnetic anomaly data based on the deep learning model according to claim 1, wherein the method comprises the following steps: in the step B, when model training is performed, a training data set is preprocessed by adopting a linear function normalization formula as follows:
to achieve equal scaling of the original data, where X norm For normalized data, X is the original data, X max 、X min Respectively, the maximum and minimum of the data in the original dataset.
5. The method for computing magnetic anomaly data based on the deep learning model according to claim 1, wherein the method comprises the following steps: in the step B, for the effect of model training, root mean square error is introduced to evaluate the effect of data fitting to analyze the proximity degree between the real data of the magnetic anomaly data and the fitting data, and the formula is as follows:
wherein, RMSE is the root mean square error, N is the number of data to be analyzed, model t Known is a test for true magnetic anomaly data for a dataset t Magnetic anomaly data calculated by a deep learning model for the test dataset.
6. The method for computing magnetic anomaly data based on the deep learning model according to claim 5, wherein: in the step B, the original magnetic force abnormal data output by the deep learning model is reversely normalized, and the actual magnetic force abnormal data output by the deep learning model is calculated:
X endout ＝X out ×(X max -X min )+X min
wherein X is endout X is the actually output magnetic force abnormal data out Normalized magnetic anomaly data, X, output for a deep learning model max To train maximum value of abnormal magnetic force data in data set, X min Is the minimum value of the magnetic force abnormal data in the training data set.
7. The method for computing magnetic anomaly data based on the deep learning model according to claim 1, wherein the method comprises the following steps: in the step C, the magnetic force abnormal data fitted by the deep learning model is further processed to match with the region where the magnetic force abnormal data is missing, specifically: firstly, summarizing the fitted magnetic force abnormal data and the original magnetic force abnormal data, then, orderly arranging and outputting the processed magnetic force abnormal data according to the longitude and latitude sequence, and finally, using IDL software programming to process the file, and outputting a complete magnetic force abnormal data image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310581807.8A CN116861347B (en) | 2023-05-22 | 2023-05-22 | Magnetic force abnormal data calculation method based on deep learning model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310581807.8A CN116861347B (en) | 2023-05-22 | 2023-05-22 | Magnetic force abnormal data calculation method based on deep learning model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116861347A true CN116861347A (en) | 2023-10-10 |
| CN116861347B CN116861347B (en) | 2024-06-11 |
Family
ID=88225686
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310581807.8A Active CN116861347B (en) | 2023-05-22 | 2023-05-22 | Magnetic force abnormal data calculation method based on deep learning model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116861347B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117235449A (en) * | 2023-11-14 | 2023-12-15 | 湖北省气象服务中心（湖北省专业气象服务台） | Method for processing wind power abnormal data based on sigmoid curve and double-wrapping algorithm |
| CN117805923A (en) * | 2024-02-07 | 2024-04-02 | 长安大学 | Fracture plane position identification method and system based on Tanh function |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019208998A1 (en) * | 2018-04-27 | 2019-10-31 | 한국과학기술원 | Gru-based cell structure design robust to missing data and noise in time series data in recurrent neural network |
| US20210034949A1 (en) * | 2019-07-31 | 2021-02-04 | Dell Products L.P. | Systems and methods for predicting information handling resource failures using deep recurrent neural network with a modified gated recurrent unit having missing data imputation |
| CN113688770A (en) * | 2021-09-02 | 2021-11-23 | 重庆大学 | Long-term wind pressure missing data completion method and device for high-rise building |
| CN113936192A (en) * | 2021-10-22 | 2022-01-14 | 国网河北省电力有限公司经济技术研究院 | Power distribution network synchronous measurement missing data repairing method, terminal and storage medium |
| WO2022083009A1 (en) * | 2020-10-20 | 2022-04-28 | 浙江大学 | Customized product performance prediction method based on heterogeneous data error compensation fusion |
| CN114724045A (en) * | 2022-03-14 | 2022-07-08 | 陕西九州遥感信息技术有限公司 | Method for inverting submarine topography of shallow sea on land |
| CN114740540A (en) * | 2022-04-24 | 2022-07-12 | 中国人民解放军61540部队 | Method and system for constructing magnetic anomaly map of ridge region in ocean based on direction constraint |
| CN114819102A (en) * | 2022-05-19 | 2022-07-29 | 东南大学溧阳研究院 | GRU-based air conditioning equipment fault diagnosis method |
| US20220317669A1 (en) * | 2019-06-13 | 2022-10-06 | Tata Consultancy Services Limited | Method and system for industrial anomaly detection |
| CN115223365A (en) * | 2022-07-15 | 2022-10-21 | 北京市智慧交通发展中心(北京市机动车调控管理事务中心) | Road network speed prediction and anomaly identification method based on damping Holt model |
| CN115437027A (en) * | 2022-09-21 | 2022-12-06 | 中国地质调查局西安地质调查中心(西北地质科技创新中心) | Method and device for calculating bump gravity anomaly by using geological information variable density forward modeling |
| WO2022262500A1 (en) * | 2021-06-15 | 2022-12-22 | 哈尔滨工程大学 | Steof-lstm-based method for predicting marine environmental elements |
| US20230112749A1 (en) * | 2021-10-12 | 2023-04-13 | Wuhan University | Transformer state evaluation method based on echo state network and deep residual neural network |
-
2023
- 2023-05-22 CN CN202310581807.8A patent/CN116861347B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019208998A1 (en) * | 2018-04-27 | 2019-10-31 | 한국과학기술원 | Gru-based cell structure design robust to missing data and noise in time series data in recurrent neural network |
| US20220317669A1 (en) * | 2019-06-13 | 2022-10-06 | Tata Consultancy Services Limited | Method and system for industrial anomaly detection |
| US20210034949A1 (en) * | 2019-07-31 | 2021-02-04 | Dell Products L.P. | Systems and methods for predicting information handling resource failures using deep recurrent neural network with a modified gated recurrent unit having missing data imputation |
| WO2022083009A1 (en) * | 2020-10-20 | 2022-04-28 | 浙江大学 | Customized product performance prediction method based on heterogeneous data error compensation fusion |
| WO2022262500A1 (en) * | 2021-06-15 | 2022-12-22 | 哈尔滨工程大学 | Steof-lstm-based method for predicting marine environmental elements |
| CN113688770A (en) * | 2021-09-02 | 2021-11-23 | 重庆大学 | Long-term wind pressure missing data completion method and device for high-rise building |
| US20230112749A1 (en) * | 2021-10-12 | 2023-04-13 | Wuhan University | Transformer state evaluation method based on echo state network and deep residual neural network |
| CN113936192A (en) * | 2021-10-22 | 2022-01-14 | 国网河北省电力有限公司经济技术研究院 | Power distribution network synchronous measurement missing data repairing method, terminal and storage medium |
| CN114724045A (en) * | 2022-03-14 | 2022-07-08 | 陕西九州遥感信息技术有限公司 | Method for inverting submarine topography of shallow sea on land |
| CN114740540A (en) * | 2022-04-24 | 2022-07-12 | 中国人民解放军61540部队 | Method and system for constructing magnetic anomaly map of ridge region in ocean based on direction constraint |
| CN114819102A (en) * | 2022-05-19 | 2022-07-29 | 东南大学溧阳研究院 | GRU-based air conditioning equipment fault diagnosis method |
| CN115223365A (en) * | 2022-07-15 | 2022-10-21 | 北京市智慧交通发展中心(北京市机动车调控管理事务中心) | Road network speed prediction and anomaly identification method based on damping Holt model |
| CN115437027A (en) * | 2022-09-21 | 2022-12-06 | 中国地质调查局西安地质调查中心(西北地质科技创新中心) | Method and device for calculating bump gravity anomaly by using geological information variable density forward modeling |
Non-Patent Citations (3)
| Title |
|---|
| LI HE ET.AL: "Application of Deep Learning in Drainage Systems Monitoring Data Repair—A Case Study Using Con-GRU Model", 《WATER》, vol. 15, no. 8, 21 April 2023 (2023-04-21), pages 1 - 19 * |
| ZHONGCHANG JI ET.AL: "A traffic data imputing method based on multisource recurrent neural network", 《INTERNATIONAL CONFERENCE ON COMPUTER APPLICATION AND INFORMATION SECURITY》, vol. 12260, 24 May 2022 (2022-05-24), pages 1 - 6, XP060159288, DOI: 10.1117/12.2637835 * |
| 胡志伟: "深度学习融合SCADA数据的风电齿轮箱状态监测方法研究", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》, no. 10, 15 October 2022 (2022-10-15) * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117235449A (en) * | 2023-11-14 | 2023-12-15 | 湖北省气象服务中心（湖北省专业气象服务台） | Method for processing wind power abnormal data based on sigmoid curve and double-wrapping algorithm |
| CN117805923A (en) * | 2024-02-07 | 2024-04-02 | 长安大学 | Fracture plane position identification method and system based on Tanh function |
| CN117805923B (en) * | 2024-02-07 | 2024-07-09 | 长安大学 | Fracture plane position identification method and system based on Tanh function |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116861347B (en) | 2024-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116861347B (en) | Magnetic force abnormal data calculation method based on deep learning model | |
| US20220043182A1 (en) | Spatial autocorrelation machine learning-based downscaling method and system of satellite precipitation data | |
| CN105243435B (en) | A kind of soil moisture content prediction technique based on deep learning cellular Automation Model | |
| CN111814956B (en) | Multi-task learning air quality prediction method based on multi-dimensional secondary feature extraction | |
| CN110472597A (en) | Rock image rate of decay detection method and system based on deep learning | |
| CN111199270A (en) | Regional wave height forecasting method and terminal based on deep learning | |
| CN108447057A (en) | SAR image change detection based on conspicuousness and depth convolutional network | |
| CN114036841A (en) | Landslide incidence prediction method and system based on semi-supervised support vector machine model | |
| CN106228197A (en) | A kind of satellite image cloud amount recognition methods based on self adaptation extreme learning machine | |
| CN115730684A (en) | Air quality detection system based on LSTM-CNN model | |
| Li et al. | Land subsidence modelling using a long short-term memory algorithm based on time-series datasets | |
| CN115877483A (en) | Typhoon path forecasting method based on random forest and GRU | |
| JP2000002769A (en) | Method and device for predicting the spatial distribution of geological structure to prepare geological map | |
| CN116628886A (en) | Real-time optimization method for shield tunneling machine crossing construction parameters based on limited engineering data | |
| CN116310646A (en) | Remote sensing map image domain migration method, system and computer equipment based on GAN | |
| Zhang et al. | Atmospheric Environment Data Generation Method Based on Stacked LSTM-GRU | |
| CN114881171A (en) | Continental facies shale oil and rock facies type identification method and system based on convolutional neural network | |
| CN115392128A (en) | Method for simulating basin runoff by utilizing space-time convolution LSTM network | |
| Wong et al. | Confidence bounds of petrophysical predictions from conventional neural networks | |
| CN114063063A (en) | Geological disaster monitoring method based on synthetic aperture radar and point-like sensor | |
| CN117688821B (en) | Prediction method for side slope stability of dumping site | |
| CN117454762B (en) | Coal-penetrating tunnel face gas concentration prediction method of Markov-neural network | |
| Rau et al. | How can we quantify, explain, and apply the uncertainty of complex soil maps predicted with neural networks? | |
| Lin et al. | A Comparative Study of Machine Learning Models with Hyperparameter Optimization Algorithm for Mapping Mineral Prospectivity. Minerals 2021, 11, 159 | |
| CN116861769A (en) | Soil organic matter spatial distribution prediction method, system and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |