CN117076858B - Deep learning-based low-frequency geomagnetic strong interference suppression method and system - Google Patents
Deep learning-based low-frequency geomagnetic strong interference suppression method and system Download PDFInfo
- Publication number
- CN117076858B CN117076858B CN202311042458.9A CN202311042458A CN117076858B CN 117076858 B CN117076858 B CN 117076858B CN 202311042458 A CN202311042458 A CN 202311042458A CN 117076858 B CN117076858 B CN 117076858B
- Authority
- CN
- China
- Prior art keywords
- noise
- signal
- frequency
- low
- magnetotelluric
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 90
- 230000001629 suppression Effects 0.000 title claims abstract description 28
- 238000013135 deep learning Methods 0.000 title claims abstract description 21
- 238000000605 extraction Methods 0.000 claims abstract description 40
- 238000005259 measurement Methods 0.000 claims abstract description 26
- 238000012549 training Methods 0.000 claims abstract description 25
- 238000013136 deep learning model Methods 0.000 claims abstract description 14
- 238000001914 filtration Methods 0.000 claims abstract description 8
- 230000009467 reduction Effects 0.000 claims abstract description 8
- 239000011159 matrix material Substances 0.000 claims description 23
- 238000012545 processing Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 16
- 238000013527 convolutional neural network Methods 0.000 claims description 16
- 230000004927 fusion Effects 0.000 claims description 14
- 238000000354 decomposition reaction Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 12
- 238000010276 construction Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 11
- 239000013598 vector Substances 0.000 claims description 11
- 230000004913 activation Effects 0.000 claims description 7
- 125000004122 cyclic group Chemical group 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 23
- 230000008901 benefit Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 230000009191 jumping Effects 0.000 description 4
- 239000007787 solid Substances 0.000 description 4
- 239000000654 additive Substances 0.000 description 3
- 230000000996 additive effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000000926 separation method Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000004445 quantitative analysis Methods 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 229910003460 diamond Inorganic materials 0.000 description 1
- 239000010432 diamond Substances 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/30—Assessment of water resources
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Noise Elimination (AREA)
Abstract
The invention discloses a low-frequency geomagnetic strong interference suppression method and a system based on deep learning, wherein the method comprises the following steps: constructing a noise-containing signal sample of a low frequency band; establishing DnCNN-GRU deep learning models, and performing model training by using noise-containing signal samples to obtain a low-frequency principal component signal extraction model; extracting a low-frequency main component signal in the magnetotelluric actual measurement signal based on the low-frequency main component extraction model, and filtering the low-frequency main component signal to separate a high-frequency noise-containing signal; noise suppression is carried out on the high-frequency noise-containing signal to obtain a noise profile of the high-frequency noise-containing signal, and then noise is removed to obtain a noise-free high-frequency effective signal; and combining and splicing the low-frequency main component signal and the high-frequency effective signal to obtain the magnetotelluric signal after noise reduction. The invention accurately extracts the low-frequency main component signals in the magnetotelluric signals through DnCNN-GRU network, effectively avoids the loss of the low-frequency signals, improves the denoising efficiency and precision, and avoids the excessive removal of the effective signals.
Description
Technical Field
The invention belongs to the crossing field of artificial intelligence and geophysical magnetotelluric signal noise reduction technology, and particularly relates to a low-frequency magnetotelluric strong interference suppression method and system based on deep learning.
Background
The magnetotelluric sounding method is a geophysical exploration means widely applied to resource exploration and deep electrical structure research of the earth, and has the advantages of simple construction, small constraint of terrain, large detection depth and the like. Because the magnetotelluric sounding research is conducted on natural field source signals, the signals have the characteristics of strong randomness of components, weak signals and the like, and are extremely easy to be interfered by various electromagnetism; along with the advancement of modern society, noise interference sources are increased continuously, and the noise interference influences later geological interpretation and inversion accuracy of a geological structure, and even serious errors are caused in investigation and judgment of important resources.
The traditional method for denoising the magnetotelluric signals comprises a remote reference station method and a Robust estimation method. Both methods have high requirements on the quality of the acquired data, and when a proper reference station cannot be found or the acquired data is affected by continuous noise, serious errors are caused to the denoising result. With the continuous development of modern signal processing technology, a large number of data processing methods introduce the combination forms of digital morphological filtering, wavelet transformation, empirical mode decomposition, compressed sensing, sparse decomposition and various methods in the denoising of magnetotelluric signals. The methods can realize noise suppression to a certain extent, but the methods process the data from the whole, and the loss of high-quality data signals can exist in the denoising process.
With the continuous improvement of hardware conditions in recent years, the artificial intelligence algorithm has received a great deal of attention from universities. Many geophysicists apply deep learning methods to the geophysical field. Therefore, how to utilize a deep learning machine learning algorithm to process the characteristics of the magnetotelluric data and realize suppression of magnetotelluric noise data is the research content of the invention, and the aim is to solve the technical problems of noise removal precision and efficiency to be optimized in the traditional magnetotelluric signal separation.
Disclosure of Invention
Aiming at the technical problems of low denoising efficiency, insufficient intelligentization of denoising means and excessive removal of effective signals existing in the traditional magnetotelluric signal-to-noise separation, the invention provides a low-frequency magnetotelluric strong interference suppression method and system based on deep learning. The method comprises the steps of extracting a low-frequency main component signal of magnetotelluric data by utilizing DnCNN-GRU network, avoiding the loss of useful signals in the denoising process, and simultaneously performing noise suppression on the separated high-frequency noise-containing signals by utilizing KSVD, thereby realizing multi-scale and multi-type noise suppression.
For this purpose, the invention provides the following technical scheme:
The low-frequency magnetotelluric strong interference suppression method based on deep learning is applied to low-frequency-band magnetotelluric signals, and comprises the following steps of:
Step 1: constructing a low-frequency-band magnetotelluric noise-containing signal sample;
Step 2: establishing DnCNN-GRU deep learning models, and performing model training by using the magnetotelluric noise-containing signal samples to obtain a low-frequency main component signal extraction model;
the input of the low-frequency main component signal extraction model is a magnetotelluric noise-containing signal, and the output of the low-frequency main component signal is the magnetotelluric noise-containing signal;
Step 3: extracting a low-frequency main component signal in the magnetotelluric actual measurement signal based on the low-frequency main component signal extraction model, filtering the low-frequency main component signal from the magnetotelluric actual measurement signal, and separating a high-frequency noise-containing signal;
The high-frequency noise-containing signal is a signal obtained by filtering a low-frequency main component signal from the magnetotelluric noise-containing signal; step 3, the actually measured signals of the magnetotelluric are the magnetotelluric noise signals to be processed;
Step 4: noise suppression is carried out on the high-frequency noise-containing signal to obtain a noise profile of the high-frequency noise-containing signal, and noise is removed to obtain a noise-free high-frequency effective signal;
Step 5: and combining and splicing the low-frequency main component signal and the noiseless high-frequency effective signal to obtain the noise-reduced magnetotelluric signal.
Further alternatively, the network of DnCNN-GRU deep learning models may be divided into DnCNN denoising convolutional neural network and GRU gating loop network;
Wherein, the DnCNN denoising convolutional neural network is composed of a convolutional layer 1, 20 CBR modules (CBR modules 1-20) and an output layer which are connected in sequence;
Wherein the convolution layer 1 is composed of a convolution layer and a ReLU activation function;
The CBR module consists of a convolution layer, a batch normalization layer and a ReLU activation function;
The output layer is formed by a convolution layer 2 and a linear connection layer 1.
The feature fusion layer 1 transversely splices the data output by the DnCNN denoising convolutional neural network and the data output by the convolutional layer 1 in the DnCNN denoising convolutional neural network, and the channel number of the spliced data is changed into the sum of the channel numbers of the original two data. And the characteristic fusion layer 2 transversely splices the output data of the linear connection layer 2 in the GRU gating circulating network with the data output by the convolution layer 1 in the DnCNN denoising convolution neural network, and the channel number of the spliced data is changed into the sum of the channel numbers of the original two data.
Further alternatively, the GRU gating cyclic network is composed of two GRU units and two linear connection layers, after the output data of the DnCNN denoising convolutional neural network is input to the GRU gating cyclic network, the output data is sequentially processed by the two GRU units and then is input to the two linear connection layers, and the output of the last linear connection layer is the low-frequency main component signal.
Further alternatively, the GRU unit structure is as follows:
zt＝σ(Wz·[ht-1,xt])
rt＝σ(Wr·[ht-1,xt])
Wherein x t represents a t moment input signal, h t and h t-1 represent hidden layer information of the current moment and the last moment respectively, and a reset gate and an update gate are represented by r t and z t respectively; Representing a hidden state calculated based on the reset gate; sigma is a sigmoid function, W z、Wr and W are z t、rt and/or W respectively Is used to calculate the weight of the model.
Further alternatively, the step 4 is performed as follows:
step 4-1: processing the separated high-frequency noise-containing signals based on KSVD dictionary learning to obtain an overcomplete dictionary reflecting the noise profile characteristics of the high-frequency noise-containing signals;
Step 4-2: and updating an overcomplete dictionary, sparsely representing a noise contour in the high-frequency noise-containing signal by adopting an orthogonal matching algorithm OMP, and removing the noise contour from the high-frequency noise-containing signal to obtain a noise-free high-frequency effective signal.
Further alternatively, the step 4-1 is performed as follows: dividing the high-frequency noise-containing signal according to equal intervals to obtain a data matrix to be processed, and randomly selecting K column vectors in the data matrix to be processed as an initializing overcomplete dictionary for KSVD dictionary learning; then, carrying out sparse coding on the basis of the data matrix to be processed and an initial overcomplete dictionary to obtain a sparse representation coefficient; obtaining a final overcomplete dictionary by alternately updating the overcomplete dictionary and the sparse representation coefficients;
The execution process of the step 4-2 is as follows: based on the overcomplete dictionary, performing sparse decomposition on a data matrix to be processed by adopting an orthogonal matching algorithm OMP to obtain a sparse representation coefficient matrix, and multiplying the overcomplete dictionary by the sparse representation coefficient matrix to obtain a noise profile in the high-frequency noise-containing signal; and removing the noise outline from the high-frequency noise-containing signal to obtain a noise-free high-frequency effective signal.
The invention also provides a system based on the method, which comprises:
The sample construction module is used for constructing a low-frequency-band magnetotelluric noise-containing signal sample;
the module construction module is used for constructing DnCNN-GRU deep learning models, and performing model training by using the magnetotelluric noise-containing signal samples to obtain a low-frequency main component signal extraction model;
the input of the low-frequency main component signal extraction model is a magnetotelluric noise-containing signal, and the output of the low-frequency main component signal is the magnetotelluric noise-containing signal;
The signal extraction module is used for extracting a low-frequency main component signal in the magnetotelluric actual measurement signal based on the low-frequency main component signal extraction model, filtering the low-frequency main component signal from the magnetotelluric actual measurement signal, and separating a high-frequency noise-containing signal;
the noise suppression module is used for performing noise suppression on the high-frequency noise-containing signal to obtain a noise profile of the high-frequency noise-containing signal, and further removing noise to obtain a noise-free high-frequency effective signal;
And the merging module is used for merging and splicing the low-frequency main component signal and the noiseless high-frequency effective signal to obtain a noise-reduced magnetotelluric signal.
The invention also provides a system based on the method, which at least comprises: a signal detector and a processor;
the signal detector is arranged in the detection area and is used for collecting the magnetotelluric actual measurement signals;
loading or calling a computer program corresponding to the steps 1-5 in the processor to realize noise reduction of the magnetotelluric actual measurement signal; or loading or calling a pre-built low-frequency main component signal extraction model and a computer program corresponding to the steps 3-5, so as to reduce the noise of the magnetotelluric actual measurement signal.
The invention also provides an electronic terminal, which at least comprises:
One or more processors;
and a memory storing one or more computer programs;
wherein the processor invokes the computer program to implement:
a method for suppressing magnetotelluric strong interference based on deep learning.
The present invention also provides a computer-readable storage medium storing a computer program that is called by a processor to implement:
a method for suppressing magnetotelluric strong interference based on deep learning.
Advantageous effects
Compared with the prior art, the invention has the advantages that:
According to the low-frequency magnetotelluric strong interference suppression method based on deep learning, the noise reduction effect is improved by deep learning machine learning, namely, a low-frequency main component signal extraction model is firstly constructed through DnCNN-GRU network, the low-frequency main component signals in magnetotelluric signals are accurately extracted, so that high-frequency noise-containing signals are separated, targeted noise suppression is carried out on the high-frequency noise-containing signals, finally, a noiseless high-frequency effective signal is obtained, and the noiseless high-frequency effective signal is combined and spliced with the separated low-frequency main component signals, so that the noise suppression of magnetotelluric actual measurement signals is realized, the denoising efficiency and the denoising precision are improved, and excessive removal of the effective signals is avoided.
The DnCNN network shows an unusual effect in the fields of hidden feature extraction and image denoising, and shows the special global feature capturing capability. The GRU network aims at solving the problems of gradient disappearance, gradient explosion and the like generated in the long-term training process of the traditional cyclic neural network, and has the special strong learning ability of the dynamic relationship of the time sequence, so that the GRU network has good effect in the time sequence processing field. The low-frequency main component of the low-frequency magnetotelluric signal has large amplitude and strong energy, is dominant in the whole signal, and has the regularity of continuous and slow change along with time. DnCNN network extracts low-frequency principal component by means of its global feature capturing capability, while GRU network can accurately learn the law of principal component variation with time, thereby ensuring the accuracy and reliability of extracted principal component. The invention combines the advantages of the two networks, skillfully combines the two deep learning models according to the characteristics of MT signals, can effectively capture the low-frequency effective main component of the magnetotelluric signal, accurately realize the signal extraction of the low-frequency main component, further separate out the high-frequency noise-containing signal, and simultaneously can improve the precision of KSVD on the denoising of the high-frequency noise-containing signal. In addition, in the feasible scheme provided by the invention, two feature fusion layers are creatively added in the traditional GRU network, and data before and after DnCNN processing are connected to different layers of the GRU gating circulation network in a jumping manner, so that key features can be prevented from being lost due to convolution, network degradation, namely, feature loss caused by more layers is prevented, and the method is more suitable for the application field requirements of the invention.
It should be emphasized that the technical solution of the present invention is different from the conventional method that various means are used to attempt to extract all the effective signals at once or to suppress all the noise. The invention develops a new way to extract the low-frequency effective main component in MT signals by deep learning. Compared with the traditional method, the deep learning extraction method has the advantages that the low-frequency main component is smoother and more accurate, so that the loss of the low-frequency signal is effectively avoided, and the separated high-frequency noise-containing signal is subjected to targeted processing.
Drawings
FIG. 1 is a denoising flow chart of a method for suppressing magnetotelluric interference based on DnCNN-GRU and KSVD.
Fig. 2 is a typical sample display diagram of a sample library, (a) is additive impulse noise, (b) is additive square wave noise, and (c) is additive charge and discharge noise.
FIG. 3 shows DnCNN-GRU-KSVD model diagrams, and (a) and (b) show two types of DnCNN-GRU-KSVD model diagrams.
FIG. 4 is a training graph of a deep learning model, (a) a graph of DnCNN-GRU, (b) a graph of DnCNN, and (c) a graph of GRU.
Fig. 5 is a diagram of denoising effect of synthetic data by different methods, (a) is synthetic noisy data, (b) is original high quality data, c is DnCNN-GRU network combined with KSVD denoising effect, d is DnCNN network combined with KSVD denoising effect, e is GRU network combined with KSVD denoising effect, and f is KSVD denoising effect.
Table one shows the quantitative analysis and comparison of the denoising of the synthesized data.
Fig. 6 shows apparent resistivity-phase changes for noiseless data, noisy data, and noisy data denoised using different methods.
Fig. 7 shows the effect of extracting the low-frequency signal from the measured data, (a) shows the original signal curve, (b) shows the low-frequency principal component signal extracted by the method of the present invention.
Fig. 8 is a graph of the effect of the method of comparing measured data before and after denoising, (a) is an original signal curve, and (b) is a signal after denoising.
Fig. 9 shows apparent resistivity and phase change before and after denoising of measured data.
Detailed Description
The low-frequency magnetotelluric strong interference suppression method based on deep learning is used for reducing the noise of the magnetotelluric signals in the low frequency range, and particularly compared with the traditional method, the low-frequency magnetotelluric strong interference suppression method based on deep learning has the advantage that the low-frequency main components extracted by using the deep learning are smoother and more accurate, and can effectively avoid the loss of the low-frequency signals. For MT signals in the low frequency band (generally, the low frequency band is regarded as a low frequency band with a frequency lower than 7.5Hz, and in other possible manners, the standard of the low frequency band may refer to the standard in the art), the low frequency principal component signal extraction model is used for extracting the low frequency principal component in the low frequency band MT noise-containing signal, and removing the high frequency noise-containing signal in the low frequency band MT signal separated from the low frequency principal component. The key of the technical scheme of the invention is to construct a low-frequency main component signal extraction module based on DnCNN-GRU network and to perform targeted noise suppression on the separated high-frequency noise-containing signal by using KSVD algorithm. The invention will be further illustrated with reference to examples.
Example 1:
The embodiment provides a low-frequency magnetotelluric strong interference suppression method based on deep learning, which comprises the following steps:
s1: and constructing a low-frequency-band geoelectric noise signal sample.
In this embodiment, high-quality signal samples are manufactured in segments according to the size of 1×1500 for the data of the non-interference measuring points (the high-quality signals of the non-interference measuring points can reflect the dynamic trend of the low-frequency main component) according to the characteristics of the high-quality data and the noise data of the magnetotelluric data preferentially, noise waveforms with various magnitudes and various shapes are generated according to the noise shapes of the measured data, the noise waveforms are superimposed on the high-quality data to form noise-containing signals, and then noise-containing signal samples with the size of 1×1500 are generated, and a noise-containing signal-effective signal sample library is manufactured in one-to-one correspondence with the original high-quality signals before the noise waveforms are superimposed (the purpose of the sample library is to learn the characteristic trend of the low-frequency component through a network, and the effect can be achieved through stable extraction of the low-frequency signal trend), as shown in fig. 2. In this embodiment, the high-quality data is preferably subjected to an amplification process, so as to compensate for the lack of the high-quality data, and the other possible embodiments are not limited specifically. In addition, the construction method of the unconstrained noise waveform, such as pulse, square wave and triangular wave, or changing the position and the scale of the noise wave, can enrich the noise waveform.
One sample corresponds to the time sequence as follows:
Wherein i represents a sample sequence number, m is a data length of the obtained magnetotelluric high-quality data, n is a sample length, q is a positive integer smaller than n, and x (i-1)*q+1 is the (i-1) th data point q+1 in the magnetotelluric data.
S2: and building DnCNN-GRU deep learning models, and performing model training by using the noise-containing signal samples to obtain a low-frequency principal component signal extraction model. In this embodiment, when training a model, a sample data set is divided into at least a training set and a verification set data, and data feature learning is performed according to a batch size of 256×1×1500.
In the embodiment, the magnetotelluric signal input into the low-frequency main component signal extraction model is a one-dimensional time sequence signal, and the size is 1 multiplied by 1500; the output low-frequency principal component signal is also a one-dimensional time-series signal, and the size is 1×1500.
Wherein the network of DnCNN-GRU deep learning models includes DnCNN denoising convolutional neural network and GRU gating loop network. DnCNN the denoising convolutional neural network consists of a 23-layer network structure, and specifically consists of a convolutional layer 1, 20 CBR modules and an output layer. The convolution layer 1 is composed of a convolution layer and a ReLU activation function; each CBR module consists of a convolution layer, a batch normalization layer, and a ReLU activation function, the convolution kernel size is set to 1×3, and the number of channels is set to 64. The output layer is composed of a convolution layer and a linear connection layer. Based on the network structure, the signal input DnCNN to the denoising convolutional neural network is firstly subjected to a convolutional layer and a ReLU activation function, the convolutional core size of the layer is 1 multiplied by 3, and the channel number is set to be 1; then the output signal with the size of 1×1500 of one sample length is finally output through a convolution layer and a linear connection layer by a 20-layer repeated CBR module. It should be noted that the sizes and dimensions of the convolution kernel, convolution block, GRU gate control unit, and linear connection layer of the above model may be adjusted according to the effect of the actual data.
The improved GRU network part of the embodiment as shown in the graph (a) of FIG. 3 is composed of two GRU units, two linear connection layers and two feature fusion layers, wherein the two linear connection layers comprise a linear connection layer 2 and a linear connection layer 3; and the output data of the DnCNN denoising convolutional neural network and the data transmitted through the jumping connection of the convolutional layer 1 are jointly input to a characteristic fusion layer 1 in the improved GRU gating cyclic network, the data output by the characteristic fusion layer 1 are input to a linear connection layer 2 after being processed by the two GRU units, and the data output by the linear connection layer 2 and the data transmitted through the jumping connection of the convolutional layer 1 are jointly input to the characteristic fusion layer 2 and then are input to a linear connection layer 3, so that the low-frequency main component signal is obtained. The feature fusion layer 1 transversely splices the data output by the DnCNN denoising convolutional neural network and the data output by the convolutional layer 1 in the DnCNN denoising convolutional neural network, and the channel number of the spliced data is changed into the sum of the channel numbers of the original two data. And the characteristic fusion layer 2 transversely splices the output data of the linear connection layer 2 in the GRU gating circulating network with the data output by the convolution layer 1 in the DnCNN denoising convolution neural network, and the channel number of the spliced data is changed into the sum of the channel numbers of the original two data. As in the present embodiment, input_size=1500, output size output_size=1500, and hidden layer size is set to hidden_size=3000.
The above-mentioned GRU network is a preferred scheme of the present invention, which adds two feature fusion layers creatively in the traditional GRU network, and connects the data before and after DnCNN processing to different layers of the GRU gate control loop network in a jumping manner, so that key features can be prevented from being lost due to convolution, and network degradation, namely, feature loss caused by more layers, can be prevented.
It should be noted that in other possible embodiments, as shown in fig. 3 (b), the conventional GRU network is also adopted to meet the technical requirements of the present invention, and the difference is that the effect is weaker than that of the modified GRU network. Therefore, in other possible embodiments, the GRU gating loop network is composed of two GRU units and two linear connection layers, the output data of the DnCNN denoising convolutional neural network is input to the GRU gating loop network, then sequentially processed by the two GRU units and input to the two linear connection layers, and the output of the last linear connection layer is the low-frequency principal component signal.
The GRU unit consists of an update door and a forget door, and the formula of the GRU is as follows:
zt＝σ(Wz·[ht-1,xt])， (2)
rt＝σ(Wr·[ht-1,xt])， (3)
Wherein x t represents a t moment input signal, h t and h t-1 represent hidden layer information of the current moment and the last moment respectively, and a reset gate and an update gate are represented by r t and z t respectively; Representing a hidden state calculated based on the reset gate; sigma is a sigmoid function, W z、Wr and W are z t、rt and/or W respectively Is used to calculate the weight of the model. The update gate is used to control the extent to which the state information at the previous time is brought into the current state, a larger value of the update gate indicates that the state information at the previous time is brought more. The reset gate controls how much information of the previous state is written to the current candidate set/>
As shown in FIG. 3, a DnCNN-GRU-KSVD model diagram is shown, namely a sample library of the manufactured 175000 pairs of noise-high quality sample pairs is split into a training set and a verification set, 128000 sample pairs of the training set and 47000 sample pairs of the verification set, preprocessed data are transmitted into a DnCNN-GRU-KSVD model for training after being processed in random sequence, the model training super-parameters are set to be 500 for the maximum iteration times, the initial learning rate is set to be 10 -5, and the learning rate is multiplied by 0.9 after 10 iterations.
S3: and extracting a low-frequency main component signal in the magnetotelluric actual measurement signal based on the low-frequency main component extraction model, and forming a high-frequency noise-containing signal by residual data. Preprocessing data after segmenting the magnetotelluric actual measurement signal according to the size of 1 multiplied by 1500, and extracting main components of the actual measurement data by using a trained model; extracting a low-frequency main component signal, and storing the extracted low-frequency main component signal and the separated high-frequency noise-containing signal separately.
If the data is preprocessed in the training stage, the corresponding preprocessing is performed on the actually measured signal. The preferred pretreatment in this embodiment includes: and carrying out mean value removal and normalization processing, wherein after carrying out mean value removal processing on the data according to the equal division length, carrying out normalization processing on the whole data.
S4: processing the high-frequency noise-containing signal to be processed based on KSVD dictionary learning to obtain an overcomplete dictionary capable of reflecting the noise profile characteristics of the high-frequency noise-containing signal; and according to the overcomplete dictionary, adopting an orthogonal matching algorithm OMP, sparsely representing the noise profile in the high-frequency noise-containing signal, and further removing the noise to obtain a noise-free high-frequency effective signal.
S4-1: an overcomplete dictionary is constructed based on the high frequency noisy signal in step S3. Firstly, the high-frequency noise-containing signals separated in the step S3 are subjected to equidistant segmentation according to the step length m to construct a data matrix Y to be processed, and K column vectors are randomly selected from the data matrix to be processed to serve as an initial overcomplete dictionary D.
The KSVD dictionary learning is mainly divided into two steps, namely sparse coding coefficient updating based on OMP algorithm and overcomplete dictionary updating based on SVD singular value decomposition. Sparse coding stage: performing sparse decomposition on the data matrix to be processed through an OMP algorithm; dictionary updating stage: the sparse coefficient vector is fixed and updated column by column for atoms in the overcomplete dictionary D.
The KSVD dictionary learning in step 4-1 is a generalization of the K-means algorithm, and the essence of the KSVD dictionary learning is that under the limitation of a sparse constraint condition, the dictionary and the sparse representation coefficients are alternately and iteratively updated through singular value decomposition. Each column Y of given training data (i.e. matrix of data to be processed) Y represents a training sample, matrixA set of N training samples; x i represents the sparse coefficient vector corresponding to each training sample y i,/>For sparsely representing a set of coefficient vectors, i.e., a coefficient matrix, the dictionary learning process can be represented by an optimization problem:
Wherein D ε R (n*K) (N > K) is a sparse representation dictionary, K is the number of column vectors in the overcomplete dictionary D, namely the number of atoms in the dictionary, |·| F adopts the Frobenius norm estimation error, | 0 adopts the 0-norm estimation error, namely the number of non-zero elements, and T 0 is the maximum value of the number of non-zero elements in the sparse coefficient. The optimization problem is realized through alternate updating, which comprises two stages of coefficient solving and dictionary updating, wherein the coefficient solving is also called sparse coding.
According to the dictionary obtained by random initial, using OMP orthogonal matching algorithm, solving sparse coefficient vector x i of each training sample y i:
In the dictionary updating stage, the known sparse coefficient vector x i and the overcomplete dictionary D are fixed, the kth column D k of the dictionary is updated, the kth row vector multiplied by D k in the sparse coefficient x i is made to be x, and then the objective function of the sample set can be written as:
Wherein, Is the j-th row in the sparse matrix X.
Definition E k is the error produced by all but the kth atom, then formula (8) can be expressed as:
Error E k can be expressed as:
Definition of the definition Representing the index of the sample signal Y using the atom d k, to ensure convergence of the result, a matrix of Ω k as nxω k is defined, where (ω k (i), i) is a non-zero value and the rest are zero values, then equation (9) is equivalent to:
Wherein, Representation/>Remove line vector of zero value term,/>Representing the error train of the atom d k used in the coefficient encoding process.
Will beSingular Value Decomposition (SVD) is used to obtain the following decomposition expression:
d k in the initial overcomplete dictionary is updated by decomposing to get the first column of U. While coefficients are replaced by the product of the first column of matrix V and delta (1, 1) Updating j=j+1 until the OMP sparse decomposition reaches a specified threshold, stopping iteration, and obtaining an updated overcomplete dictionary.
S4-2: and (3) performing sparse decomposition on the noisy signals in the S3 by using the constructed overcomplete dictionary. And (3) carrying out sparse decomposition on the data sample to be processed in the step (4-1) by adopting an orthogonal matching algorithm (OMP) based on the updated overcomplete dictionary to obtain a noise interference profile in the high-frequency noise-containing signal, and further obtaining the high-frequency effective signal by removing the noise profile. And (3) through the mutual combination of an OMP orthogonal matching algorithm and KSVD dictionary learning, setting a proper threshold parameter, realizing the construction of a noise profile of the high-frequency noise-containing signal separated in the step (S3), and finally realizing the signal-to-noise separation of the high-frequency noise-containing signal to obtain a high-frequency effective signal.
The orthogonal matching algorithm (OMP) algorithm in S4-2 is improved by adding an orthogonal constraint on the basis of a matching pursuit algorithm (MP). Since this technology is a prior art, it is not disclosed in detail, and is briefly described as follows: according to sparse representation theory, signals may be represented linearly by certain atoms in a particular dictionary. The discrete signal y of length m in the data matrix to be processed can be represented by a specific dictionary D:
Where d j denotes the j-th column of the dictionary. This is an underdetermined problem with infinite solutions when K > m. In order to find the optimal sparsity coefficient, a constraint is required. Sparse representation theory reveals that the optimal solution has the smallest non-zero coefficients. The expression updates as follows:
In equation (14), 0 represents the norm of L 0, and the OMP solves the above equation as follows:
First initialize: x=0, setting epsilon 0 as a residual error threshold, and setting the initial residual error as r 0 =y; optimal atomic index set Primitive subset/>I=1; all atoms in the dictionary are then traversed, the best matching atomic index λ i＝arg max|<ri-1,dj > | is selected by using the inner product maximum, and the index set Λ i＝Λi-1∪{λi } and dictionary set are updated with λ i Updating the sparse coefficient by x=argmin||y- ψ jx||2; the residual ri=y- ψ i x is updated, and i=i+1. And finally judging whether the iteration stop condition r i≤ε0 is met, if yes, outputting a sparse coefficient x, otherwise, repeating the operation until the iteration stop condition is met.
S5: and combining and sequentially splicing the low-frequency main component signal and the high-frequency effective signal to obtain the noise-reduced magnetotelluric signal.
It should be noted that, in this embodiment, the S4 is used to perform targeted noise reduction on the high-frequency noise-containing signal, and a KSVD method is preferred, and in other possible embodiments, a technical means capable of extracting a noise profile of the high-frequency noise-containing signal falls within the protection scope of the present invention, and meets the requirements of the technical scheme of the present invention.
Wherein, as shown in FIG. 4, the comparison effect of the training of the three models DnCNN-GRU, dnCNN, GRU is shown. As can be seen from FIG. 4, the improved DnCNN-GRU network has faster convergence speed than the DnCNN model and the GRU model, and the training effect is better because the final convergence loss value is smaller.
The composite data denoising comparison is shown in fig. 5. In fig. 5, b is a selected high-quality MT signal, a is a noise-containing signal obtained by artificially synthesizing pulse-like, square-like, charge-discharge-like noise waveforms superimposed on b signal, c is a DnCNN-GRU-KSVD denoised signal, d is a DnCNN-KSVD denoised signal, e is a GRU-KSVD denoised signal, and f is a KSVD denoised signal. The method can obviously find that the signal after being denoised by DnCNN-GRU-KSVD method is similar to the original high-quality non-interference signal, and the effect after denoising by the other three methods is obviously different from that of the method of the invention. By further quantitative analysis, parameters such as signal-to-noise ratio (SNR), mean Square Error (MSE), normalized correlation (NCC), reconstruction Error (RE) and the like of the denoised data and the original data are compared. Compared with the original high-quality data, the signal-to-noise ratio of the data after noise addition is at least-4.965, the mean square error is at most 6175.3, the normalized correlation is at least 0.4991, and the reconstruction error is at most 1.771. After three methods are used for denoising, the signal to noise ratio after denoising is improved to 45.76, the mean square error is minimum, the normalized cross correlation is highest, and the reconstruction error is lowest. The analysis result of the synthetic data is obviously superior to other methods. The denoising advantage of the method of the invention is demonstrated.
TABLE 1
The apparent resistivity-phase contrast effect in the xy and yx directions of the selected near noiseless data segment, which was denoised by various methods, is shown in fig. 6. As shown in the figure, the hollow circular curve is a sounding curve of a noise-free signal, and the sounding curve of the original data is smooth; the solid inverted triangle curve is a sounding curve after noise addition, the apparent resistivity-phase becomes disordered after noise addition, the apparent resistivity phase within 0.01-10HZ is greatly influenced, and the data distortion is serious. The open square curve is the apparent resistivity-phase after denoising by using a Robust Robust estimation method, the data is slightly improved after the processing by the method, and serious distortion still exists in the apparent resistivity-phase in all directions. The solid diamond-shaped curve is the apparent resistivity-phase curve processed by the method provided by the invention, the apparent improvement can be obviously seen, and the processed apparent resistivity-phase curve almost coincides with the original noiseless signal data curve; this shows that the method provided by the invention has obvious denoising effect. The black dashed line is the data processed by the method and is further denoised by using Robust estimation of Robust, and the method of the invention is obviously improved on the synthesized data, so that the further Robust processing is almost consistent with the improvement of the processing of the method.
Fig. 7 shows the effect of the low-frequency signal extraction time domain of the actual measurement data of a certain Qinghai measuring point. The low-frequency main component signal extracted by the method accords with the change trend of the whole data, and the signals of the noiseless section and the interfered section can be extracted accurately without distinction.
As shown in fig. 8, which is a time domain effect comparison graph of the data after denoising of fig. 7, impulse-like noise and noise with different sizes are well suppressed, a high-quality signal segment of the denoised signal is effectively protected, multi-type noise interference is effectively removed, and an effective signal is greatly reserved.
The apparent resistivity and the phase curve before and after denoising the measured data of the measuring point can be compared, so that the method can effectively remove the interference in the magnetotelluric signal, and as shown in fig. 9, the apparent resistivity-phase curve change before and after denoising the original data of the measured data and the method adopting a Robust estimation method, a DnCNN-GRU-KSVD method and a DnCNN-GRU-KSVD combined with a Robust estimation method can be verified. As shown in the figure, the solid circle curve is the apparent resistivity-phase curve of the measured data, and the apparent noise interference and apparent resistivity-phase disorder can be seen in the range of 0.1-10 Hz. The hollow diamond curve is the denoising result by using the Robust estimation method, the integral curve is obviously improved, and the apparent resistivity curve has few point jump and weaker distortion. The solid square curve is the apparent resistivity-phase curve after denoising by the method provided by the invention, and the apparent resistivity-phase curve is smoother than the apparent resistivity-phase curve after denoising by the Robust estimation method, but a weaker jump point still exists near 10 Hz. The black dotted curve is a sounding curve effect diagram of denoising data after DnCNN-GRU-KSVD by further adopting a Robust estimation method, and the whole curve tends to be smooth after denoising. Through apparent resistance-phase comparison, the method provided by the invention can be used for obviously suppressing the data noise interference of the magnetotelluric, and the denoising effect is obvious, so that the data quality is obviously improved.
Example 2:
The embodiment provides a system based on a method for suppressing magnetotelluric strong interference, which comprises: the device comprises a sample construction module, a module construction module, a signal extraction module, a noise contour filtering module and a merging module.
The sample construction module is used for constructing a magnetotelluric noise-containing signal sample; the module construction module is used for constructing DnCNN-GRU deep learning models, and performing model training by using the magnetotelluric noise-containing signal samples to obtain a low-frequency main component signal extraction model; the signal extraction module is used for extracting a low-frequency effective main component signal in the magnetotelluric actual measurement signal based on the low-frequency main component signal extraction model, and the residual data form a high-frequency noise-containing signal; the noise suppression module is used for performing noise suppression on the high-frequency noise-containing signal to obtain a noise profile of the high-frequency noise-containing signal, and further removing noise to obtain a noise-free high-frequency effective signal; and the merging module is used for merging and splicing the low-frequency main component signal and the high-frequency effective signal to obtain the noise-reduced magnetotelluric signal. Wherein, in some embodiments, the noise suppression module comprises:
The overcomplete dictionary generating module is used for processing the separated high-frequency noise-containing signals based on KSVD dictionary learning to obtain an overcomplete dictionary reflecting the noise profile characteristics of the high-frequency noise-containing signals; and the noise reduction module is used for sparsely representing the noise profile in the high-frequency noise-containing signal by adopting an orthogonal matching algorithm OMP based on the updated overcomplete dictionary, and further removing the noise to obtain a noise-free high-frequency effective signal.
It should be understood that the implementation of the respective modules may be stated with reference to the foregoing method, and the above-described division of the functional modules is merely a division of logic functions, and there may be another division manner when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted or not performed. Meanwhile, the integrated units can be realized in a hardware form or a software functional unit form.
Example 3:
The embodiment provides a system based on a method for suppressing magnetotelluric strong interference, which comprises a signal detector and a processor;
the signal detector is arranged in the detection area and is used for collecting the magnetotelluric actual measurement signals;
Loading or calling a computer program corresponding to S1-S5 in the processor to realize noise reduction of the magnetotelluric actual measurement signal; or loading or calling a pre-constructed low-frequency main component signal extraction model and a computer program corresponding to S3-S5, so as to reduce the noise of the magnetotelluric actual measurement signal.
Example 4:
the embodiment provides an electronic terminal, which at least includes: one or more processors; and a memory storing one or more computer programs; wherein the processor invokes the computer program to implement: a method for suppressing magnetotelluric strong interference based on deep learning. The method specifically comprises the following steps:
s1: constructing a low-frequency-band magnetotelluric noise-containing signal sample;
S2: establishing DnCNN-GRU deep learning models, and performing model training by using the magnetotelluric noise-containing signal samples to obtain a low-frequency main component signal extraction model;
The input of the low-frequency main component signal extraction model is a magnetotelluric noise-containing signal sample, and the output is a low-frequency main component signal;
s3: extracting a low-frequency main component signal in the magnetotelluric actual measurement signal based on the low-frequency main component signal extraction model, and forming a high-frequency noise-containing signal by residual data;
S4: and processing the high-frequency noise-containing signal to be processed based on KSVD dictionary learning to obtain an overcomplete dictionary capable of reflecting the noise profile characteristics of the high-frequency noise-containing signal. Sparse decomposition is carried out on the separated matrix to be processed by adopting an orthogonal matching algorithm (OMP) based on the updated overcomplete dictionary to obtain a noise interference profile in the high-frequency noise-containing signal, and the noise profile is further removed from the high-frequency noise-containing signal to obtain a high-frequency effective signal;
s5: and combining and sequentially splicing the low-frequency main component signal and the high-frequency effective signal to obtain the noise-reduced magnetotelluric signal.
The memory may comprise high-speed RAM memory, and may also include a non-volatile defibrillator, such as at least one disk memory.
If the memory and the processor are implemented independently, the memory, the processor, and the communication interface may be interconnected by a bus and communicate with each other. The bus may be an industry standard architecture bus, an external device interconnect bus, or an extended industry standard architecture bus, among others. The buses may be classified as address buses, data buses, control buses, etc.
Alternatively, in a specific implementation, if the memory and the processor are integrated on a chip, the memory and the processor may communicate with each other through an internal interface.
It should be appreciated that in embodiments of the present invention, the Processor may be a central processing unit (Central Processing Unit, CPU), which may also be other general purpose Processor, digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), off-the-shelf Programmable gate array (Field-Programmable GATEARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The memory may include read only memory and random access memory and provide instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store information of the device type.
Example 5:
The present embodiment provides a computer-readable storage medium storing a computer program that is called by a processor to implement: a method for suppressing magnetotelluric strong interference based on deep learning. The method specifically comprises the following steps:
s1: constructing a low-frequency-band magnetotelluric noise-containing signal sample;
S2: establishing DnCNN-GRU deep learning models, and performing model training by using the magnetotelluric noise-containing signal samples to obtain a low-frequency main component signal extraction model;
The input of the low-frequency main component signal extraction model is a magnetotelluric noise-containing signal sample, and the output is a low-frequency main component signal;
s3: extracting a low-frequency main component signal in the magnetotelluric actual measurement signal based on the low-frequency main component signal extraction model, and forming a high-frequency noise-containing signal by residual data;
S4: processing the high-frequency noise-containing signal to be processed based on KSVD dictionary learning to obtain an overcomplete dictionary capable of reflecting the noise profile characteristics of the high-frequency noise-containing signal; obtaining a contour of noise interference in the high-frequency noise-containing signal by adopting an orthogonal matching algorithm (OMP) based on the updated overcomplete dictionary, and further obtaining a high-frequency effective signal by removing the noise contour;
s5: and combining and sequentially splicing the low-frequency main component signal and the high-frequency effective signal to obtain the noise-reduced magnetotelluric signal.
For a specific implementation of each step, please refer to the description of the foregoing method.
The readable storage medium is a computer readable storage medium, which may be an internal storage unit of the controller according to any one of the foregoing embodiments, for example, a hard disk or a memory of the controller. The readable storage medium may also be an external storage device of the controller, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD), or the like, which are provided on the controller. Further, the readable storage medium may also include both an internal storage unit and an external storage device of the controller. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium may also be used to temporarily store data that has been output or is to be output.
Based on such understanding, the technical solution of the present invention is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned readable storage medium includes: a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
It should be emphasized that the examples described herein are illustrative rather than limiting, and that this invention is not limited to the examples described in the specific embodiments, but is capable of other embodiments in accordance with the teachings of the present invention, as long as they do not depart from the spirit and scope of the invention, whether modified or substituted, and still fall within the scope of the invention.
Claims (9)
1. The low-frequency magnetotelluric strong interference suppression method based on deep learning is characterized by comprising the following steps of:
Step 1: constructing a low-frequency-band magnetotelluric noise-containing signal sample;
Step 2: establishing DnCNN-GRU deep learning models, and performing model training by using the magnetotelluric noise-containing signal samples to obtain a low-frequency main component signal extraction model;
the input of the low-frequency main component signal extraction model is a magnetotelluric noise-containing signal, and the output of the low-frequency main component signal is the magnetotelluric noise-containing signal;
Step 3: extracting a low-frequency main component signal in the magnetotelluric actual measurement signal based on the low-frequency main component signal extraction model, filtering the low-frequency main component signal from the magnetotelluric actual measurement signal, and separating a high-frequency noise-containing signal;
Step 4: noise suppression is carried out on the high-frequency noise-containing signal to obtain a noise profile of the high-frequency noise-containing signal, and noise is removed to obtain a noise-free high-frequency effective signal;
Step 5: combining and splicing the low-frequency main component signal and the noiseless high-frequency effective signal to obtain a noise-reduced magnetotelluric signal;
wherein, the network of DnCNN-GRU deep learning model can be divided into DnCNN denoising convolutional neural network and GRU gating cyclic network;
Wherein, the DnCNN denoising convolutional neural network is composed of a convolutional layer 1, 20 CBR modules (CBR modules 1-20) and an output layer which are connected in sequence;
Wherein the convolution layer 1 is composed of a convolution layer and a ReLU activation function;
The CBR module consists of a convolution layer, a batch normalization layer and a ReLU activation function;
The output layer is formed by a convolution layer 2 and a linear connection layer 1.
2. The method according to claim 1, characterized in that: the GRU gating circulating network consists of two GRU units, two linear connecting layers and two characteristic fusion layers;
wherein the two linear connection layers are a linear connection layer 2 and a linear connection layer 3; the data output by DnCNN denoising convolutional neural network and the data transmitted through the jump connection of the convolutional layer 1 are input to a characteristic fusion layer 1 in the GRU gating cyclic network together, and the data output by the characteristic fusion layer 1 are input to the two GRU units for processing and then are input to a linear connection layer 2;
the data output by the linear connection layer 2 and the data transmitted through the jump connection of the convolution layer 1 are input to the feature fusion layer 2 together, and then input to the linear connection layer 3, so that the low-frequency main component signal is obtained.
3. The method according to claim 1, characterized in that: the GRU gating circulating network consists of two GRU units and two linear connection layers, wherein the output data of the DnCNN denoising convolutional neural network is input to the GRU gating circulating network, then sequentially processed by the two GRU units and input to the two linear connection layers, and the output of the last linear connection layer is the low-frequency main component signal.
4. The method according to claim 1, characterized in that: the execution process of the step 4 is as follows:
step 4-1: processing the separated high-frequency noise-containing signals based on KSVD dictionary learning to obtain an overcomplete dictionary reflecting the noise profile characteristics of the high-frequency noise-containing signals;
Step 4-2: and updating an overcomplete dictionary, sparsely representing a noise contour in the high-frequency noise-containing signal by adopting an orthogonal matching algorithm OMP, and removing the noise contour from the high-frequency noise-containing signal to obtain a noise-free high-frequency effective signal.
5. The method according to claim 4, wherein: the execution process of the step 4-1 is as follows: dividing the high-frequency noise-containing signal according to equal intervals to obtain a data matrix to be processed, and randomly selecting K column vectors in the data matrix to be processed as an initial overcomplete dictionary for dictionary learning; then, carrying out sparse coding on the basis of the data matrix to be processed and an initial overcomplete dictionary to obtain a sparse representation coefficient; obtaining a final overcomplete dictionary by alternately updating the overcomplete dictionary and the sparse representation coefficients;
the execution process of the step 4-2 is as follows: based on the overcomplete dictionary, performing sparse decomposition on a data matrix to be processed by adopting an orthogonal matching algorithm OMP to obtain a sparse representation coefficient matrix, and multiplying the overcomplete dictionary by the sparse representation coefficient matrix to obtain a noise profile of the high-frequency noise-containing signal; and removing the noise outline from the high-frequency noise-containing signal to obtain a noise-free high-frequency effective signal.
6. A system based on the method of any one of claims 1-5, characterized in that: comprising the following steps:
The sample construction module is used for constructing a low-frequency-band magnetotelluric noise-containing signal sample;
the module construction module is used for constructing DnCNN-GRU deep learning models, and performing model training by using the magnetotelluric noise-containing signal samples to obtain a low-frequency main component signal extraction model;
the input of the low-frequency main component signal extraction model is a magnetotelluric noise-containing signal, and the output of the low-frequency main component signal is the magnetotelluric noise-containing signal;
The signal extraction module is used for extracting a low-frequency main component signal in the magnetotelluric actual measurement signal based on the low-frequency main component signal extraction model, filtering the low-frequency main component signal from the magnetotelluric actual measurement signal, and separating a high-frequency noise-containing signal;
the noise suppression module is used for performing noise suppression on the high-frequency noise-containing signal to obtain a noise profile of the high-frequency noise-containing signal, and further removing noise to obtain a noise-free high-frequency effective signal;
And the merging module is used for merging and splicing the low-frequency main component signal and the noiseless high-frequency effective signal to obtain a noise-reduced magnetotelluric signal.
7. A system based on the method of any one of claims 1-5, characterized in that: at least comprises: a signal detector and a processor;
the signal detector is arranged in the detection area and is used for collecting the magnetotelluric actual measurement signals;
loading or calling a computer program corresponding to the steps 1-5 in the processor to realize noise reduction of the magnetotelluric actual measurement signal; or loading or calling a pre-built low-frequency main component signal extraction model and a computer program corresponding to the steps 3-5, so as to reduce the noise of the magnetotelluric actual measurement signal.
8. An electronic terminal, characterized in that: at least comprises:
One or more processors;
and a memory storing one or more computer programs;
wherein the processor invokes the computer program to implement:
the method of any one of claims 1-5.
9. A computer-readable storage medium, characterized by: a computer program is stored, which is called by a processor to implement:
the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311042458.9A CN117076858B (en) | 2023-08-18 | 2023-08-18 | Deep learning-based low-frequency geomagnetic strong interference suppression method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311042458.9A CN117076858B (en) | 2023-08-18 | 2023-08-18 | Deep learning-based low-frequency geomagnetic strong interference suppression method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117076858A CN117076858A (en) | 2023-11-17 |
| CN117076858B true CN117076858B (en) | 2024-06-04 |
Family
ID=88714634
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311042458.9A Active CN117076858B (en) | 2023-08-18 | 2023-08-18 | Deep learning-based low-frequency geomagnetic strong interference suppression method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117076858B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117574056B (en) * | 2023-11-21 | 2024-05-10 | 中南大学 | Wide-area electromagnetic data denoising method and system based on hybrid neural network model |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106154344A (en) * | 2016-08-01 | 2016-11-23 | 湖南文理学院 | A kind of Magnetotelluric signal denoising method based on combined filter |
| CN110146929A (en) * | 2019-05-21 | 2019-08-20 | 东华理工大学 | Low frequency magnetotelluric data denoising method based on excessively complete dictionary Yu compressed sensing restructing algorithm |
| CN110147766A (en) * | 2019-05-21 | 2019-08-20 | 东华理工大学 | Based on the low frequency Magnetotelluric signal denoising method for moving constant sparse coding |
| CN110989020A (en) * | 2019-12-12 | 2020-04-10 | 核工业北京地质研究院 | Filtering method and system for noise interference of audio magnetotelluric data |
| CN112966611A (en) * | 2021-03-09 | 2021-06-15 | 中国科学技术大学 | Energy trace noise self-adaption method of DWT attention mechanism |
| CN113761805A (en) * | 2021-09-14 | 2021-12-07 | 东华理工大学 | Controllable source electromagnetic data denoising method, system, terminal and readable storage medium based on time domain convolution network |
| CN114429151A (en) * | 2021-12-16 | 2022-05-03 | 中南大学 | Magnetotelluric signal identification and reconstruction method and system based on depth residual error network |
| CN115017964A (en) * | 2022-08-09 | 2022-09-06 | 湖南师范大学 | Magnetotelluric signal denoising method and system based on attention mechanism sparse representation |
| CN115267899A (en) * | 2022-08-15 | 2022-11-01 | 河北地质大学 | DnCNN mixed seismic source seismic data separation method and system based on boundary preservation |
| US11599223B1 (en) * | 2020-03-13 | 2023-03-07 | Apple Inc. | System and machine learning method for separating noise and signal in multitouch sensors |
| CN115828069A (en) * | 2022-12-29 | 2023-03-21 | 电子科技大学 | End-to-end magnetic anomaly signal noise reduction method based on deep learning |
| CN116452918A (en) * | 2023-04-15 | 2023-07-18 | 西华大学 | Method and system for recovering countermeasure sample based on wavelet transformation and attention mechanism |
| CN116561515A (en) * | 2023-05-15 | 2023-08-08 | 吉林大学 | Power frequency noise suppression method based on cyclic neural network magnetic resonance signals |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10996372B2 (en) * | 2017-08-25 | 2021-05-04 | Exxonmobil Upstream Research Company | Geophysical inversion with convolutional neural networks |
| US20230057444A1 (en) * | 2021-08-04 | 2023-02-23 | Ciena Corporation | Compressing network data using Deep Neural Network (DNN) deployment |
-
2023
- 2023-08-18 CN CN202311042458.9A patent/CN117076858B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106154344A (en) * | 2016-08-01 | 2016-11-23 | 湖南文理学院 | A kind of Magnetotelluric signal denoising method based on combined filter |
| CN110146929A (en) * | 2019-05-21 | 2019-08-20 | 东华理工大学 | Low frequency magnetotelluric data denoising method based on excessively complete dictionary Yu compressed sensing restructing algorithm |
| CN110147766A (en) * | 2019-05-21 | 2019-08-20 | 东华理工大学 | Based on the low frequency Magnetotelluric signal denoising method for moving constant sparse coding |
| CN110989020A (en) * | 2019-12-12 | 2020-04-10 | 核工业北京地质研究院 | Filtering method and system for noise interference of audio magnetotelluric data |
| US11599223B1 (en) * | 2020-03-13 | 2023-03-07 | Apple Inc. | System and machine learning method for separating noise and signal in multitouch sensors |
| CN112966611A (en) * | 2021-03-09 | 2021-06-15 | 中国科学技术大学 | Energy trace noise self-adaption method of DWT attention mechanism |
| CN113761805A (en) * | 2021-09-14 | 2021-12-07 | 东华理工大学 | Controllable source electromagnetic data denoising method, system, terminal and readable storage medium based on time domain convolution network |
| CN114429151A (en) * | 2021-12-16 | 2022-05-03 | 中南大学 | Magnetotelluric signal identification and reconstruction method and system based on depth residual error network |
| CN115017964A (en) * | 2022-08-09 | 2022-09-06 | 湖南师范大学 | Magnetotelluric signal denoising method and system based on attention mechanism sparse representation |
| CN115267899A (en) * | 2022-08-15 | 2022-11-01 | 河北地质大学 | DnCNN mixed seismic source seismic data separation method and system based on boundary preservation |
| CN115828069A (en) * | 2022-12-29 | 2023-03-21 | 电子科技大学 | End-to-end magnetic anomaly signal noise reduction method based on deep learning |
| CN116452918A (en) * | 2023-04-15 | 2023-07-18 | 西华大学 | Method and system for recovering countermeasure sample based on wavelet transformation and attention mechanism |
| CN116561515A (en) * | 2023-05-15 | 2023-08-08 | 吉林大学 | Power frequency noise suppression method based on cyclic neural network magnetic resonance signals |
Non-Patent Citations (4)
| Title |
|---|
| Deep Learning Optimized Dictionary Learning and Its Application in Eliminating Strong Magnetotelluric Noise;Guang Li 等;《Minerals》;20220812;第1-22页 * |
| Low-Frequency Magnetotelluric Data Denoising Using Improved Denoising Convolutional Neural Network and Gated Recurrent Unit;Guang Li等;《IEEE Transactions on Geoscience and Remote Sensing》;20240327;第62卷;第1-15页 * |
| Low-Frequency Noise Suppression Method Based on Improved DnCNN in Desert Seismic Data;Yuxing Zhao等;《IEEE Geoscience and Remote Sensing Letters》;20181130;第19卷(第5期);第811-815页 * |
| The Combination of GRU and Dense Block for Image Denoising Network;Zhu, F等;《Exploration of Novel Intelligent Optimization Algorithms. ISICA 2021》;20220801;第95-102页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117076858A (en) | 2023-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107506712B (en) | Human behavior identification method based on 3D deep convolutional network | |
| Qu et al. | Radar signal intra-pulse modulation recognition based on convolutional denoising autoencoder and deep convolutional neural network | |
| CN109870729B (en) | Deep neural network magnetic resonance signal denoising method based on discrete cosine transform | |
| CN111723701B (en) | Underwater target identification method | |
| CN117076858B (en) | Deep learning-based low-frequency geomagnetic strong interference suppression method and system | |
| CN113094993B (en) | Modulation signal denoising method based on self-coding neural network | |
| CN113158553B (en) | Magnetotelluric signal noise suppression method and system based on CNN-LSTM | |
| CN113642484B (en) | Magnetotelluric signal noise suppression method and system based on BP neural network | |
| CN113657491A (en) | Neural network design method for signal modulation type recognition | |
| Chen et al. | TEMDNet: A novel deep denoising network for transient electromagnetic signal with signal-to-image transformation | |
| CN112115821B (en) | Multi-signal intelligent modulation mode identification method based on wavelet approximate coefficient entropy | |
| CN113761805B (en) | Controllable source electromagnetic data denoising method, system, terminal and readable storage medium based on time domain convolution network | |
| CN114492521A (en) | Intelligent lithology while drilling identification method and system based on acoustic vibration signals | |
| Qian et al. | Ground truth-free 3-D seismic random noise attenuation via deep tensor convolutional neural networks in the time-frequency domain | |
| CN110569728A (en) | Kernel signal extraction method based on dictionary training and orthogonal matching pursuit | |
| Qian et al. | DTAE: Deep tensor autoencoder for 3-D seismic data interpolation | |
| CN114429151A (en) | Magnetotelluric signal identification and reconstruction method and system based on depth residual error network | |
| CN115146667A (en) | Multi-scale seismic noise suppression method based on curvelet transform and multi-branch deep self-coding | |
| Wang et al. | An adaptive learning image denoising algorithm based on eigenvalue extraction and the GAN model | |
| CN113568058B (en) | Magnetotelluric signal-noise separation method and system based on multi-resolution singular value decomposition | |
| Cao et al. | Sparse representation of classified patches for CS-MRI reconstruction | |
| CN115017964B (en) | Magnetotelluric signal denoising method and system based on attention mechanism sparse representation | |
| Zhang et al. | Signal processing using dictionaries, atoms, and deep learning: A common analysis-synthesis framework | |
| CN114626412A (en) | Multi-class target identification method and system for unattended sensor system | |
| CN113722893A (en) | Seismic record inversion method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |