CN117056680A - Data noise reduction and signal detection method, device and system and storage medium - Google Patents
Data noise reduction and signal detection method, device and system and storage medium Download PDFInfo
- Publication number
- CN117056680A CN117056680A CN202310892526.4A CN202310892526A CN117056680A CN 117056680 A CN117056680 A CN 117056680A CN 202310892526 A CN202310892526 A CN 202310892526A CN 117056680 A CN117056680 A CN 117056680A
- Authority
- CN
- China
- Prior art keywords
- data
- training
- neural network
- signal
- noise reduction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000009467 reduction Effects 0.000 title claims abstract description 148
- 238000001514 detection method Methods 0.000 title claims abstract description 116
- 238000012549 training Methods 0.000 claims abstract description 172
- 238000013528 artificial neural network Methods 0.000 claims abstract description 99
- 238000012545 processing Methods 0.000 claims abstract description 52
- 230000004927 fusion Effects 0.000 claims abstract description 50
- 238000000034 method Methods 0.000 claims abstract description 46
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 40
- 230000007246 mechanism Effects 0.000 claims abstract description 22
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 230000006870 function Effects 0.000 claims description 33
- 230000015654 memory Effects 0.000 claims description 24
- 230000008569 process Effects 0.000 claims description 17
- 238000002156 mixing Methods 0.000 claims description 12
- 230000002087 whitening effect Effects 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 7
- 230000035945 sensitivity Effects 0.000 claims description 7
- 238000001228 spectrum Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 5
- 238000004088 simulation Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 230000004913 activation Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 125000004122 cyclic group Chemical group 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- VNWKTOKETHGBQD-UHFFFAOYSA-N methane Chemical compound C VNWKTOKETHGBQD-UHFFFAOYSA-N 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 239000003245 coal Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000005553 drilling Methods 0.000 description 1
- 239000007789 gas Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 239000003345 natural gas Substances 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000003209 petroleum derivative Substances 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
- G06F18/15—Statistical pre-processing, e.g. techniques for normalisation or restoring missing data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
Abstract
The application discloses a method, a device, a system and a storage medium for data noise reduction and signal detection, which relate to the technical field of data and signal processing and comprise the following steps: generating analog data with low signal-to-noise ratio; preprocessing the mixed signal in the analog data to obtain training data; fusing the first convolutional neural network, the second convolutional neural network, the self-attention mechanism and the multi-layer fully-connected neural network to obtain a fused deep neural network; step-by-step training is carried out on the fusion type deep neural network through training data, and a target model with a noise reduction function is obtained; and inputting the real data with low signal-to-noise ratio into a target model so that the target model can perform noise reduction and signal detection on the real data to obtain a processing result of the noise reduction and the signal detection. The application can accurately reduce noise and detect the low signal-to-noise ratio data.
Description
Technical Field
The present application relates to the field of data and signal processing technologies, and in particular, to a method, an apparatus, a system, and a storage medium for data noise reduction and signal detection.
Background
In some application scenarios, such as deep space communication, gravitational wave detection, seismic exploration, the difficulty of low signal-to-noise ratio data processing is often faced. In the received signal, since the noise intensity is much higher than that of the signal corresponding to the effective information, an effective method is required to noise-reduce the data. In a general data signal processing method, matched filtering and nonlinear filters are often adopted, and when the method is adopted for processing low signal-to-noise ratio data, the problem of poor noise reduction effect exists, and accurate noise reduction of the data and detection of potential signals in the data are difficult. Therefore, how to process the low signal-to-noise ratio data becomes a technical problem to be solved.
Disclosure of Invention
The present application aims to solve at least one of the technical problems existing in the prior art. Therefore, the application provides a data noise reduction and signal detection method, a device, a system and a storage medium, which can accurately reduce noise and detect low signal-to-noise ratio data.
According to an embodiment of the first aspect of the present application, a data noise reduction and signal detection method includes:
generating analog data with low signal-to-noise ratio, wherein the analog data comprises a plurality of mixed signals, the mixed signals are obtained by mixing noise and pure signals, and the pure signals represent noise-free signals;
Preprocessing the mixed signal in the simulation data to obtain training data;
fusing the first convolutional neural network, the second convolutional neural network, the self-attention mechanism and the multi-layer fully-connected neural network to obtain a fused deep neural network;
step-by-step training is carried out on the fusion type deep neural network through the training data, and a target model with a noise reduction function is obtained;
and inputting the real data with low signal-to-noise ratio into the target model so that the target model can perform noise reduction and signal detection on the real data to obtain a processing result of the noise reduction and the signal detection.
The data noise reduction and signal detection method provided by the embodiment of the application has at least the following beneficial effects: firstly, generating analog data with low signal-to-noise ratio; secondly, preprocessing the mixed signal in the analog data to obtain training data; then, fusing the first convolutional neural network, the second convolutional neural network, the self-attention mechanism and the multi-layer fully-connected neural network to obtain a fused deep neural network; then, training the fusion type deep neural network step by step through training data to obtain a target model with a noise reduction function; and finally, inputting the real data with low signal-to-noise ratio into a target model so that the target model can perform noise reduction and signal detection on the real data to obtain a processing result of the noise reduction and the signal detection. According to the data noise reduction and signal detection method, on one hand, through full training based on analog data, noise reduction and detection accuracy can be improved when the target model carries out data processing on real data with low signal to noise ratio, frequent calculation is not needed, and noise reduction and detection speed is improved; on the other hand, the fusion type deep neural network is constructed, so that the ultra-long data can be processed, and the signal characteristics of the real data with low signal-to-noise ratio can be extracted from different dimensions, so that higher noise reduction and detection precision can be realized. Therefore, the data noise reduction and signal detection method can accurately reduce noise and detect low signal-to-noise ratio data.
According to some embodiments of the application, the generating analog data with low signal-to-noise ratio includes:
simulating and generating noise according to the sensitivity curve of the gravitational wave detector;
selecting corresponding parameter ranges for different types of gravitational wave sources;
generating a plurality of pure signals corresponding to a plurality of gravitational wave sources according to the parameter range, wherein the gravitational wave sources and the pure signals are in one-to-one correspondence;
and mixing each pure signal with the noise respectively to obtain analog data comprising a plurality of mixed signals.
According to some embodiments of the application, the preprocessing the mixed signal in the analog data includes:
performing whitening treatment on the mixed signal in the simulation data so as to uniformly distribute the noise spectrum of the mixed signal;
and carrying out normalization processing on the mixed signal after the whitening processing so that all data in the analog data have the same scale.
According to some embodiments of the present application, the fusing the first convolutional neural network, the second convolutional neural network, the self-attention mechanism, and the multi-layer fully-connected neural network to obtain a fused deep neural network includes:
Building the first convolutional neural network to perform extraction of feature information on the mixed signal;
building a fusion network of the self-attention mechanism and the second convolution neural network so as to perform noise reduction processing on the mixed signal;
and building the multi-layer fully-connected neural network to detect a target signal.
According to some embodiments of the present application, the step-by-step training of the fused deep neural network by the training data, to obtain a target model with a noise reduction function, includes:
taking the mean square error between the mixed signal and the pure signal as a loss function, and carrying out initial training on the fusion type deep neural network through the loss function;
setting an alternating cycle number, wherein the alternating cycle number comprises a first training cycle number and a second training cycle number, the first training cycle number and the second training cycle number comprise a plurality of training cycles, and each training cycle represents a process of training all samples in the training data once;
noise reduction training is carried out on the fusion type deep neural network in the first training period number, and detection training is carried out on the fusion type deep neural network in the second training period number;
Based on the alternating cycle number, noise reduction training and detection training are alternately performed;
and adjusting parameters of the fusion type deep neural network by using an optimizer, and adjusting the learning rate of the fusion type deep neural network by using a learning rate adjustment strategy.
According to some embodiments of the application, the performing noise reduction training on the fused deep neural network during the first training period and performing probing training on the fused deep neural network during the second training period includes:
noise reduction training is performed on the fused deep neural network in the first training period number so as to minimize the mean square error between the mixed signal and the pure signal;
and performing detection training on the fused deep neural network in the second training period number to minimize cross entropy loss of two categories, wherein the period number of the first training period number is equal to the period number of the second training period number.
According to some embodiments of the application, the data noise reduction and signal detection method further comprises:
and evaluating the processing result to determine the noise reduction precision and the detection precision of the target model.
According to a second aspect of the present application, a data noise reduction and signal detection apparatus includes:
the generation module is used for generating analog data with low signal-to-noise ratio, wherein the analog data comprises a plurality of mixed signals, the mixed signals are obtained by mixing noise and pure signals, and the pure signals represent noise-free signals;
the processing module is used for preprocessing the mixed signals in the analog data to obtain training data;
the fusion module is used for fusing the first convolutional neural network, the second convolutional neural network, the self-attention mechanism and the multi-layer fully-connected neural network to obtain a fusion type deep neural network;
the training module is used for step-by-step training of the fusion type deep neural network through the training data to obtain a target model with a noise reduction function;
the noise reduction and detection module is used for inputting real data with low signal to noise ratio into the target model so that the target model can carry out noise reduction and signal detection on the real data to obtain a processing result of the noise reduction and the signal detection.
According to an embodiment of the third aspect of the present application, a data noise reduction and signal detection system includes:
At least one memory;
at least one processor;
at least one program;
the program is stored in the memory, and the processor executes at least one of the programs to implement the data noise reduction and signal detection method as described in the embodiment of the first aspect.
According to a fourth aspect embodiment of the present application, a computer-readable storage medium stores computer-executable instructions for causing a computer to perform the data denoising and signal detection method according to the first aspect embodiment.
Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.
Drawings
The application is further described with reference to the accompanying drawings and examples, in which:
fig. 1 is a flow chart of a data noise reduction and signal detection method according to an embodiment of the present application;
FIG. 2 is a waveform diagram of low SNR data provided by an embodiment of the present application;
FIG. 3 is a schematic diagram of a fused deep neural network according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a data noise reduction and signal detection device according to an embodiment of the present application;
Fig. 5 is a schematic structural diagram of a data noise reduction and signal detection system according to an embodiment of the present application.
Reference numerals:
the system comprises a generation module 100, a processing module 110, a fusion module 120, a training module 130, a noise reduction and detection module 140, a memory 200 and a processor 300.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the application.
It should be noted that although functional block diagrams are depicted as block diagrams, and logical sequences are shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the block diagrams in the system. The terms and the like in the description and in the claims, and in the above-described drawings, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
In the description of the present application, the meaning of a number is one or more, the meaning of a number is two or more, and greater than, less than, exceeding, etc. are understood to exclude the present number, and the meaning of a number is understood to include the present number. The description of the first and second is for the purpose of distinguishing between technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present application, unless explicitly defined otherwise, terms such as arrangement, installation, connection, etc. should be construed broadly and the specific meaning of the terms in the present application can be reasonably determined by a person skilled in the art in combination with the specific contents of the technical scheme.
In the description of the present application, the descriptions of the terms "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Noun interpretation:
low signal-to-noise ratio: meaning that the signal strength is below 0dB.
Data with low signal-to-noise ratio: typically, data with signal strength below 0dB is referred to, and the signal is completely drowned out by noise;
noise reduction: noise in the data is suppressed.
Signal detection: it is determined whether a signal of interest, such as a gravitational wave signal, a communication signal, a seismic wave signal, or the like, exists in the data.
Deep space communication: means the communication between the communication entity on the earth and the aircraft entering the solar system from the earth satellite orbit, the distance can reach millions of kilometers, tens of thousands of kilometers, to hundreds of millions of kilometers or more;
gravitational wave: gravitational waves are ripples generated by the accelerating mass in space-time.
Gravitational wave detector (Gravitational-wave observator): is a device for detecting gravitational waves in gravitational wave astronomy. By detecting the gravitational wave, the generalized relativity can be experimentally verified. Commonly used detectors include rod detectors and laser interferometers, which operate primarily by measuring the effect of gravitational waves on the distance between two remote locations as they pass. Since 1960 s, multiple gravitational wave detectors were built and activated sequentially and there was a continual improvement in detector sensitivity. Nowadays, these detectors have the function of detecting gravitational wave sources inside and outside the lunar system, and are the main detection tools of gravitational wave astronomy.
Seismic prospecting: the geophysical prospecting method is to infer the property and the morphology of the underground rock stratum by utilizing the elasticity and the density difference of the underground medium and observing and analyzing the propagation rule of the earthquake waves generated by artificial earthquakes in the underground. Seismic exploration is the most important method in geophysical exploration and is the most effective method for solving the problem of oil and gas exploration. It is an important means for surveying petroleum and natural gas resources before drilling, and is widely applied in the aspects of coal field and engineering geological investigation, regional geological research, crust research and the like.
In scientific scenes, such as the fields of deep space communication, gravitational wave detection, seismic exploration and the like, the difficult problem of low signal-to-noise ratio data processing is often faced. Because the signal is usually submerged by noise, the traditional signal processing method including matched filtering and nonlinear filtering has the problems of poor noise reduction effect, low calculation efficiency and the like when the data with low signal to noise ratio is processed, and the data cannot be accurately noise reduced and potential signals in the data cannot be detected.
In particular, the processing of data with low signal-to-noise ratio has been a research difficulty in the scientific field, especially in the physical and communication fields, and because the noise intensity is far higher than that of the signals, an efficient method is needed to reduce the noise of the data, so as to detect whether useful or interesting signals exist in the data. The current method has the following defects:
the nonlinear filter and other methods have low calculation speed, low noise reduction and detection precision and can not obtain accurate results;
the existing neural network algorithm cannot process ultra-long data, and noise reduction, detection speed and detection accuracy are improved.
Based on the above, the application provides a data noise reduction and signal detection method, which can effectively improve noise reduction and detection precision and can realize data noise reduction and signal detection at the same time.
Next, a data noise reduction and signal detection method according to an embodiment of the present application will be described with reference to fig. 1 to 3.
It can be appreciated that as shown in fig. 1, there is provided a data noise reduction and signal detection method, including:
step S100, generating analog data with low signal-to-noise ratio, wherein the analog data comprises a plurality of mixed signals, the mixed signals are obtained by mixing noise and pure signals, and the pure signals represent noise-free signals;
step S110, preprocessing the mixed signal in the analog data to obtain training data;
step S120, fusing the first convolutional neural network, the second convolutional neural network, the self-attention mechanism and the multi-layer fully-connected neural network to obtain a fused deep neural network;
step S130, training the fusion type deep neural network step by step through training data to obtain a target model with a noise reduction function;
step S140, inputting the real data with low signal-to-noise ratio into the target model, so that the target model can perform noise reduction and signal detection on the real data to obtain the processing results of the noise reduction and the signal detection.
Firstly, generating analog data with low signal-to-noise ratio, wherein the analog data comprises a plurality of mixed signals, the mixed signals are obtained by mixing noise and pure signals, and the pure signals represent noise-free signals; secondly, preprocessing the mixed signal in the analog data to obtain training data; then, fusing the first convolutional neural network, the second convolutional neural network, the self-attention mechanism and the multi-layer fully-connected neural network to obtain a fused deep neural network; then, training the fusion type deep neural network step by step through training data to obtain a target model with a noise reduction function; and finally, inputting the real data with low signal-to-noise ratio into a target model so that the target model can perform noise reduction and signal detection on the real data to obtain a processing result of the noise reduction and the signal detection. According to the data noise reduction and signal detection method, on one hand, through full training based on analog data, noise reduction and detection accuracy can be improved when the target model carries out data processing on real data with low signal to noise ratio, frequent calculation is not needed, and noise reduction and detection speed is improved; on the other hand, the fusion type deep neural network is constructed, so that the ultra-long data can be processed, and the signal characteristics of the real data with low signal-to-noise ratio can be extracted from different dimensions, so that higher noise reduction and detection precision can be realized. Therefore, the data noise reduction and signal detection method can accurately reduce noise and detect low signal-to-noise ratio data.
It will be appreciated that generating analog data of low signal to noise ratio includes:
simulating and generating noise according to the sensitivity curve of the gravitational wave detector;
selecting corresponding parameter ranges for different types of gravitational wave sources;
generating a plurality of pure signals corresponding to a plurality of gravitation wave sources according to the parameter range, wherein the gravitation wave sources and the pure signals are in one-to-one correspondence;
each clean signal is mixed with noise respectively to obtain analog data comprising a plurality of mixed signals.
It should be noted that, in the gravitational wave detection scenario, analog data with low signal-to-noise ratio may be generated by the following steps:
step S101: simulating and generating noise according to the sensitivity curve of the gravitational wave detector;
step S102: selecting proper parameter ranges for different types of gravitational wave sources to generate corresponding signals; the gravitational wave source can be an extreme mass ratio precession system, an oversized mass double black hole, a double white dwarf star or a random gravitational wave background;
step S103: mixing the data generated in the step S101 and the step S102 to obtain data with different signal to noise ratios; in this step, it is important to set a specific Signal-Noise Ratio (SNR) according to the following formula.
Where s|s is the inner product, which may be a|b, calculated by the following formula:
wherein the symbols represent complex conjugates, S n Is a sensitivity curve.
In the deep space communication field, noise can be simulated and generated based on a signal curve of communication between a communication entity and an earth satellite; in the field of seismic exploration, noise can be simulated and generated based on a curve corresponding to seismic waves.
It is emphasized that the present application uses a matched filter snr and that the snr of the training data is deliberately set to 50, which is approximately equal to-40 dB, when simulating an actual very low snr environment. Through the steps, a large amount of low signal-to-noise ratio analog data are obtained. An example of the analog data is shown in fig. 2, where in fig. 2, orange is a pure signal and blue is signal+noise.
It will be appreciated that preprocessing the mixed signal in the analog data includes:
performing whitening treatment on the mixed signal in the analog data to uniformly distribute the noise spectrum of the mixed signal;
and carrying out normalization processing on the mixed signal after the whitening processing so that all data in the analog data have the same scale.
It should be noted that the data preprocessing portion mainly comprises two steps: the whitening and normalization specifically comprises the following steps:
Step S111: whitening of data, the purpose of whitening is to remove redundancy of the input data and improve its statistical properties; the specific calculation mode is as follows, the noise spectrum of the original data is converted into a uniform distribution mode, so that all frequency components are uniformly distributed on the power spectrum, the complexity of noise can be effectively reduced, and more uniform data can be provided for the subsequent fusion type deep neural network;
step S112: normalizing the whitened data; normalization can ensure that the data have the same scale in the whole data set, so that the training of the fusion type deep neural network is more stable; in this step, the analog data is adjusted to [ -1,1], and the phenomenon of numerical overflow, inaccurate calculation or instability in the learning process caused by excessive or insufficient numerical values is avoided.
It can be appreciated that fusing the first convolutional neural network, the second convolutional neural network, the self-attention mechanism and the multi-layer fully-connected neural network to obtain a fused deep neural network, including:
constructing a first convolutional neural network to perform extraction of characteristic information on the mixed signal;
constructing a fusion network of a self-attention mechanism and a second convolution neural network so as to perform noise reduction treatment on the mixed signal;
And constructing a multi-layer fully-connected neural network to detect the target signal.
It should be noted that in the model building, please provide a fusion type deep neural network algorithm, the network is mainly composed of a Convolutional Neural Network (CNN), a self-attention mechanism and a multi-layer fully connected neural network (MLP); the fusion type deep neural network model constructed by the application is designed end to end, and can directly reduce noise and detect input low signal-to-noise ratio data at the same time; the design avoids manual feature selection and threshold setting, and realizes automatic processing and analysis of data.
An example of the overall structure of a model is given in fig. 3.
The fusion type deep neural network model building specific steps are as follows:
step S121: building a first convolutional neural network; performing a task of feature extraction by using a convolutional neural network, wherein the data obtained in the step S100 firstly passes through the CNN network; CNNs have excellent performance in processing data such as images, voices, time series, and the like; in the model of the present application, CNN functions to extract useful feature information from input data of low signal-to-noise ratio; more importantly, through the stacking of the convolution layers, the model has a larger receptive field, and can process ultra-long sequence data, so that the noise reduction and the signal detection precision are enhanced;
Step S122: building a self-attention mechanism and a second convolution neural network: the application designs an encoder module which integrates a self-attention mechanism and a convolutional neural network and is specially responsible for noise reduction processing. The principle of self-attention mechanism is shown in the following formula, wherein Q, K and V are trainable parameters, the mechanism is helpful for capturing long-distance dependence in a sequence by a neural network, and CNN is focused on extracting local characteristics; this unique combination enables the fused deep neural network to take global and local information into account simultaneously when processing noise reduction tasks.
Step S123: constructing a multi-layer fully-connected neural network; the multi-layer fully-connected neural network is used as a classifier in the fused deep neural network model and is used for detecting a target signal; the MLP has the capability of learning a nonlinear mapping relation, and can accurately detect potential target signals from noise reduction data. The classifier formula is as follows:
wherein ω is ij For training parameters of classifier, x j For the output of step S123, logistic is a logistic regression function.
It can be understood that the step-by-step training is performed on the fusion type deep neural network through training data to obtain a target model with a noise reduction function, which comprises the following steps:
Taking the mean square error between the mixed signal and the pure signal as a loss function, and carrying out initial training on the fusion type deep neural network through the loss function;
setting alternating cycle numbers, wherein the alternating cycle numbers comprise a first training cycle number and a second training cycle number, the first training cycle number and the second training cycle number comprise a plurality of training cycles, and each training cycle represents the process of training all samples in training data once;
noise reduction training is carried out on the fusion type deep neural network in the first training period number, and detection training is carried out on the fusion type deep neural network in the second training period number;
based on the alternating cycle number, noise reduction training and detection training are alternately performed;
and adjusting parameters of the fusion type deep neural network by using an optimizer, and adjusting the learning rate of the fusion type deep neural network by using a learning rate adjustment strategy.
It may be appreciated that performing noise reduction training on the fused deep neural network during a first training period and performing probing training on the fused deep neural network during a second training period includes:
noise reduction training is carried out on the fusion type deep neural network in a first training period number so as to minimize the mean square error between the mixed signal and the pure signal;
The fused deep neural network is probed for a second number of training cycles to minimize cross entropy loss of the two classifications, wherein the number of cycles of the first number of training cycles is equal to the number of cycles of the second number of training cycles.
It should be noted that, in the model training phase, a unique step training strategy is introduced. This strategy involves alternating training of different loss functions to achieve simultaneous noise reduction and detection of the signal; the method comprises the following specific steps:
step S131: an initial stage; firstly, using a mean square error between a mixed signal containing noise and a pure signal without noise as a loss function in an initial stage, wherein the main aim is to make a network learn to reduce noise; the training in this stage is helpful for the model to primarily understand and extract valuable signal characteristics, and can also prevent the classifier from capturing excessive noise characteristics;
step S132: alternating training; after the preliminary training is completed, a phased alternating training strategy is adopted, wherein the training strategy is carried out with each 6 epochs (one epoch represents the process of training once by using all samples in a training set) as a period (two steps of ab are described below); in the training strategy, the model can be alternately performed between noise reduction training and detection training, so that two tasks can be mutually supplemented, and the overall performance of the model is improved. The method comprises the following specific steps:
a) Noise reduction training: training is mainly carried out on noise reduction tasks in the first three epochs of each alternate period; the training goal of the model at this stage is to minimize the mean square error between the mixed signal and the clean signal, so that the model can learn better how to reduce noise on the signal;
b) Detection training: converting the last three epochs of each alternate period into training of detecting tasks for the model; the training target of the model at this stage is to minimize the cross entropy loss of two classifications, and improve the detection precision of the model to the signals;
step S133: the optimizer and the learning rate are adjusted; in the model training process, an Adam optimizer is used for adjusting network parameters, the initial learning rate is 0.01, and the total training round is 100epoch; meanwhile, a learning rate adjustment strategy is introduced, and when the loss of training is not in front, the learning rate is properly reduced so as to ensure that the model can continue to learn effectively; through the steps, the whole model training process not only improves the learning efficiency of the network, but also accelerates the convergence rate of the network, so that the model can reach ideal performance more quickly.
It can be appreciated that the data noise reduction and signal detection method further includes:
and evaluating the processing result to determine the noise reduction precision and the detection precision of the target model.
In the deep scenario, model reasoning is a step of judging whether the model meets the requirements; in the model reasoning stage, the noise reduction and detection operation is a core step, and the specific process is as follows:
step S141: noise reduction and detection: inputting the low signal-to-noise ratio data to be processed into the model trained in the step S130, wherein the model can automatically execute the tasks of noise reduction and signal detection; firstly, a noise reduction network part performs feature extraction and noise reduction processing through a convolutional neural network and a self-attention mechanism; then, signal detection is performed through a fully-connected network trained in advance.
Step S142: evaluation of results: evaluating the noise-reduced data and the signal detection result output in the step S141, wherein the evaluation mode mainly depends on a specific application scene;
for example, in gravitational wave applications, noise reduction accuracy is evaluated by calculating the degree of matching (overlay) between the output noise reduction signal and the real signal, and detection accuracy is evaluated by ROC curve;
wherein:
in the two formulas, h is network output, s is a real signal, the value of overlap is between 0 and 1, and the bigger the numerical value is, the higher the matching degree is, and the higher the noise reduction precision is.
The data noise reduction and signal detection method of the present application will be further described with reference to the above embodiments.
It should be noted that, the conventional signal denoising methods include wavelet denoising, mode maximum denoising, and the like, which have low generalization ability and require a certain priori knowledge. For example, determination of a wavelet transform threshold, selection of a wavelet. Based on the learning of the data characterization, the deep learning can better extract the key feature quantity hidden in the data, and automatically filter the influence of interference noise.
In deep learning techniques, typical neural networks include convolutional neural networks, recurrent neural networks, fully-connected neural networks, and the like. The cyclic neural network is a neural network with nodes directionally connected into a cycle, can better display dynamic time sequence behaviors, can process an input sequence with any time sequence by utilizing an internal memory unit, and can process some time sequence problems, such as tasks of natural language processing, voice recognition and the like.
The long-time and short-time memory network is a special cyclic neural network. On the basis of a cyclic neural network, an input threshold, a forgetting threshold and an output threshold are added, so that the weight is changed in the self-circulation process, and the integral scale on different time nodes can be dynamically changed, thereby skillfully avoiding the problem of gradient disappearance or gradient expansion in the circulation process.
Specifically, wavelet transformation is a frequency analysis method of radio signals. Compared with the traditional Fourier transform analysis, the wavelet transform has the advantages of time domain localization, frequency domain localization and the like, can realize the decomposition of signals under different scales, and retains the characteristics of the signals under different modulation types. The signal has a certain continuity in space or time, the effective signal of which has a larger wavelet coefficient in the wavelet domain, and the noise signal generally has a discrete state in space or time, and the wavelet coefficient in the wavelet domain of which is smaller. By utilizing this property, the noise reduction processing of the signal can be realized by a wavelet transform method. And carrying out wavelet decomposition processing on the input original signal, calculating to obtain different wavelet coefficients, and assuming that the noise signal is subjected to Gaussian distribution, most of the noise coefficients are positioned in a certain interval, and setting the coefficients in the interval to zero so as to realize the maximum suppression of the noise signal. And (3) utilizing the wavelet coefficient after the threshold processing to realize the reconstruction of the radio signal and obtain the signal after noise reduction.
In particular, convolutional neural network-based self-encoders are generally composed of a three-layer network including an input layer, a hidden layer, and an output layer, where the number of neurons of the input layer and the output layer are equal. During training, the self-encoder generates an output sample of the same size for each input sample, and the self-encoder training is optimized to make the output sample as close as possible to the input sample. On the basis of a self-encoder, the application provides a noise-reducing self-encoder (DAE), and noise signals are added into input data, so that the self-encoder obtained by training has a noise-reducing function, is more robust, and improves the generalization capability of a model.
The existing radio signal identification methods based on deep learning have good identification effect on the identification of a large amount of radio signal data, but in a low signal-to-noise ratio area, the identification accuracy of the identification methods is still low. In view of the problem, a data noise reduction and signal detection method based on deep learning is provided.
The data noise reduction and signal detection method is implemented by setting a low signal-to-noise ratio classifier, a noise reduction self-encoder and an identification network, wherein the low signal-to-noise ratio classifier is essentially a classifier, and the high signal-to-noise ratio signal and the low signal-to-noise ratio signal in the signal-to-noise ratio can be identified by setting different signal-to-noise ratio thresholds. The noise reduction self-encoder can realize noise reduction processing of a low signal-to-noise ratio radio signal. The modulation type recognition network is constituted by a long short-term memory network (LSTM network) capable of recognizing the modulation type of the inputted radio signal.
Specifically, inspired by a long-short-time memory neural network, the application designs a radio signal modulation type identification model based on the long-short-time memory network, which comprises three layers of LSTM (128), LSTM (32) and FC (11), wherein the input original signal size is lenx 2, len represents the number of sampling nodes, and 2 represents the time dimension of a certain sampling node. LSTM (128) means mapping the time dimension of the input data into a feature space of size len x 128. FC (11) represents a fully connected network, mapping the input data to 11 distribution areas, where 11 is determined by the training data set modulation type category. The LSTM layer uses tanh as the activation function, dropout is 0.8. The fully connected layer uses softmax as the activation function, training uses cross entropy as the loss function, adam with a learning rate of 0.001 is selected as the optimizer, batch size is 64, epoch is 20. And selecting the top_one as an evaluation index of the model, namely, only when the class label corresponding to the highest confidence coefficient is a correct class label, identifying the model correctly.
And extracting characteristic information of the data from the radio signal data by using a long-short-time memory network, and classifying different types of signals according to the characteristic information. And judging the noise reduction effect of the model according to the comparison of the type recognition accuracy of the radio signal recognition model based on the long-short-term memory network before and after the noise reduction of the radio signal.
Specifically, for the low signal-to-noise ratio classifier, the recognition model has good classification effect in the high signal-to-noise ratio region, but the recognition accuracy in the low signal-to-noise ratio region is extremely low. Therefore, noise reduction processing is required for the radio signal in the low signal-to-noise ratio section to improve the accuracy of the recognition section of the recognition model. Noise reduction processing is performed on the low signal-to-noise ratio signal, and the low signal-to-noise ratio signal needs to be extracted from the signal data first. In order to screen out the low signal-to-noise ratio signal more accurately, the application designs a radio signal classifier based on LSTM, which is used for noise reduction treatment of the low signal-to-noise ratio signal.
For example, the classifier may be such that the input signal size is lenx 2, where len represents the number of sampling nodes and 2 represents the time dimension of a certain sampling node. LSTM (32) represents mapping the time dimension of the input data into a feature space of size len x 32. The LSTM layer uses tanh as the activation function, dropout is 0.8. The FC layer is a fully connected layer, using softmax as the activation function. Training uses cross entropy as a loss function, and Adam with a learning rate of 0.008 is selected as an optimizer, with a batch size of 64 and epoch of 20.
The low signal-to-noise ratio classification network can be utilized to map the radio signal to two intervals of high signal-to-noise ratio and low signal-to-noise ratio, so that the LSTM-based radio signal classification task is realized.
The application provides a radio signal noise reduction reconstruction model based on a self-coding technology, white noise which is subjected to Gaussian distribution is added into a high signal-to-noise ratio signal, and the signal containing the additive noise and an original high signal-to-noise ratio signal are input into the noise reduction model for training. And using the trained noise reduction model to realize noise reduction reconstruction processing of the low signal-to-noise ratio signal.
In general, the application provides a data noise reduction and signal detection method of a self-coding technology in combination with the self-coder technology for the problem of low recognition accuracy of the existing model in a low signal-to-noise ratio region, can improve the recognition accuracy of a modulation type recognition model based on deep learning in the low signal-to-noise ratio region to a certain extent, and can more fully utilize limited signal resources.
According to the application, firstly, a radio signal type recognition model is built through LSTM to perform type recognition on radio signals with different signal to noise ratios, a radio signal classification task based on LSTM is realized through a low signal to noise ratio classifier, then, a radio signal noise reduction reconstruction model is built based on a self-coding technology to realize noise reduction reconstruction work on the low signal to noise ratio radio signals, and finally, the type recognition model is retrained by using data after noise reduction reconstruction and is tested.
It can be understood that as shown in fig. 4, the present application further provides a data noise reduction and signal detection device, including:
a generating module 100, configured to generate analog data with a low signal-to-noise ratio, where the analog data includes a plurality of mixed signals, the mixed signals are obtained by mixing noise and pure signals, and the pure signals represent noise-free signals;
the processing module 110 is configured to pre-process the mixed signal in the analog data to obtain training data;
the fusion module 120 is configured to fuse the first convolutional neural network, the second convolutional neural network, the self-attention mechanism, and the multi-layer fully-connected neural network to obtain a fused deep neural network;
the training module 130 is configured to perform step-by-step training on the fused deep neural network through training data to obtain a target model with a noise reduction function;
the noise reduction and detection module 140 is configured to input real data with a low signal-to-noise ratio into the target model, so that the target model performs noise reduction and signal detection on the real data, and a processing result of the noise reduction and the signal detection is obtained.
The generating module 100 includes:
the simulation module is used for simulating and generating noise according to the sensitivity curve of the gravitational wave detector;
The selection module is used for selecting corresponding parameter ranges for different types of gravitation wave sources;
the signal module is used for generating a plurality of pure signals corresponding to the gravitational wave sources according to the parameter range, wherein the gravitational wave sources and the pure signals are in one-to-one correspondence;
and the mixing module is used for respectively mixing each pure signal with noise to obtain analog data comprising a plurality of mixed signals.
It should be noted that the processing module 110 includes:
the whitening processing module is used for whitening the mixed signal in the analog data so as to uniformly distribute the noise spectrum of the mixed signal;
and the normalization processing module is used for carrying out normalization processing on the mixed signal after the whitening processing so as to enable all data in the analog data to have the same scale.
Note that, the fusion module 120 includes:
the first construction module is used for constructing a first convolutional neural network so as to extract characteristic information of the mixed signal;
the second construction module is used for constructing a fusion network of the self-attention mechanism and the second convolution neural network so as to perform noise reduction treatment on the mixed signal;
and the third building module is used for building a multi-layer fully-connected neural network so as to detect a target signal.
Note that, the training module 130 includes:
the initial training module is used for taking the mean square error between the mixed signal and the pure signal as a loss function, and carrying out initial training on the fusion type deep neural network through the loss function;
the setting module is used for setting alternating cycle numbers, wherein the alternating cycle numbers comprise a first training cycle number and a second training cycle number, the first training cycle number and the second training cycle number comprise a plurality of training cycles, and each training cycle represents the process of training all samples in training data once;
the noise reduction and detection training module is used for carrying out noise reduction training on the fusion type deep neural network in a first training period number and carrying out detection training on the fusion type deep neural network in a second training period number;
the alternating training module is used for alternately carrying out noise reduction training and detection training based on the alternating cycle number;
and the adjusting module is used for adjusting parameters of the fusion type deep neural network by using the optimizer and adjusting the learning rate of the fusion type deep neural network by using the learning rate adjusting strategy.
It should be noted that the noise reduction and detection training module includes:
the first minimizing module is used for conducting noise reduction training on the fusion type deep neural network in a first training period number so as to minimize the mean square error between the mixed signal and the pure signal;
And the second minimizing module is used for detecting and training the fusion type deep neural network in a second training period number so as to minimize cross entropy loss of two categories, wherein the period number of the first training period number is equal to that of the second training period number.
It should be noted that the data noise reduction and signal detection device further includes:
and the evaluation module is used for evaluating the processing result to determine the noise reduction precision and the detection precision of the target model.
A data noise reduction and signal detection system according to an embodiment of the present application is described below with reference to fig. 5.
It will be appreciated that as shown in fig. 5, the data noise reduction and signal detection system includes:
at least one memory 200;
at least one processor 300;
at least one program;
the programs are stored in the memory 200, and the processor 300 executes at least one program to implement the data noise reduction and signal detection methods described above. Fig. 5 illustrates a processor 300.
The processor 300 and the memory 200 may be connected by a bus or other means, fig. 5 being an example of a connection via a bus.
The memory 200 is used as a non-transitory computer readable storage medium for storing non-transitory software programs, non-transitory computer executable programs, and signals, such as program instructions/signals corresponding to the data noise reduction and signal detection system in the embodiments of the present application. The processor 300 performs various functional applications and data processing by running non-transitory software programs, instructions, and signals stored in the memory 200, i.e., implements the data noise reduction and signal detection methods of the above-described method embodiments.
Memory 200 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store related data of the above-mentioned data noise reduction and signal detection methods, etc. In addition, memory 200 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 200 may optionally include memory located remotely from processor 300, which may be connected to the data noise reduction and signal detection system via a network. Examples of such networks include, but are not limited to, the internet of things, software defined networks, sensor networks, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more signals are stored in memory 200 that, when executed by the one or more processors 300, perform the data denoising and signal detection method of any of the method embodiments described above. For example, the method of fig. 1 described above is performed.
A computer-readable storage medium according to an embodiment of the present application is described below with reference to fig. 5.
As shown in fig. 5, the computer-readable storage medium stores computer-executable instructions that are executed by one or more processors 300, for example, by one of the processors 300 in fig. 5, to cause the one or more processors 300 to perform the data noise reduction and signal detection method in the above-described method embodiments. For example, the method of fig. 1 described above is performed.
The system embodiments described above are merely illustrative, in which elements illustrated as separate elements may or may not be physically separate, and elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
From the description of the embodiments above, those skilled in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media and communication media. The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable signals, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and may include any information delivery media.
The embodiments of the present application have been described in detail with reference to the accompanying drawings, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge of one of ordinary skill in the art without departing from the spirit of the present application. Furthermore, embodiments of the application and features of the embodiments may be combined with each other without conflict.
Claims (10)
1. The data noise reduction and signal detection method is characterized by comprising the following steps:
generating analog data with low signal-to-noise ratio, wherein the analog data comprises a plurality of mixed signals, the mixed signals are obtained by mixing noise and pure signals, and the pure signals represent noise-free signals;
preprocessing the mixed signal in the simulation data to obtain training data;
fusing the first convolutional neural network, the second convolutional neural network, the self-attention mechanism and the multi-layer fully-connected neural network to obtain a fused deep neural network;
step-by-step training is carried out on the fusion type deep neural network through the training data, and a target model with a noise reduction function is obtained;
and inputting the real data with low signal-to-noise ratio into the target model so that the target model can perform noise reduction and signal detection on the real data to obtain a processing result of the noise reduction and the signal detection.
2. The method of data noise reduction and signal detection according to claim 1, wherein said generating analog data of low signal-to-noise ratio comprises:
simulating and generating noise according to the sensitivity curve of the gravitational wave detector;
selecting corresponding parameter ranges for different types of gravitational wave sources;
generating a plurality of pure signals corresponding to a plurality of gravitational wave sources according to the parameter range, wherein the gravitational wave sources and the pure signals are in one-to-one correspondence;
and mixing each pure signal with the noise respectively to obtain analog data comprising a plurality of mixed signals.
3. The method of data noise reduction and signal detection according to claim 1, wherein the preprocessing the mixed signal in the analog data comprises:
performing whitening treatment on the mixed signal in the simulation data so as to uniformly distribute the noise spectrum of the mixed signal;
and carrying out normalization processing on the mixed signal after the whitening processing so that all data in the analog data have the same scale.
4. The method for data noise reduction and signal detection according to claim 1, wherein the fusing the first convolutional neural network, the second convolutional neural network, the self-attention mechanism and the multi-layer fully-connected neural network to obtain a fused deep neural network comprises:
Building the first convolutional neural network to perform extraction of feature information on the mixed signal;
building a fusion network of the self-attention mechanism and the second convolution neural network so as to perform noise reduction processing on the mixed signal;
and building the multi-layer fully-connected neural network to detect a target signal.
5. The method for data denoising and signal detection according to claim 1, wherein the step-by-step training of the fused deep neural network by the training data to obtain a target model with a denoising function comprises:
taking the mean square error between the mixed signal and the pure signal as a loss function, and carrying out initial training on the fusion type deep neural network through the loss function;
setting an alternating cycle number, wherein the alternating cycle number comprises a first training cycle number and a second training cycle number, the first training cycle number and the second training cycle number comprise a plurality of training cycles, and each training cycle represents a process of training all samples in the training data once;
noise reduction training is carried out on the fusion type deep neural network in the first training period number, and detection training is carried out on the fusion type deep neural network in the second training period number;
Based on the alternating cycle number, noise reduction training and detection training are alternately performed;
and adjusting parameters of the fusion type deep neural network by using an optimizer, and adjusting the learning rate of the fusion type deep neural network by using a learning rate adjustment strategy.
6. The method of data denoising and signal detection according to claim 5, wherein the performing denoising training on the fused deep neural network in the first training period number and performing detection training on the fused deep neural network in the second training period number comprises:
noise reduction training is performed on the fused deep neural network in the first training period number so as to minimize the mean square error between the mixed signal and the pure signal;
and performing detection training on the fused deep neural network in the second training period number to minimize cross entropy loss of two categories, wherein the period number of the first training period number is equal to the period number of the second training period number.
7. The data denoising and signal detection method of claim 1, further comprising:
And evaluating the processing result to determine the noise reduction precision and the detection precision of the target model.
8. The data noise reduction and signal detection device is characterized by comprising:
the generation module is used for generating analog data with low signal-to-noise ratio, wherein the analog data comprises a plurality of mixed signals, the mixed signals are obtained by mixing noise and pure signals, and the pure signals represent noise-free signals;
the processing module is used for preprocessing the mixed signals in the analog data to obtain training data;
the fusion module is used for fusing the first convolutional neural network, the second convolutional neural network, the self-attention mechanism and the multi-layer fully-connected neural network to obtain a fusion type deep neural network;
the training module is used for step-by-step training of the fusion type deep neural network through the training data to obtain a target model with a noise reduction function;
the noise reduction and detection module is used for inputting real data with low signal to noise ratio into the target model so that the target model can carry out noise reduction and signal detection on the real data to obtain a processing result of the noise reduction and the signal detection.
9. The data noise reduction and signal detection system is characterized by comprising:
At least one memory;
at least one processor;
at least one program;
the program is stored in the memory, and the processor executes at least one of the programs to implement the data noise reduction and signal detection method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing computer-executable instructions for causing a computer to perform the data noise reduction and signal detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310892526.4A CN117056680A (en) | 2023-07-20 | 2023-07-20 | Data noise reduction and signal detection method, device and system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310892526.4A CN117056680A (en) | 2023-07-20 | 2023-07-20 | Data noise reduction and signal detection method, device and system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117056680A true CN117056680A (en) | 2023-11-14 |
Family
ID=88663564
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310892526.4A Pending CN117056680A (en) | 2023-07-20 | 2023-07-20 | Data noise reduction and signal detection method, device and system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117056680A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118028980A (en) * | 2024-04-12 | 2024-05-14 | 浙江康鹏半导体有限公司 | Intelligent temperature monitoring method for gallium arsenide semiconductor growth |
| CN118035652A (en) * | 2024-04-10 | 2024-05-14 | 东北石油大学三亚海洋油气研究院 | Processing method and device for measurement while drilling data and electronic equipment |
| CN118091791A (en) * | 2024-04-29 | 2024-05-28 | 太原无线电一厂有限公司 | Meteorological sonde detection control system based on communication network self-adaption |
-
2023
- 2023-07-20 CN CN202310892526.4A patent/CN117056680A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118035652A (en) * | 2024-04-10 | 2024-05-14 | 东北石油大学三亚海洋油气研究院 | Processing method and device for measurement while drilling data and electronic equipment |
| CN118035652B (en) * | 2024-04-10 | 2024-07-02 | 东北石油大学三亚海洋油气研究院 | Processing method and device for measurement while drilling data and electronic equipment |
| CN118028980A (en) * | 2024-04-12 | 2024-05-14 | 浙江康鹏半导体有限公司 | Intelligent temperature monitoring method for gallium arsenide semiconductor growth |
| CN118091791A (en) * | 2024-04-29 | 2024-05-28 | 太原无线电一厂有限公司 | Meteorological sonde detection control system based on communication network self-adaption |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Mousavi et al. | Deep-learning seismology | |
| Stork et al. | Application of machine learning to microseismic event detection in distributed acoustic sensing data | |
| CN117056680A (en) | Data noise reduction and signal detection method, device and system and storage medium | |
| Van den Ende et al. | A self-supervised deep learning approach for blind denoising and waveform coherence enhancement in distributed acoustic sensing data | |
| Yuan et al. | A robust first-arrival picking workflow using convolutional and recurrent neural networks | |
| Duan et al. | Multitrace first-break picking using an integrated seismic and machine learning method | |
| Li et al. | IncepTCN: A new deep temporal convolutional network combined with dictionary learning for strong cultural noise elimination of controlled-source electromagnetic data | |
| Saad et al. | Real‐time earthquake detection and magnitude estimation using vision transformer | |
| Si et al. | Attenuation of random noise using denoising convolutional neural networks | |
| CN114595732B (en) | Radar radiation source sorting method based on depth clustering | |
| CN113687433A (en) | Bi-LSTM-based magnetotelluric signal denoising method and system | |
| Wang et al. | Data-driven S-wave velocity prediction method via a deep-learning-based deep convolutional gated recurrent unit fusion network | |
| Di et al. | Using relative geologic time to constrain convolutional neural network-based seismic interpretation and property estimation | |
| CN113962381B (en) | Weak signal data enhancement method | |
| Smith et al. | Robust deep learning-based seismic inversion workflow using temporal convolutional networks | |
| Song et al. | k NN-based gas-bearing prediction using local waveform similarity gas-indication attribute—An application to a tight sandstone reservoir | |
| Li et al. | Magnetotelluric data denoising method combining two deep-learning-based models | |
| Zhang et al. | Phase arrival picking for bridging multi-source downhole microseismic data using deep transfer learning | |
| Jiang et al. | 3D seismic geometry quality control and corrections by applying machine learning | |
| Alali et al. | Time-lapse data matching using a recurrent neural network approach | |
| Sun et al. | Prestack seismic inversion for elastic parameters using model-data-driven generative adversarial networks | |
| Li et al. | Pertinent multigate mixture-of-experts-based prestack three-parameter seismic inversion | |
| Gayathri et al. | Enhancing the sensitivity of transient gravitational wave searches with Gaussian mixture models | |
| Martinović et al. | Solar wind electron parameters determination on Wind spacecraft using quasi‐thermal noise spectroscopy | |
| CN112329535B (en) | CNN-based quick identification method for low-frequency oscillation modal characteristics of power system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |