CN110411566A - A kind of Intelligent light spectrum signal denoising method - Google Patents
A kind of Intelligent light spectrum signal denoising method Download PDFInfo
- Publication number
- CN110411566A CN110411566A CN201910707262.4A CN201910707262A CN110411566A CN 110411566 A CN110411566 A CN 110411566A CN 201910707262 A CN201910707262 A CN 201910707262A CN 110411566 A CN110411566 A CN 110411566A
- Authority
- CN
- China
- Prior art keywords
- network
- training
- data
- denoising
- contractility
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000001228 spectrum Methods 0.000 title claims abstract description 13
- 238000012549 training Methods 0.000 claims abstract description 26
- 230000003595 spectral effect Effects 0.000 claims abstract description 11
- 238000004611 spectroscopical analysis Methods 0.000 claims abstract description 7
- 238000010606 normalization Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 3
- 238000012545 processing Methods 0.000 abstract description 3
- 230000001737 promoting effect Effects 0.000 abstract description 2
- 230000009467 reduction Effects 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 241000512668 Eunectes Species 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000009659 non-destructive testing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J2003/283—Investigating the spectrum computer-interfaced
Abstract
Data processing field of the present invention discloses a kind of Intelligent light spectrum signal denoising method, for promoting the denoising effect of spectral signal.The present invention is reconstructed spectroscopic data using stack contractility autoencoder network, to realize the denoising of spectral signal；Stack contractility autoencoder network is in training, using the greedy method of layering, using the standard spectral data of not Noise as target desired value, first DAE network is entered data into, obtains accordingly exporting y1, then using y1 as the input of second level DAE network, it is trained step by step, the Optimal Parameters value that finally network that every grade of training is completed is obtained is finely adjusted training and obtains final model as the initial parameter value of cascade network.The present invention is suitable for the noise remove of spectral signal.
Description
Technical field
The present invention relates to data processing field, in particular to a kind of Intelligent light spectrum signal denoising method.
Background technique
In recent years, along with the continuous improvement of people 's material life level, also increasingly to requirements such as the precision of non-destructive testing
Increase, spectral detection as a kind of current nondestructiving detecting means of prevalence, though have the advantages that it is a variety of, also because by environment and itself
The data of the influence of the factors such as optical design, acquisition usually contain much noise, and denoising must be carried out before establishing model,
Denoising mostly uses the modes such as wavelet transformation now, is extremely difficult to filtering out completely for noise, the presence of noise seriously affects model and builds
Vertical accuracy, seriously glides so as to cause detection accuracy.
Summary of the invention
The technical problem to be solved by the present invention is a kind of Intelligent light spectrum signal denoising method is provided, for promoting spectrum
The denoising effect of signal.
To solve the above problems, the technical solution adopted by the present invention is that: a kind of Intelligent light spectrum signal denoising method uses
Spectroscopic data is reconstructed in stack contractility autoencoder network, to realize the denoising of spectral signal；Stack contractility is certainly
Coding network is in training,, will using the standard spectral data of not Noise as target desired value using the greedy method of layering
Data input first DAE network, obtain accordingly exporting y1, then using y1 as the input of second level DAE network, carry out step by step
Training, the Optimal Parameters value for finally obtaining the network that every grade of training is completed carry out micro- as the initial parameter value of cascade network
Training is adjusted to obtain final model.
Further, to ensure the validity in training process, when making training set, using the method pair of linear normalization
Spectroscopic data is normalized.
The beneficial effects of the present invention are: the present invention by inside deep learning field stack autoencoder network and denoising it is self-editing
Code network is combined to form stack autoencoder network, and in the creative denoising for spectroscopic data, obtains than tradition
The better noise filtering effect of the denoising methods such as wavelet transformation.
Detailed description of the invention
Fig. 1 is stack contractility autoencoder network structure of the invention.
Fig. 2 is specific implementation flow chart of the invention.
Fig. 3 is hands-on test chart of the invention.
Specific embodiment
The present invention provides a kind of Intelligent light spectrum signal denoising methods, based on Python, in Anaconda
On platform, model is built using TensorFlow as basic framework, realizes spy from the thought of coding model reconstruction using noise reduction
Sign automatically extracts, using unsupervised layer-by-layer greedy pre-training and the method for thering is supervision to finely tune to depth from coding neural network into
Row training, to realize the removal to spectral signal noise.
The structure chart of noise reduction autoencoder network (DAE) used in the present invention as shown in Figure 1, the work of noise reduction autoencoder network
With being that desired output to be made is approximately equal with noise free data, input data forms median feature vector by coding stage, in
Between feature vector reconstructing input vector by decoding stage.It replaces passing using contractility autoencoder network in the present invention
The autoencoder network of system, shrinkable autoencoder network is inherently modified the regular terms of common autoencoder network, general
Logical autoencoder network directly punishes the chain matrice level, and contractility autoencoder network is then to utilize hidden layer
The Jacobian matrix about input is exported to adjust to optimize.The network carries out optimizing, final mesh by gradient descent algorithm
Mark is that error to be reached is minimum, i.e. searching loss function:
Optimal value, wherein Jx(x) Jacobian matrix exported for hidden layer about input will be each in hands-on
A DAE is trained as a module, the Optimal Parameters after every level-one network training, as shape after final multiple DAE cascades
At stack noise reduction autoencoder network initial parameter value, then using the method for model fine tuning, the complete network of training, the present invention
Module of the contractility autoencoder network of use as stack autoencoder network, because common autoencoder network can only be specific
Regularization (punishment) is carried out to sample data on direction, and its regular terms of contractility autoencoder network is Jacobian matrix,
It contains the information of sample in all directions, can inhibit the disturbance in training sample all directions.
It should be noted that the spectrum sensor that used data when present invention training are portable spectrometer acquires
Initial data, use the data of large-scale conventional spectrometers acquisition as normal data, trained purpose is to remove spectrum sensor
The noise of the data of acquisition is substantially the fitting of data；It is still spectroscopic data after denoising output.Then pass through bluetooth or USB again
It reaches host computer and carries out modeling processing.
It is specific implementation flow chart of the present invention as shown in Figure 2, to ensure the validity in training process, is making training number
When according to collection, spectroscopic data is normalized using the method for linear normalization, when training, using the greedy side of layering
Method enters data into first DAE network using the standard spectral data of not Noise as target desired value, obtains corresponding defeated
Y1 out is trained step by step then using y1 as the input of second level DAE network, finally obtains the network that every grade of training is completed
The Optimal Parameters value arrived is finely adjusted training and obtains final model as the initial parameter value of cascade network.
It is the process schematic that is trained in anaconda development platform of the present invention as shown in Figure 3, it can be with from the figure
Find out by 120 training, accuracy rate can achieve 96.3% substantially, and the denoising for using common autoencoder network to constitute is self-editing
Code network accuracy is typically only capable to reach 95.2%；In terms of generalization ability, stack contractility of the present invention is self-editing
Code network is more better than common.All considerably beyond traditional wavelet transformation etc. on training speed and denoising precision
Denoising method.
Claims (2)
1. a kind of Intelligent light spectrum signal denoising method, which is characterized in that using stack contractility autoencoder network to spectrum
Data are reconstructed, to realize the denoising of spectral signal；Stack contractility autoencoder network is greedy using layering in training
Method enter data into first DAE network using the standard spectral data of not Noise as target desired value, obtain phase
Y1 should be exported, then using y1 as the input of second level DAE network, is trained step by step, the net for finally completing every grade of training
The Optimal Parameters value that network obtains is finely adjusted training and obtains final model as the initial parameter value of cascade network.
2. a kind of Intelligent light spectrum signal denoising method as described in claim 1, which is characterized in that when production training set, use
Spectroscopic data is normalized in the method for linear normalization.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910707262.4A CN110411566A (en) | 2019-08-01 | 2019-08-01 | A kind of Intelligent light spectrum signal denoising method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910707262.4A CN110411566A (en) | 2019-08-01 | 2019-08-01 | A kind of Intelligent light spectrum signal denoising method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110411566A true CN110411566A (en) | 2019-11-05 |
Family
ID=68365159
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910707262.4A Pending CN110411566A (en) | 2019-08-01 | 2019-08-01 | A kind of Intelligent light spectrum signal denoising method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110411566A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114417937A (en) * | 2022-01-26 | 2022-04-29 | 山东捷讯通信技术有限公司 | Deep learning-based Raman spectrum denoising method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120209601A1 (en) * | 2011-01-10 | 2012-08-16 | Aliphcom | Dynamic enhancement of audio (DAE) in headset systems |
| CN106096650A (en) * | 2016-06-12 | 2016-11-09 | 西安电子科技大学 | Based on the SAR image sorting technique shrinking own coding device |
| CN107122733A (en) * | 2017-04-25 | 2017-09-01 | 西安电子科技大学 | Hyperspectral image classification method based on NSCT and SAE |
| CN109598336A (en) * | 2018-12-05 | 2019-04-09 | 国网江西省电力有限公司信息通信分公司 | A kind of Data Reduction method encoding neural network certainly based on stack noise reduction |
-
2019
- 2019-08-01 CN CN201910707262.4A patent/CN110411566A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120209601A1 (en) * | 2011-01-10 | 2012-08-16 | Aliphcom | Dynamic enhancement of audio (DAE) in headset systems |
| CN106096650A (en) * | 2016-06-12 | 2016-11-09 | 西安电子科技大学 | Based on the SAR image sorting technique shrinking own coding device |
| CN107122733A (en) * | 2017-04-25 | 2017-09-01 | 西安电子科技大学 | Hyperspectral image classification method based on NSCT and SAE |
| CN109598336A (en) * | 2018-12-05 | 2019-04-09 | 国网江西省电力有限公司信息通信分公司 | A kind of Data Reduction method encoding neural network certainly based on stack noise reduction |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114417937A (en) * | 2022-01-26 | 2022-04-29 | 山东捷讯通信技术有限公司 | Deep learning-based Raman spectrum denoising method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115049936B (en) | High-resolution remote sensing image-oriented boundary enhanced semantic segmentation method | |
| CN106874956B (en) | The construction method of image classification convolutional neural networks structure | |
| CN109828251A (en) | Radar target identification method based on feature pyramid light weight convolutional neural networks | |
| CN106226212B (en) | EO-1 hyperion haze monitoring method based on depth residual error network | |
| CN110222773A (en) | Based on the asymmetric high spectrum image small sample classification method for decomposing convolutional network | |
| CN110349170B (en) | Full-connection CRF cascade FCN and K mean brain tumor segmentation algorithm | |
| CN106991355A (en) | The face identification method of the analytical type dictionary learning model kept based on topology | |
| CN109492596A (en) | A kind of pedestrian detection method and system based on K-means cluster and region recommendation network | |
| CN109063687A (en) | A kind of microseism P wave recognition methods and system based on depth convolutional neural networks | |
| CN111339935A (en) | Optical remote sensing picture classification method based on interpretable CNN image classification model | |
| CN111709443B (en) | Calligraphy character style classification method based on rotation invariant convolution neural network | |
| CN111896495A (en) | Method and system for discriminating Taiping Houkui production places based on deep learning and near infrared spectrum | |
| Li et al. | An object-oriented CNN model based on improved superpixel segmentation for high-resolution remote sensing image classification | |
| CN110411566A (en) | A kind of Intelligent light spectrum signal denoising method | |
| Zhao et al. | Hyperspectral target detection method based on nonlocal self-similarity and rank-1 tensor | |
| CN108388918A (en) | Data characteristics selection method with structure retention performance | |
| Bian et al. | CEEMD: A new method to identify mine water inrush based on the signal processing and laser-induced fluorescence | |
| CN116109826A (en) | Road crack detection method | |
| CN115828069A (en) | End-to-end magnetic anomaly signal noise reduction method based on deep learning | |
| CN114067169A (en) | Raman spectrum analysis method based on convolutional neural network | |
| CN113343861B (en) | Remote sensing image water body region extraction method based on neural network model | |
| Li et al. | Automatic identification of semi-tracks on apatite and mica using a deep learning method | |
| CN114782740A (en) | Remote sensing water quality monitoring method combining genetic optimization and extreme gradient promotion | |
| CN114464152A (en) | Music genre classification method and system based on visual transformation network | |
| CN112257791A (en) | Classification method of multi-attribute classification tasks based on CNN and PCA |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191105 |
|
| RJ01 | Rejection of invention patent application after publication |