CN114422049A - Frequency spectrum monitoring big data cleaning method and system based on deep learning detection - Google Patents
Frequency spectrum monitoring big data cleaning method and system based on deep learning detection Download PDFInfo
- Publication number
- CN114422049A CN114422049A CN202210085970.0A CN202210085970A CN114422049A CN 114422049 A CN114422049 A CN 114422049A CN 202210085970 A CN202210085970 A CN 202210085970A CN 114422049 A CN114422049 A CN 114422049A
- Authority
- CN
- China
- Prior art keywords
- signal
- signals
- time
- frequency
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B17/00—Monitoring; Testing
- H04B17/30—Monitoring; Testing of propagation channels
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R23/00—Arrangements for measuring frequencies; Arrangements for analysing frequency spectra
- G01R23/16—Spectrum analysis; Fourier analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Electromagnetism (AREA)
- Mathematical Physics (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention provides a frequency spectrum monitoring big data cleaning method and a system based on deep learning detection, comprising the following steps: the system comprises a data acquisition module, a signal detection module, a multi-signal separation module, a signal measurement module, a signal compression module and a signal label injection module. The method specifically comprises the following steps: receiving a broadband electromagnetic spectrum monitoring signal in real time; generating a time-frequency domain matrix; detecting all signals in the time-frequency domain matrix by using a signal detection algorithm; carrying out independence separation on a plurality of detection signals distributed in the time-frequency domain matrix; measuring core parameters of the separated signals; extracting and compressing the separated signals; the compressed signal is labeled. The invention has the advantages of easy development, effective saving of database storage amount, increase of unit database information amount and the like.
Description
Technical Field
The invention relates to the technical field of wireless communication, in particular to a frequency spectrum monitoring big data cleaning method and system based on deep learning detection.
Background
The purpose of electromagnetic spectrum monitoring is to support the spectrum management process as a whole, solve the interference problem, and provide scientific basis and technical support for frequency assignment, scientific management of spectrum, and radio planning. The monitoring signal analysis and the monitoring data processing are main work in daily radio monitoring, and the basic task of the monitoring signal analysis and the monitoring data processing is to determine information such as a modulation mode, signal parameters, occupancy degree and the like of a received signal under the conditions of a multi-signal environment and noise interference so as to provide a basis for further analyzing and processing the signal.
The non-commission civil radio monitoring stations of all levels and the electronic reconnaissance monitoring equipment of all military and military troops can acquire a large amount of electromagnetic spectrum environment sensing data of multiple types such as electromagnetic spectrum monitoring, electronic reconnaissance and the like, and the data comprises military and civil communication signals such as AM, FM broadcasting, broadcasting television, interphones, digital clusters, ground-air talkback, short wave ultra-short wave radio stations, data chains and the like and various radar signals. Therefore, the electromagnetic spectrum environment is becoming increasingly complex, the electromagnetic spectrum data monitored by the radio system is increasing, and the storage, retrieval and processing of the large data monitored by the broadband electromagnetic spectrum face severe technical challenges. In the face of massive monitoring data, how to effectively reduce the storage scale, improve the signal retrieval efficiency and improve the signal processing capability to realize the rapid positioning of the target is an important problem to be solved urgently in the field of electromagnetic spectrum monitoring control and electronic reconnaissance.
In order to solve the problems, the invention provides a novel frequency spectrum monitoring big data cleaning method and system based on deep learning detection.
Disclosure of Invention
In order to solve the problems, the invention provides a method and a system for cleaning large frequency spectrum monitoring data based on deep learning detection, which are used for solving the problem of database memory waste caused by overlarge monitoring signal data in the existing system, solving the problem of incomplete target signal parameter information in a common method and solving the problem of difficulty in independent processing of multi-signal time domain aliasing in broadband frequency spectrum sensing data.
In order to achieve the above purpose, the present invention provides the following technical solutions.
The method and the system for cleaning the frequency spectrum monitoring big data based on deep learning detection comprise the following steps:
carrying out Fourier transform on the radio frequency spectrum sensing data received in real time to obtain frequency spectrum data; splicing the frequency spectrum data according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data;
performing signal detection on all signals in the time-frequency domain matrix according to a signal detection algorithm based on deep learning to obtain time-frequency domain distribution information of each signal;
carrying out time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals;
performing core parameter measurement on the separated independent signals to obtain parameter information corresponding to the signals;
selecting a common multiple of signal bandwidth as a new sampling rate according to the bandwidth parameter of the signal under the condition of no signal distortion, and performing down-sampling rate compression processing on the separated independent signal to obtain a compressed independent signal;
generating an independence data file for each compressed independent signal and the corresponding parameter information thereof;
and performing homologous identification and classification on each signal according to the parameter information of the compressed independent signal, and realizing target radiation source positioning by combining a frequency difference joint estimation and positioning model.
Preferably, the method further comprises storing and labeling the compressed signal, comprising the steps of:
writing each compressed independent signal into a specified signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the name of the generated specified signal file.
Preferably, the radio frequency spectrum sensing data comprises electromagnetic spectrum monitoring data and electronic reconnaissance data, and the monitored objects comprise AM, FM broadcast, broadcast television, interphone, digital cluster, ground-to-air intercom, short-wave ultra-short wave radio station, data link and various radar signals.
Preferably, the real-time reception of the broadband electromagnetic spectrum monitoring signal comprises the following steps:
scanning at the central frequency point of the equipment through a receiver and acquiring a broadband electromagnetic spectrum monitoring signal acquired by a frequency band in the bandwidth of the equipment;
real and virtual IQ signals in the broadband electromagnetic spectrum monitoring signals are converted into complex signals through complex addition of IQ two paths of signals;
and performing Hilbert transform on the I/Q signal in the broadband electromagnetic spectrum monitoring signal to obtain another path of Q/I signal corresponding to the I/Q signal, and performing complex addition on the IQ two paths of signals to convert the IQ two paths of signals into complex signals.
Preferably, the generation of the time-frequency domain matrix includes the following steps:
taking the total number of sampling points as N, equally dividing the complex signals into M equal parts, carrying out zero filling operation on data less than N points until the number of the complex signals is N, and carrying out Fourier transform on each equal part of the complex signals to obtain the frequency spectrum data of the equal part:
in the formula, ceil (·) denotes a round-up operation, fsRepresenting the sampling rate of the receiver, and L representing the complexThe frequency spectrum resolution after the digital signal transformation is an integer randomly selected within the range of 100-500 Hz;
splicing the frequency spectrum data after Fourier transform according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data; the rows in the time-frequency domain matrix represent the sampling bandwidth of the data and the columns represent the sampling times of the data.
Preferably, the signal detection of all signals in the time-frequency domain matrix according to the signal detection algorithm based on deep learning includes the following steps:
defining thresholds for signal elements in the time-frequency domain matrix:
in the formula, AnoiseRepresenting the noise amplitude in the time-frequency domain matrix;
comparing each element in the time-frequency domain matrix row by row and column by column with a threshold, and forming a signal detection area by more than 10 continuous elements larger than the threshold;
and respectively taking the total number of the maximum rows and the maximum columns in each signal detection area as the length and the width of a rectangular frame corresponding to the maximum rows and the maximum columns, and respectively representing the continuous occurrence time of the signal, signal bandwidth information and frequency offset information by the central values of the length, the width and the width to obtain 4 elements of the upper left, the lower left, the upper right and the lower right of each rectangular frame as detection signals in a time-frequency domain matrix.
Preferably, the time-frequency domain separating the time-frequency domain distribution information of each signal includes:
the central value of the length, width and width of each rectangular frame is calculated respectively as shown in the following formula:
tm＝(ym2-ym1)×N/fs1
swm＝(xm2-xm1)×fs1/NFFT
frem＝(xm2+xm1)×fs1/(2×NFFT)
in the formula, tm、swm、fremRespectively represents the length, width and width of the m-th rectangular frame, xm1And ym1Coordinate value, x, representing the upper left element of the mth rectangular framem2And ym2Coordinate value, f, representing the lower right element of the mth rectangular frames1Is the sampling rate;
truncating each detection signal by length tmThen the intercepted signal is subjected to down-conversion operation to obtain a zero-frequency signal corresponding to the intercepted signal, and the passband value is used as swmThe adaptive filter carries out filtering processing on the zero-frequency signal;
shifting the frequency spectrum of the filtered signal to a frequency offset value fremObtaining separated independent signals;
the down-conversion operation is as follows:
y(n)＝x(n)*exp(-j*2*π*frem)
where y (n) represents a signal sequence after down-conversion, and x (n) represents a signal sequence before the down-conversion.
Preferably, the performing signal core parameter measurement on the separated independent signals includes:
and respectively carrying out modulation type identification, frequency offset accurate measurement, signal strength measurement and signal bandwidth measurement on each separated independent signal, and carrying out code rate measurement on the signal with the modulation type parameter of a digital type in the modulation type identification result.
Preferably, the signal distortion-free condition is that the nyquist sampling law, i.e. the compressed signal sampling rate f, is satisfieds2Greater than the bandwidth of the separated independent signal;
the separated independent signals are compressed at a reduced sampling rate, which is shown as the following formula:
in the formula, y (l)2) Denotes the decimated compressed signal sequence, x (-) denotes the separated independent signal sequence, floor (-) denotesAnd (5) carrying out downward rounding operation.
Frequency spectrum monitoring big data cleaning system based on deep learning detection includes:
the data acquisition module scans the central frequency point of the equipment through the receiver and acquires broadband electromagnetic spectrum monitoring signals acquired by the frequency band in the bandwidth;
a signal detection module comprising:
the preprocessing module is used for carrying out Fourier transform on the broadband electromagnetic spectrum monitoring signals to obtain frequency spectrum data, and splicing the frequency spectrum data according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data;
the deep learning detection module is used for carrying out signal detection on all signals in the time-frequency domain matrix according to a signal detection algorithm based on deep learning to obtain time-frequency domain distribution information of each signal;
the multi-signal separation module is used for carrying out time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals;
the signal measurement module is used for measuring the core parameters of the separated independent signals to obtain the bandwidth parameters of the signals;
the signal compression module is used for carrying out down-sampling rate compression processing on the separated independent signals to obtain compressed signals;
and the signal label injection module is used for writing each compressed signal into an appointed signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the name of the generated appointed signal file.
The invention has the beneficial effects that:
firstly, aiming at a large-scale spectrum monitoring database formed by each level of non-commission civil spectrum monitoring stations and military electronic reconnaissance equipment, a compression processing module in the system has a multi-bandwidth self-adaptive signal extraction and compression function, so that the data storage scale can be effectively reduced, and the defects of overlarge storage pressure, overhigh storage system construction cost and the like in the traditional spectrum database construction are overcome.
Secondly, aiming at the problem that the time domain aliasing of a plurality of signals is difficult to process independently in the broadband frequency spectrum monitoring data, the multi-signal independence separation technology based on the time-frequency domain matrix detection in the method can effectively realize the separation and extraction processing of all detected signals in the time domain aliasing data, form an independence data file of each signal, and greatly facilitate the subsequent functions to directly retrieve and call target signals for processing.
Thirdly, aiming at the problem that the identity of each signal is unknown by an electromagnetic spectrum passive monitoring system, target signal parameter information estimation and labeling processing is adopted in the method, core characteristic parameters after independence separation are estimated and are labeled on a generated corresponding signal file, the information content of a generated small-scale database is effectively enhanced, and the subsequent identification and positioning of the target signal can be directly supported.
Drawings
FIG. 1 is a block diagram of a system architecture of an embodiment of the present invention;
FIG. 2 is a flow diagram of a method of an embodiment of the invention;
fig. 3 is a schematic diagram of a signal detection method according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example 1
The invention provides a frequency spectrum monitoring big data cleaning method based on deep learning detection, which takes radio frequency spectrum sensing data such as electromagnetic frequency spectrum monitoring data and electronic reconnaissance data as processing objects, wherein the monitoring or reconnaissance objects comprise military and civil communication signals such as AM, FM (amplitude modulation) broadcasting, broadcasting television, interphones, digital clusters, ground-air talkback, short-wave ultrashort-wave radio stations, data chains and the like, and also comprise various radar signals and the like, the data sources comprise various levels of radio monitoring stations such as countries, provinces and municipalities, prefectures and counties and electronic reconnaissance monitoring equipment of various military troops and the like.
The method flow chart is shown in fig. 1, and comprises the following steps:
s1: carrying out Fourier transform on the radio frequency spectrum sensing data received in real time to obtain frequency spectrum data; and splicing the frequency spectrum data according to the sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data.
In particular, the method comprises the following steps of,
s1.1: scanning at the central frequency point of the equipment through a receiver and acquiring a broadband electromagnetic spectrum monitoring signal acquired by a frequency band in the bandwidth of the equipment;
real and virtual IQ signals in the broadband electromagnetic spectrum monitoring signals are converted into complex signals through complex addition of IQ two paths of signals;
and performing Hilbert transform on the I/Q signal in the broadband electromagnetic spectrum monitoring signal to obtain another path of Q/I signal corresponding to the I/Q signal, and performing complex addition on the IQ two paths of signals to convert the IQ two paths of signals into complex signals.
S1.2: taking the total number of sampling points as N, equally dividing the complex signals into M equal parts, carrying out zero filling operation on data less than N points until the number of the complex signals is N, and carrying out Fourier transform on each equal part of the complex signals to obtain the frequency spectrum data of the equal part:
in the formula, ceil (·) denotes a round-up operation, fsThe sampling rate of a receiver is represented, L represents the frequency spectrum resolution of the complex signal after conversion, and the value of L is an integer randomly selected within the range of 100-500 Hz;
s1.3: splicing the frequency spectrum data after Fourier transform according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data; the rows in the time-frequency domain matrix represent the sampling bandwidth of the data and the columns represent the sampling times of the data.
S2: according to a signal detection algorithm based on deep learning, signal detection is performed on all signals in the time-frequency domain matrix, specifically as shown in fig. 3, time-frequency domain distribution information of each signal is obtained.
Specifically, the method comprises the following steps: defining thresholds for signal elements in the time-frequency domain matrix:
in the formula, AnoiseRepresenting the noise amplitude in the time-frequency domain matrix;
comparing each element in the time-frequency domain matrix row by row and column by column with a threshold, and forming a signal detection area by more than 10 continuous elements larger than the threshold;
and respectively taking the total number of the maximum rows and the maximum columns in each signal detection area as the length and the width of a rectangular frame corresponding to the maximum rows and the maximum columns, and respectively representing the continuous occurrence time of the signal, signal bandwidth information and frequency offset information by the central values of the length, the width and the width to obtain 4 elements of the upper left, the lower left, the upper right and the lower right of each rectangular frame as detection signals in a time-frequency domain matrix.
S3: and carrying out time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals.
Specifically, as shown in fig. 3: the central value of the length, width and width of each rectangular frame is calculated respectively as shown in the following formula:
tm＝(ym2-ym1)×N/fs1
swm＝(xm2-xm1)×fs1/NFFT
frem＝(xm2+xm1)×fs1/(2×NFFT)
in the formula, tm、swm、fremRespectively represents the length, width and width of the m-th rectangular frame, xm1And ym1Coordinate value, x, representing the upper left element of the mth rectangular framem2And ym2Coordinate value, f, representing the lower right element of the mth rectangular frames1Is the sampling rate;
truncating each detection signal by length tmThen the down-conversion operation is carried out on the intercepted signal to obtain the corresponding intercepted signalThe zero-frequency signal of (1) utilizes the passband value as swmThe adaptive filter carries out filtering processing on the zero-frequency signal;
shifting the frequency spectrum of the filtered signal to a frequency offset value fremAnd (4) obtaining separated independent signals.
Wherein the down-conversion operation is as follows:
y(n)＝x(n)*exp(-j*2*π*frem)
where y (n) represents a signal sequence after down-conversion, and x (n) represents a signal sequence before the down-conversion.
S4: and measuring the core parameters of the signals of the separated independent signals to obtain the bandwidth parameters of the signals.
Specifically, the method comprises the following steps:
and respectively carrying out modulation type identification, frequency offset accurate measurement, signal strength measurement and signal bandwidth measurement on each separated independent signal, and carrying out code rate measurement on the signal with the modulation type parameter of a digital type in the modulation type identification result.
S5: and selecting the common multiple of the signal bandwidth as a new sampling rate according to the bandwidth parameter of the signal under the condition of satisfying the signal distortion-free condition, performing down-sampling rate compression processing on the separated independent signal to obtain a compressed signal, and storing each compressed signal.
Satisfies the signal distortion-free condition, i.e. satisfies the Nyquist sampling law, i.e. the signal sampling rate f after compressions2Greater than the bandwidth of the separated independent signals.
And performing down-sampling rate compression processing on the separated independent signals, as shown in the following formula:
in the formula, y (l)2) Represents the decimated compressed signal sequence, x (-) represents the separated independent signal sequence, floor (-) represents the floor operation.
S6: writing each compressed signal into a designated signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the name of the generated designated signal file.
S7: and performing homologous identification and classification on each signal according to the parameter information of the compressed independent signal, and realizing target radiation source positioning by combining a frequency difference joint estimation and positioning model.
A spectrum monitoring big data cleaning system based on deep learning detection is disclosed, and the system structure is shown in FIG. 2, and includes:
the data acquisition module scans the central frequency point of the equipment through the receiver and acquires broadband electromagnetic spectrum monitoring signals acquired by the frequency band in the bandwidth;
a signal detection module comprising:
the preprocessing module is used for carrying out Fourier transform on the broadband electromagnetic spectrum monitoring signals to obtain frequency spectrum data, and splicing the frequency spectrum data according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data;
the deep learning detection module is used for carrying out signal detection on all signals in the time-frequency domain matrix according to a signal detection algorithm based on deep learning to obtain time-frequency domain distribution information of each signal;
the multi-signal separation module is used for carrying out time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals;
the signal measurement module is used for measuring the core parameters of the separated independent signals to obtain the bandwidth parameters of the signals;
the signal compression module is used for carrying out down-sampling rate compression processing on the separated independent signals to obtain compressed signals;
and the signal label injection module is used for writing each compressed signal into an appointed signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the name of the generated appointed signal file.
In this embodiment, the effect of the present invention is further explained by combining with a simulation experiment:
1. simulation experiment conditions are as follows:
the hardware platform of the simulation experiment of the invention is as follows: the processor is an Intel i7-10850H CPU, the main frequency is 2.7GHz, and the internal memory is 16 GB.
The software platform of the simulation experiment of the invention is as follows: windows 10 operating system and Matlab2020b, python 3.6.
The input data used by the simulation experiment of the invention is data of the school district of the SiAn electronic technology university collected by the applicant by using a radio monitoring station, and the data format is mat.
2. Simulation content and result analysis thereof:
the simulation experiment 1 of the invention adopts two prior arts (a compression sampling signal detection method based on characteristic value energy, a signal type identification method based on characteristic parameters) to perform signal detection and signal parameter measurement processing on collected data, and obtains a frequency spectrum data file.
The simulation experiment 2 of the invention utilizes the method of the invention to carry out signal detection and signal parameter measurement processing on the acquired data, and then carries out multi-signal separation and signal compression on the data after the detection and measurement processing to obtain a compressed data file.
In the simulation experiment, two prior arts are adopted:
the compressed sampling signal detection method based on the characteristic value energy refers to a compressed sampling signal detection method based on the characteristic value energy, which is proposed by Pan-Yi et al in the methods of compressed sampling signal detection based on the characteristic value energy, signal processing, 2016,32(07):849-858.
The signal type identification method based on the characteristic parameters refers to the signal modulation type identification method based on the characteristic parameters, which is proposed by Schaheifei et al in' non-cooperative signal modulation identification algorithm based on the combined characteristic parameter extraction, Communications 2020,41(07):172-185.
In order to verify the effect of the simulation experiment of the invention, the memory size occupied by the frequency spectrum data obtained by the simulation experiment 1 and the originally collected data is compared, and the compression rate values are respectively as follows:
comparing the memory size occupied by the compressed data obtained in the simulation experiment 2 and the originally acquired data, wherein the compression rate value is as follows:
combined compressibility1And compression ratio2Compared with the frequency spectrum data compression ratio, the compression ratio of the compressed data is improved by 50 percent and the compression ratio of the original data is improved by 70 percent, so that the problems that the stored data cannot be subjected to signal processing on a non-frequency domain and the common information parameters of signals are incomplete during database retrieval in the prior art are solved, and the system and the method for cleaning the large signal data are very practical.
The present invention is not limited to the above preferred embodiments, and any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. The frequency spectrum monitoring big data cleaning method based on deep learning detection is characterized by comprising the following steps:
carrying out Fourier transform on the radio frequency spectrum sensing data received in real time to obtain frequency spectrum data; splicing the frequency spectrum data according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data;
performing signal detection on all signals in the time-frequency domain matrix according to a signal detection algorithm based on deep learning to obtain time-frequency domain distribution information of each signal;
carrying out time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals;
performing core parameter measurement on the separated independent signals to obtain parameter information corresponding to the signals;
selecting a common multiple of signal bandwidth as a new sampling rate according to the bandwidth parameter of the signal under the condition of no signal distortion, and performing down-sampling rate compression processing on the separated independent signal to obtain a compressed independent signal;
generating an independence data file for each compressed independent signal and the corresponding parameter information thereof;
and performing homologous identification and classification on each signal according to the parameter information of the compressed independent signal, and realizing target radiation source positioning by combining a frequency difference joint estimation and positioning model.
2. The method for cleaning the spectrum monitoring big data based on the deep learning detection as claimed in claim 1, further comprising storing and labeling the compressed signal, comprising the steps of:
writing each compressed independent signal into a specified signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the name of the generated specified signal file.
3. The method for cleaning spectrum monitoring big data based on deep learning detection as claimed in claim 1, wherein the radio spectrum sensing data comprises electromagnetic spectrum monitoring data and electronic reconnaissance data, and the monitored objects comprise AM, FM broadcast, broadcast television, interphone, digital cluster, ground-air intercom, short wave ultrashort wave radio, data link and various radar signals.
4. The method for cleaning the spectrum monitoring big data based on the deep learning detection as claimed in claim 1, wherein the real-time receiving of the broadband electromagnetic spectrum monitoring signal comprises the following steps:
scanning at the central frequency point of the equipment through a receiver and acquiring a broadband electromagnetic spectrum monitoring signal acquired by a frequency band in the bandwidth of the equipment;
real and virtual IQ signals in the broadband electromagnetic spectrum monitoring signals are converted into complex signals through complex addition of IQ two paths of signals;
and performing Hilbert transform on the I/Q signal in the broadband electromagnetic spectrum monitoring signal to obtain another path of Q/I signal corresponding to the I/Q signal, and performing complex addition on the IQ two paths of signals to convert the IQ two paths of signals into complex signals.
5. The method for cleaning the large data of the spectrum monitoring based on the deep learning detection as claimed in claim 4, wherein the generation of the time-frequency domain matrix comprises the following steps:
taking the total number of sampling points as N, equally dividing the complex signals into M equal parts, carrying out zero filling operation on data less than N points until the number of the complex signals is N, and carrying out Fourier transform on each equal part of the complex signals to obtain the frequency spectrum data of the equal part:
in the formula, ceil (·) denotes a round-up operation, fsThe sampling rate of a receiver is represented, L represents the frequency spectrum resolution of the complex signal after conversion, and the value of L is an integer randomly selected within the range of 100-500 Hz;
splicing the frequency spectrum data after Fourier transform according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data; the rows in the time-frequency domain matrix represent the sampling bandwidth of the data and the columns represent the sampling times of the data.
6. The method for cleaning big data of spectrum monitoring based on deep learning detection as claimed in claim 1, wherein the signal detection is performed on all signals in the time-frequency domain matrix according to the signal detection algorithm based on deep learning, comprising the following steps:
defining thresholds for signal elements in the time-frequency domain matrix:
in the formula, AnoiseRepresenting the noise amplitude in the time-frequency domain matrix;
comparing each element in the time-frequency domain matrix row by row and column by column with a threshold, and forming a signal detection area by more than 10 continuous elements larger than the threshold;
and respectively taking the total number of the maximum rows and the maximum columns in each signal detection area as the length and the width of a rectangular frame corresponding to the maximum rows and the maximum columns, and respectively representing the continuous occurrence time of the signal, signal bandwidth information and frequency offset information by the central values of the length, the width and the width to obtain 4 elements of the upper left, the lower left, the upper right and the lower right of each rectangular frame as detection signals in a time-frequency domain matrix.
7. The method for cleaning the large data of the frequency spectrum monitoring based on the deep learning detection as claimed in claim 6, wherein the time-frequency domain separation is performed on the time-frequency domain distribution information of each signal, and the method comprises the following steps:
the central value of the length, width and width of each rectangular frame is calculated respectively as shown in the following formula:
tm＝(ym2-ym1)×N/fs1
swm＝(xm2-xm1)×fs1/NFFT
frem＝(xm2+xm1)×fs1/(2×NFFT)
in the formula, tm、swm、fremRespectively represents the length, width and width of the m-th rectangular frame, xm1And ym1Coordinate value, x, representing the upper left element of the mth rectangular framem2And ym2Coordinate value, f, representing the lower right element of the mth rectangular frames1Is the sampling rate;
truncating each detection signal by length tmThen the intercepted signal is subjected to down-conversion operation to obtain a zero-frequency signal corresponding to the intercepted signal, and the passband value is used as swmThe adaptive filter carries out filtering processing on the zero-frequency signal;
filtering the obtained signalThe processed signal spectrum is shifted to the frequency offset value fremObtaining separated independent signals;
the down-conversion operation is as follows:
y(n)＝x(n)*exp(-j*2*π*frem)
where y (n) represents a signal sequence after down-conversion, and x (n) represents a signal sequence before the down-conversion.
8. The method for cleaning spectrum monitoring big data based on deep learning detection as claimed in claim 1, wherein the performing signal core parameter measurement on the separated independent signals comprises:
and respectively carrying out modulation type identification, frequency offset accurate measurement, signal strength measurement and signal bandwidth measurement on each separated independent signal, and carrying out code rate measurement on the signal with the modulation type parameter of a digital type in the modulation type identification result.
9. The method for cleaning spectral monitoring big data based on deep learning detection as claimed in claim 7, wherein the signal distortion-free condition is that Nyquist sampling law, i.e. compressed signal sampling rate f, is satisfieds2Greater than the bandwidth of the separated independent signal;
the separated independent signals are compressed at a reduced sampling rate, which is shown as the following formula:
in the formula, y (l)2) Represents the decimated compressed signal sequence, x (-) represents the separated independent signal sequence, floor (-) represents the floor operation.
10. Big data cleaning system of spectrum monitoring based on deep learning detects, its characterized in that includes:
the data acquisition module scans the central frequency point of the equipment through the receiver and acquires broadband electromagnetic spectrum monitoring signals acquired by the frequency band in the bandwidth;
a signal detection module comprising:
the preprocessing module is used for carrying out Fourier transform on the broadband electromagnetic spectrum monitoring signals to obtain frequency spectrum data, and splicing the frequency spectrum data according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data;
the deep learning detection module is used for carrying out signal detection on all signals in the time-frequency domain matrix according to a signal detection algorithm based on deep learning to obtain time-frequency domain distribution information of each signal;
the multi-signal separation module is used for carrying out time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals;
the signal measurement module is used for measuring the core parameters of the separated independent signals to obtain the bandwidth parameters of the signals;
the signal compression module is used for carrying out down-sampling rate compression processing on the separated independent signals to obtain compressed signals;
and the signal label injection module is used for writing each compressed signal into an appointed signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the name of the generated appointed signal file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210085970.0A CN114422049B (en) | 2022-01-25 | 2022-01-25 | Frequency spectrum monitoring big data cleaning method and system based on deep learning detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210085970.0A CN114422049B (en) | 2022-01-25 | 2022-01-25 | Frequency spectrum monitoring big data cleaning method and system based on deep learning detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114422049A true CN114422049A (en) | 2022-04-29 |
| CN114422049B CN114422049B (en) | 2022-09-23 |
Family
ID=81277557
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210085970.0A Active CN114422049B (en) | 2022-01-25 | 2022-01-25 | Frequency spectrum monitoring big data cleaning method and system based on deep learning detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114422049B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107301434A (en) * | 2017-07-28 | 2017-10-27 | 西安交通大学 | Blind separation hybrid matrix method of estimation based on synchronous compression Short Time Fourier Transform |
| CN110244271A (en) * | 2019-05-17 | 2019-09-17 | 中国人民解放军战略支援部队信息工程大学 | Radar emitter sorting recognition methods and device based on multiple simultaneous compressed transform |
| US20200143279A1 (en) * | 2018-11-06 | 2020-05-07 | DeepSig Inc. | Radio frequency band segmentation, signal detection and labelling using machine learning |
| CN112751629A (en) * | 2021-01-15 | 2021-05-04 | 中国人民解放军战略支援部队信息工程大学 | Broadband specific signal detection method based on time-frequency image processing |
-
2022
- 2022-01-25 CN CN202210085970.0A patent/CN114422049B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107301434A (en) * | 2017-07-28 | 2017-10-27 | 西安交通大学 | Blind separation hybrid matrix method of estimation based on synchronous compression Short Time Fourier Transform |
| US20200143279A1 (en) * | 2018-11-06 | 2020-05-07 | DeepSig Inc. | Radio frequency band segmentation, signal detection and labelling using machine learning |
| CN110244271A (en) * | 2019-05-17 | 2019-09-17 | 中国人民解放军战略支援部队信息工程大学 | Radar emitter sorting recognition methods and device based on multiple simultaneous compressed transform |
| CN112751629A (en) * | 2021-01-15 | 2021-05-04 | 中国人民解放军战略支援部队信息工程大学 | Broadband specific signal detection method based on time-frequency image processing |
Non-Patent Citations (3)
| Title |
|---|
| FARRUKH AZIZ BHATTI等: "Shared Spectrum Monitoring Using Deep Learning", 《IEEE TRANSACTIONS ON COGNITIVE COMMUNICATIONS AND NETWORKING》 * |
| 李润东: "基于深度学习的通信信号智能盲检测与识别技术研究", 《中国博士学位论文全文数据库信息科技辑》 * |
| 林冲等: "通信信号调制识别综述", 《中国电子科学研究院学报》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114422049B (en) | 2022-09-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111585671B (en) | Electric power LTE wireless private network electromagnetic interference monitoring and identifying method | |
| CN102148987B (en) | Compressed sensing image reconstructing method based on prior model and 10 norms | |
| US8478319B2 (en) | Feature extraction and data compression system and method for distributed sensor networks | |
| CN110197127A (en) | Wireless signal detection and electromagnetic interference categorizing system and method based on deep learning | |
| CN109117747A (en) | Radar signal classification method based on cyclo-stationary Yu depth convolutional neural networks | |
| CN106301631B (en) | A kind of coprime lack sampling frequency spectrum sensing method and its device based on Subspace Decomposition | |
| CN109379698B (en) | Cell measurement report positioning method and system based on channel model feature extraction | |
| CN111950410B (en) | Online electromagnetic environment monitoring device and method based on radio fingerprint identification | |
| CN108881096B (en) | Spatial modulation MQAM base station based on phase decision | |
| CN102176779A (en) | Wireless multimedia sensing network video signal adaptive sampling and spectrum allocation method | |
| CN112867010A (en) | Radio frequency fingerprint embedded real-time identification method and system based on convolutional neural network | |
| CN101635937B (en) | Idle frequency band detector applied to UHF frequency band | |
| CN103364646A (en) | Rapid microwave anechoic room antenna far field measurement method | |
| CN112183279A (en) | Communication radiation source individual identification method based on IQ graph characteristics | |
| CN87103288A (en) | Method that in the short-wave radio network, communicates and employed equipment | |
| CN114422049B (en) | Frequency spectrum monitoring big data cleaning method and system based on deep learning detection | |
| CN104408474B (en) | Emitter Recognition and device | |
| CN202918327U (en) | Automatic classification system for digital modulation signal | |
| CN109085532B (en) | Large-aperture two-dimensional short wave direction finding system | |
| CN114584227B (en) | Automatic burst signal detection method | |
| CN103152790B (en) | Two-graded fusion Modulation Identification method based on dependency sub-clustering | |
| CN104181508A (en) | Threatening radar signal detection method based on compressed sensing | |
| CN113705646A (en) | Radio frequency fine characteristic information extraction method and system based on semi-supervised meta learning | |
| Raghavendra et al. | Wideband spectrum sensing using sub-Nyquist sampling approaches | |
| Zhang et al. | DTV radio spectrum anomaly detection based on an improved GAN |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |