CN114422049B - Frequency spectrum monitoring big data cleaning method and system based on deep learning detection - Google Patents
Frequency spectrum monitoring big data cleaning method and system based on deep learning detection Download PDFInfo
- Publication number
- CN114422049B CN114422049B CN202210085970.0A CN202210085970A CN114422049B CN 114422049 B CN114422049 B CN 114422049B CN 202210085970 A CN202210085970 A CN 202210085970A CN 114422049 B CN114422049 B CN 114422049B
- Authority
- CN
- China
- Prior art keywords
- signal
- signals
- frequency
- time
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B17/00—Monitoring; Testing
- H04B17/30—Monitoring; Testing of propagation channels
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R23/00—Arrangements for measuring frequencies; Arrangements for analysing frequency spectra
- G01R23/16—Spectrum analysis; Fourier analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Electromagnetism (AREA)
- Mathematical Physics (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention provides a frequency spectrum monitoring big data cleaning method and a system based on deep learning detection, comprising the following steps: the system comprises a data acquisition module, a signal detection module, a multi-signal separation module, a signal measurement module, a signal compression module and a signal label injection module. The method specifically comprises the following steps: receiving a broadband electromagnetic spectrum monitoring signal in real time; generating a time-frequency domain matrix; detecting all signals in the time-frequency domain matrix by using a signal detection algorithm; carrying out independence separation on a plurality of detection signals distributed in the time-frequency domain matrix; measuring core parameters of the separated signals; extracting and compressing the separated signals; the compressed signal is labeled. The invention has the advantages of easy development, effective saving of database storage amount, increase of unit database information amount and the like.
Description
Technical Field
The invention relates to the technical field of wireless communication, in particular to a frequency spectrum monitoring big data cleaning method and system based on deep learning detection.
Background
The purpose of electromagnetic spectrum monitoring is to support the spectrum management process as a whole, solve the interference problem, and provide scientific basis and technical support for frequency assignment, scientific management of spectrum, and radio planning. Monitoring signal analysis and monitoring data processing are main work in daily radio monitoring, and the basic task of the monitoring signal analysis and monitoring data processing is to determine information such as a modulation mode, signal parameters and occupancy rate of a received signal under the conditions of a multi-signal environment and noise interference so as to provide a basis for further signal analysis and processing.
The military and civil communication signals and various radar signals of AM, FM broadcasting, broadcasting television, interphone, digital cluster, ground-air talkback, short wave ultra-short wave radio stations, data chains and the like are contained in the data. As a result, electromagnetic spectrum environments are becoming increasingly complex, and radio systems are monitoring more and more electromagnetic spectrum data, and storage, retrieval and processing of large broadband electromagnetic spectrum monitoring data face serious technical challenges. In the face of massive monitoring data, how to effectively reduce the storage scale, improve the signal retrieval efficiency and improve the signal processing capacity to realize the rapid positioning of the target is an important problem to be solved urgently in the field of electromagnetic spectrum monitoring and control and electronic reconnaissance.
In order to solve the problems, the invention provides a novel frequency spectrum monitoring big data cleaning method and system based on deep learning detection.
Disclosure of Invention
In order to solve the problems, the invention provides a method and a system for cleaning frequency spectrum monitoring big data based on deep learning detection, which are used for solving the problem of database memory waste caused by overlarge monitoring signal data in the existing system, solving the problem of incomplete target signal parameter information in a common method and solving the problem of difficulty in independent processing of multi-signal time domain aliasing in broadband frequency spectrum sensing data.
In order to achieve the purpose, the invention provides the following technical scheme.
The method and the system for cleaning the frequency spectrum monitoring big data based on deep learning detection comprise the following steps:
carrying out Fourier transform on the radio frequency spectrum sensing data received in real time to obtain frequency spectrum data; splicing the frequency spectrum data according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data;
performing signal detection on all signals in the time-frequency domain matrix according to a signal detection algorithm based on deep learning to obtain time-frequency domain distribution information of each signal;
performing time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals;
measuring core parameters of the separated independent signals to obtain parameter information corresponding to the signals;
selecting a common multiple of signal bandwidth as a new sampling rate according to the bandwidth parameter of the signal under the condition of no signal distortion, and performing down-sampling rate compression processing on the separated independent signal to obtain a compressed independent signal;
generating an independence data file for each compressed independent signal and the corresponding parameter information thereof;
and performing homologous identification and classification on each signal according to the compressed parameter information of the independent signal, and combining a frequency difference joint estimation and positioning model to realize target radiation source positioning.
Preferably, the method further comprises storing and labeling the compressed signal, comprising the steps of:
writing each compressed independent signal into a specified signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the name of the generated specified signal file.
Preferably, the radio frequency spectrum sensing data comprises electromagnetic spectrum monitoring data and electronic reconnaissance data, and the monitored objects comprise AM, FM broadcast, broadcast television, interphone, digital cluster, ground-to-air intercom, short-wave ultra-short wave radio station, data link and various radar signals.
Preferably, the real-time reception of the broadband electromagnetic spectrum monitoring signal comprises the following steps:
scanning at the central frequency point of the equipment through a receiver and acquiring a broadband electromagnetic spectrum monitoring signal acquired by a frequency band in the bandwidth of the equipment;
real and virtual IQ signals in the broadband electromagnetic spectrum monitoring signals are converted into complex signals through complex addition of IQ two paths of signals;
and performing Hilbert transform on the I/Q signal in the broadband electromagnetic spectrum monitoring signal to obtain another path of Q/I signal corresponding to the I/Q signal, and performing complex addition on the IQ two paths of signals to convert the IQ two paths of signals into complex signals.
Preferably, the generation of the time-frequency domain matrix includes the following steps:
taking the total number of sampling points as N, equally dividing the complex signals into M equal parts, carrying out zero filling operation on data less than N points until the number of the complex signals is N, and carrying out Fourier transform on each equal part of the complex signals to obtain the frequency spectrum data of the equal part:
in the formula, ceil (·) denotes a round-up operation, f s The sampling rate of a receiver is represented, L represents the frequency spectrum resolution of the complex signal after conversion, and the value of L is an integer randomly selected within the range of 100-500 Hz;
splicing the frequency spectrum data after Fourier transform according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data; the rows in the time-frequency domain matrix represent the sampling bandwidth of the data and the columns represent the sampling times of the data.
Preferably, the signal detection of all signals in the time-frequency domain matrix according to the signal detection algorithm based on deep learning includes the following steps:
defining thresholds for signal elements in the time-frequency domain matrix:
in the formula, A noise Representing the noise amplitude in the time-frequency domain matrix;
comparing each element in the time-frequency domain matrix row by row and column by column with a threshold, and forming a signal detection area by more than 10 continuous elements larger than the threshold;
and respectively taking the total number of the maximum rows and the maximum columns in each signal detection area as the length and the width of a rectangular frame corresponding to the maximum rows and the maximum columns, and respectively representing the continuous occurrence time of the signal, signal bandwidth information and frequency offset information by the central values of the length, the width and the width to obtain 4 elements of the upper left, the lower left, the upper right and the lower right of each rectangular frame as detection signals in a time-frequency domain matrix.
Preferably, the time-frequency domain separating the time-frequency domain distribution information of each signal includes:
the central value of the length, width and width of each rectangular frame is calculated respectively as shown in the following formula:
t m ＝(y m2 -y m1 )×N/f s1
sw m ＝(x m2 -x m1 )×f s1 /NFFT
fre m ＝(x m2 +x m1 )×f s1 /(2×NFFT)
in the formula, t m 、sw m 、fre m Respectively represents the length, width and width of the m-th rectangular frame, x m1 And y m1 Coordinate value, x, representing the upper left element of the mth rectangular frame m2 And y m2 Coordinate value, f, representing the lower right element of the mth rectangular frame s1 Is the sampling rate;
truncating each detection signal by length t m Then, the down-conversion operation is performed on the intercepted signal to obtain the zero corresponding to the intercepted signalFrequency signal, using pass band value of sw m The adaptive filter carries out filtering processing on the zero-frequency signal;
frequency-shifting the filtered signal frequency spectrum to a frequency offset fre m Obtaining separated independent signals;
the down-conversion operation is shown as follows:
y(n)＝x(n)*exp(-j*2*π*fre m )
in the formula, y (n) represents a signal sequence after down-conversion, and x (n) represents a signal sequence before frequency conversion.
Preferably, the signal core parameter measurement on the separated independent signals includes:
and respectively carrying out modulation type identification, frequency offset accurate measurement, signal strength measurement and signal bandwidth measurement on each separated independent signal, and carrying out code rate measurement on the signal with the modulation type parameter of a digital type in the modulation type identification result.
Preferably, the signal distortion-free condition is that the nyquist sampling law, i.e. the compressed signal sampling rate f, is satisfied s2 Greater than the bandwidth of the separated independent signal;
the separated independent signals are compressed at a reduced sampling rate, which is shown as the following formula:
in the formula, y (l) 2 ) Represents the decimated compressed signal sequence, x (-) represents the separated independent signal sequence, floor (-) represents the floor operation.
Frequency spectrum monitoring big data cleaning system based on deep learning detection includes:
the data acquisition module scans the central frequency point of the equipment through the receiver and acquires broadband electromagnetic spectrum monitoring signals acquired by the frequency band in the bandwidth;
a signal detection module comprising:
the preprocessing module is used for carrying out Fourier transform on the broadband electromagnetic spectrum monitoring signals to obtain frequency spectrum data, and splicing the frequency spectrum data according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data;
the deep learning detection module is used for carrying out signal detection on all signals in the time-frequency domain matrix according to a signal detection algorithm based on deep learning to obtain time-frequency domain distribution information of each signal;
the multi-signal separation module is used for carrying out time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals;
the signal measurement module is used for measuring the core parameters of the separated independent signals to obtain the bandwidth parameters of the signals;
the signal compression module is used for carrying out down-sampling rate compression processing on the separated independent signals to obtain compressed signals;
and the signal label injection module is used for writing each compressed signal into an appointed signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the name of the generated appointed signal file.
The invention has the beneficial effects that:
firstly, aiming at a large-scale spectrum monitoring database formed by each level of non-commission civil spectrum monitoring stations and military electronic reconnaissance equipment, a compression processing module in the system has a multi-bandwidth self-adaptive signal extraction and compression function, so that the data storage scale can be effectively reduced, and the defects of overlarge storage pressure, overhigh storage system construction cost and the like in the traditional spectrum database construction are overcome.
Secondly, aiming at the problem that the time domain aliasing of a plurality of signals is difficult to process independently in the broadband frequency spectrum monitoring data, the multi-signal independence separation technology based on the time-frequency domain matrix detection in the method can effectively realize the separation and extraction processing of all detected signals in the time domain aliasing data, form an independence data file of each signal, and greatly facilitate the subsequent functions to directly retrieve and call target signals for processing.
Thirdly, aiming at the problem that the identity of each signal is unknown by an electromagnetic spectrum passive monitoring system, target signal parameter information estimation and labeling processing is adopted in the method, core characteristic parameters after independence separation are estimated and are labeled on a generated corresponding signal file, the information content of a generated small-scale database is effectively enhanced, and the subsequent identification and positioning of the target signal can be directly supported.
Drawings
FIG. 1 is a block diagram of a system architecture of an embodiment of the present invention;
FIG. 2 is a flow diagram of a method of an embodiment of the invention;
fig. 3 is a schematic diagram of a signal detection method according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example 1
The invention provides a frequency spectrum monitoring big data cleaning method based on deep learning detection, which takes radio frequency spectrum sensing data such as electromagnetic frequency spectrum monitoring data and electronic reconnaissance data as processing objects, wherein the monitoring or reconnaissance objects comprise military and civil communication signals such as AM, FM (amplitude modulation) broadcasting, broadcasting television, interphones, digital clusters, ground-air talkbacks, short wave ultrashort wave radio stations, data chains and the like, and further comprise various radar signals and the like, the data sources comprise various levels of radio monitoring stations such as countries, provinces and municipalities, prefectures and counties and the like, and various military and military variety electronic reconnaissance monitoring equipment and the like.
The method flow chart is shown in fig. 1, and comprises the following steps:
s1: carrying out Fourier transform on the radio frequency spectrum sensing data received in real time to obtain frequency spectrum data; and splicing the frequency spectrum data according to the sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data.
In particular, the method comprises the following steps of,
s1.1: scanning at the central frequency point of the equipment through a receiver and acquiring broadband electromagnetic spectrum monitoring signals acquired by the frequency band in the bandwidth of the equipment;
real and virtual IQ signals in the broadband electromagnetic spectrum monitoring signals are converted into complex signals through complex addition of IQ two paths of signals;
and performing Hilbert transform on the I/Q signal in the broadband electromagnetic spectrum monitoring signal to obtain another path of Q/I signal corresponding to the I/Q signal, and performing complex addition on the IQ two paths of signals to convert the IQ two paths of signals into complex signals.
S1.2: taking the total number of sampling points as N, equally dividing the complex signals into M equal parts, carrying out zero filling operation on data less than N points until the number of the complex signals is N, and carrying out Fourier transform on each equal part of the complex signals to obtain the frequency spectrum data of the equal part:
where ceil (·) denotes a round-up operation, f s The sampling rate of a receiver is represented, L represents the frequency spectrum resolution of the complex signal after conversion, and the value of L is an integer randomly selected within the range of 100-500 Hz;
s1.3: splicing the frequency spectrum data after Fourier transform according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data; the rows in the time-frequency domain matrix represent the sampling bandwidth of the data and the columns represent the sampling times of the data.
S2: according to a signal detection algorithm based on deep learning, signal detection is performed on all signals in the time-frequency domain matrix, specifically as shown in fig. 3, time-frequency domain distribution information of each signal is obtained.
Specifically, the method comprises the following steps: defining thresholds for signal elements in the time-frequency domain matrix:
in the formula, A noise Representing the noise amplitude in the time-frequency domain matrix;
comparing each element in the time-frequency domain matrix row by row and column by column with a threshold, and forming a signal detection area by more than 10 continuous elements larger than the threshold;
and respectively taking the total number of the maximum rows and the maximum columns in each signal detection area as the length and the width of a rectangular frame corresponding to the maximum rows and the maximum columns, and respectively representing the continuous occurrence time of the signal, signal bandwidth information and frequency offset information by the central values of the length, the width and the width to obtain 4 elements of the upper left, the lower left, the upper right and the lower right of each rectangular frame as detection signals in a time-frequency domain matrix.
S3: and carrying out time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals.
Specifically, as shown in fig. 3: the central value of the length, width and width of each rectangular frame is calculated respectively as shown in the following formula:
t m ＝(y m2 -y m1 )×N/f s1
sw m ＝(x m2 -x m1 )×f s1 /NFFT
fre m ＝(x m2 +x m1 )×f s1 /(2×NFFT)
in the formula, t m 、sw m 、fre m Respectively represents the length, width and width of the m-th rectangular frame, x m1 And y m1 Coordinate value, x, representing the top left element of the mth rectangular frame m2 And y m2 Coordinate value, f, representing the lower right element of the mth rectangular frame s1 Is the sampling rate;
for each detection signal, truncating to length t m Then the intercepted signal is subjected to down-conversion operation to obtain a zero-frequency signal corresponding to the intercepted signal, and the passband value is used as sw m The adaptive filter carries out filtering processing on the zero-frequency signal;
shifting the frequency spectrum of the filtered signal to a frequency offset value fre m And (4) obtaining separated independent signals.
Wherein the down-conversion operation is shown as follows:
y(n)＝x(n)*exp(-j*2*π*fre m )
where y (n) represents a signal sequence after down-conversion, and x (n) represents a signal sequence before the down-conversion.
S4: and measuring the core parameters of the signals of the separated independent signals to obtain the bandwidth parameters of the signals.
Specifically, the method comprises the following steps:
and respectively carrying out modulation type identification, frequency offset accurate measurement, signal strength measurement and signal bandwidth measurement on each separated independent signal, and carrying out code rate measurement on the signal of which the modulation type parameter is a digital type in the modulation type identification result.
S5: and selecting a common multiple of the signal bandwidth as a new sampling rate according to the bandwidth parameters of the signal under the condition of satisfying the signal non-distortion condition, performing down-sampling rate compression processing on the separated independent signal to obtain a compressed signal, and storing each compressed signal.
Satisfies the signal distortion-free condition, i.e. satisfies the Nyquist sampling law, i.e. the compressed signal sampling rate f s2 Greater than the bandwidth of the separated independent signals.
And performing down-sampling rate compression processing on the separated independent signals, as shown in the following formula:
in the formula, y (l) 2 ) Represents the decimated compressed signal sequence, x (-) represents the separated independent signal sequence, floor (-) represents the floor operation.
S6: writing each compressed signal into a designated signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the name of the generated designated signal file.
S7: and performing homologous identification and classification on each signal according to the parameter information of the compressed independent signal, and realizing target radiation source positioning by combining a frequency difference joint estimation and positioning model.
A spectrum monitoring big data cleaning system based on deep learning detection is disclosed, and the system structure is shown in FIG. 2, and includes:
the data acquisition module scans the central frequency point of the equipment through the receiver and acquires broadband electromagnetic spectrum monitoring signals acquired by the frequency band in the bandwidth;
a signal detection module comprising:
the preprocessing module is used for carrying out Fourier transform on the broadband electromagnetic spectrum monitoring signals to obtain frequency spectrum data, and splicing the frequency spectrum data according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data;
the deep learning detection module is used for carrying out signal detection on all signals in the time-frequency domain matrix according to a signal detection algorithm based on deep learning to obtain time-frequency domain distribution information of each signal;
the multi-signal separation module is used for carrying out time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals;
the signal measurement module is used for measuring the core parameters of the separated independent signals to obtain the bandwidth parameters of the signals;
the signal compression module is used for carrying out down-sampling rate compression processing on the separated independent signals to obtain compressed signals;
and the signal label injection module is used for writing each compressed signal into an appointed signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the generated appointed signal file name.
In this embodiment, the effect of the present invention is further explained by combining with a simulation experiment:
1. simulation experiment conditions are as follows:
the hardware platform of the simulation experiment of the invention is as follows: the processor is an Intel i7-10850H CPU, the main frequency is 2.7GHz, and the memory is 16 GB.
The software platform of the simulation experiment of the invention is as follows: windows 10 operating system and Matlab2020b, python 3.6.
The input data used by the simulation experiment of the invention is the school district data of the SiAn electronic technology university collected by the applicant by using a radio monitoring station, and the data format is mat.
2. Simulation content and result analysis thereof:
the simulation experiment 1 of the invention adopts two prior arts (a compression sampling signal detection method based on characteristic value energy, a signal type identification method based on characteristic parameters) to perform signal detection and signal parameter measurement processing on collected data, and obtains a frequency spectrum data file.
The simulation experiment 2 of the invention utilizes the method of the invention to carry out signal detection and signal parameter measurement processing on the acquired data, and then carries out multi-signal separation and signal compression on the data after the detection and measurement processing to obtain a compressed data file.
In the simulation experiments, two prior arts are adopted to mean:
the compressed sampling signal detection method based on the characteristic value energy refers to a compressed sampling signal detection method based on the characteristic value energy, which is proposed by Pan-Yi et al in the methods of compressed sampling signal detection based on the characteristic value energy, signal processing, 2016,32(07):849-858.
The signal type identification method based on the characteristic parameters refers to the signal modulation type identification method based on the characteristic parameters, which is proposed in' non-cooperative signal modulation identification algorithm based on combined characteristic parameter extraction, communication science, 2020,41(07):172-185.
In order to verify the effect of the simulation experiment of the invention, the memory size occupied by the frequency spectrum data obtained by the simulation experiment 1 and the originally collected data is compared, and the compression rate values are respectively as follows:
comparing the memory size occupied by the compressed data obtained in the simulation experiment 2 and the originally acquired data, wherein the compression rate value is as follows:
combined compressibility 1 And compression ratio 2 Compared with the frequency spectrum data compression rate, the compression rate of the compressed data is improved by 50% and the compression rate of the original data is improved by 70%, the problems that the stored data cannot be subjected to signal processing in a non-frequency domain and common information parameters of signals are incomplete when the database is searched in the prior art are solved, and the system and the method for cleaning the large signal data are very practical.
The present invention is not limited to the above preferred embodiments, and any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. The frequency spectrum monitoring big data cleaning method based on deep learning detection is characterized by comprising the following steps:
carrying out Fourier transform on the radio frequency spectrum sensing data received in real time to obtain frequency spectrum data; splicing the frequency spectrum data according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data;
performing signal detection on all signals in the time-frequency domain matrix according to a signal detection algorithm based on deep learning to obtain time-frequency domain distribution information of each signal;
carrying out time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals;
performing core parameter measurement on the separated independent signal to obtain a bandwidth parameter corresponding to the signal;
selecting a common multiple of a signal bandwidth as a new sampling rate according to a bandwidth parameter of the signal under the condition of satisfying the signal non-distortion condition, and performing down-sampling rate compression processing on the separated independent signal to obtain a compressed independent signal;
generating an independence data file for each compressed independent signal and the corresponding parameter information thereof;
according to the parameter information of the compressed independent signals, performing homologous identification and classification on the signals, and realizing target radiation source positioning by combining a frequency difference joint estimation and positioning model;
writing each compressed independent signal into a specified signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the name of the generated specified signal file.
2. The method for cleaning spectrum monitoring big data based on deep learning detection as claimed in claim 1, wherein the radio spectrum sensing data comprises electromagnetic spectrum monitoring data and electronic reconnaissance data, and the monitored objects comprise AM, FM broadcast, broadcast television, interphone, digital cluster, ground-air intercom, short wave ultrashort wave radio, data link and various radar signals.
3. The method for cleaning the spectrum monitoring big data based on the deep learning detection as claimed in claim 1, wherein the real-time receiving of the broadband electromagnetic spectrum monitoring signal comprises the following steps:
scanning at the central frequency point of the equipment through a receiver and acquiring a broadband electromagnetic spectrum monitoring signal acquired by a frequency band in the bandwidth of the equipment;
real and virtual IQ signals in the broadband electromagnetic spectrum monitoring signals are converted into complex signals through complex addition of IQ two paths of signals;
IQ signals in the broadband electromagnetic spectrum monitoring signals are subjected to Hilbert transformation to obtain QI signals of the other path corresponding to the IQ signals, and the IQ signals are subjected to complex addition to be converted into complex signals.
4. The method for cleaning the large data of the spectrum monitoring based on the deep learning detection as claimed in claim 3, wherein the generation of the time-frequency domain matrix comprises the following steps:
taking the total number of sampling points as N, equally dividing the complex signals into M equal parts, carrying out zero filling operation on data less than N points until the number of the complex signals is N, and carrying out Fourier transform on each equal part of the complex signals to obtain the frequency spectrum data of the equal part:
in the formula, ceil (·) denotes a round-up operation, f s The sampling rate of a receiver is represented, L represents the frequency spectrum resolution of the complex signal after conversion, and the value of L is an integer randomly selected within the range of 100-500 Hz;
splicing the frequency spectrum data after Fourier transform according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data; the rows in the time-frequency domain matrix represent the sampling bandwidth of the data and the columns represent the sampling times of the data.
5. The method for cleaning spectrum monitoring big data based on deep learning detection as claimed in claim 1, wherein the signal detection is performed on all signals in the time-frequency domain matrix according to a signal detection algorithm based on deep learning, comprising the following steps:
defining thresholds for signal elements in the time-frequency domain matrix:
in the formula, A noise Representing the noise amplitude in the time-frequency domain matrix;
comparing each element in the time-frequency domain matrix row by row and column by column with a threshold, and forming a signal detection area by more than 10 continuous elements larger than the threshold;
and respectively taking the total number of the maximum rows and the maximum columns in each signal detection area as the length and the width of a rectangular frame corresponding to the maximum rows and the maximum columns, and respectively representing the continuous occurrence time of the signal, signal bandwidth information and frequency offset information by the central values of the length, the width and the width to obtain 4 elements of the upper left, the lower left, the upper right and the lower right of each rectangular frame as detection signals in a time-frequency domain matrix.
6. The method for cleaning the large data of the frequency spectrum monitoring based on the deep learning detection as claimed in claim 5, wherein the time-frequency domain separation is performed on the time-frequency domain distribution information of each signal, and the method comprises the following steps:
the central value of the length, width and width of each rectangular frame is calculated respectively as shown in the following formula:
t m ＝(y m2 -y m1 )×N/f s1
sw m ＝(x m2 -x m1 )×f s1 /NFFT
fre m ＝(x m2 +x m1 )×f s1 /(2×NFFT)
in the formula, t m 、sw m 、fre m Respectively represents the length, width and width of the m-th rectangular frame, x m1 And y m1 Coordinate value, x, representing the upper left element of the mth rectangular frame m2 And y m2 Coordinate value, f, representing the lower right element of the mth rectangular frame s1 Is the sampling rate;
truncating each detection signal by length t m Then the intercepted signal is subjected to down-conversion operation to obtain a zero-frequency signal corresponding to the intercepted signal, and the passband value is used as sw m The adaptive filter carries out filtering processing on the zero-frequency signal;
shifting the frequency spectrum of the filtered signal to a frequency offset value fre m Obtaining separated independent signals;
the down-conversion operation is as follows:
y(n)＝x(n)*exp(-j*2*π*fre m )
in the formula, y (n) represents a signal sequence after down-conversion, and x (n) represents a signal sequence before frequency conversion.
7. The method for cleaning spectrum monitoring big data based on deep learning detection as claimed in claim 1, wherein the performing signal core parameter measurement on the separated independent signals comprises:
and respectively carrying out modulation type identification, frequency offset accurate measurement, signal strength measurement and signal bandwidth measurement on each separated independent signal, and carrying out code rate measurement on the signal with the modulation type parameter of a digital type in the modulation type identification result.
8. The method for cleaning spectral monitoring big data based on deep learning detection as claimed in claim 6, wherein the signal distortion-free condition is that Nyquist sampling law, i.e. compressed signal sampling rate f, is satisfied s2 Greater than the bandwidth of the separated independent signal;
the separated independent signals are compressed at a reduced sampling rate, which is shown as the following formula:
in the formula, y (l) 2 ) Represents the decimated compressed signal sequence, x (-) represents the separated independent signal sequence, floor (-) represents the floor operation.
9. Big data cleaning system of spectrum monitoring based on deep learning detects, its characterized in that includes:
the data acquisition module scans the central frequency point of the equipment through the receiver and acquires broadband electromagnetic spectrum monitoring signals acquired by the frequency band in the bandwidth;
a signal detection module comprising:
the preprocessing module is used for carrying out Fourier transform on the broadband electromagnetic spectrum monitoring signals to obtain frequency spectrum data, and splicing the frequency spectrum data according to a sampling sequence to obtain a time-frequency domain matrix of the frequency spectrum data;
the deep learning detection module is used for carrying out signal detection on all signals in the time-frequency domain matrix according to a signal detection algorithm based on deep learning to obtain time-frequency domain distribution information of each signal;
the multi-signal separation module is used for carrying out time-frequency domain separation on the time-frequency domain distribution information of each signal to obtain separated independent signals;
the signal measurement module is used for measuring the core parameters of the separated independent signals to obtain the bandwidth parameters of the signals;
the signal compression module is used for selecting the common multiple of the signal bandwidth as a new sampling rate according to the bandwidth parameter of the signal under the condition of meeting the signal distortion-free condition, and performing down-sampling rate compression processing on the separated independent signal to obtain a compressed signal;
and the signal label injection module is used for writing each compressed signal into an appointed signal file, and marking the model information, the center frequency information, the bandwidth information, the acquisition time information and the core parameter information of the compressed signal of the original receiver equipment corresponding to the compressed signal on the generated appointed signal file name.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210085970.0A CN114422049B (en) | 2022-01-25 | 2022-01-25 | Frequency spectrum monitoring big data cleaning method and system based on deep learning detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210085970.0A CN114422049B (en) | 2022-01-25 | 2022-01-25 | Frequency spectrum monitoring big data cleaning method and system based on deep learning detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114422049A CN114422049A (en) | 2022-04-29 |
| CN114422049B true CN114422049B (en) | 2022-09-23 |
Family
ID=81277557
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210085970.0A Active CN114422049B (en) | 2022-01-25 | 2022-01-25 | Frequency spectrum monitoring big data cleaning method and system based on deep learning detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114422049B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107301434A (en) * | 2017-07-28 | 2017-10-27 | 西安交通大学 | Blind separation hybrid matrix method of estimation based on synchronous compression Short Time Fourier Transform |
| CN110244271A (en) * | 2019-05-17 | 2019-09-17 | 中国人民解放军战略支援部队信息工程大学 | Radar emitter sorting recognition methods and device based on multiple simultaneous compressed transform |
| CN112751629A (en) * | 2021-01-15 | 2021-05-04 | 中国人民解放军战略支援部队信息工程大学 | Broadband specific signal detection method based on time-frequency image processing |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200143279A1 (en) * | 2018-11-06 | 2020-05-07 | DeepSig Inc. | Radio frequency band segmentation, signal detection and labelling using machine learning |
-
2022
- 2022-01-25 CN CN202210085970.0A patent/CN114422049B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107301434A (en) * | 2017-07-28 | 2017-10-27 | 西安交通大学 | Blind separation hybrid matrix method of estimation based on synchronous compression Short Time Fourier Transform |
| CN110244271A (en) * | 2019-05-17 | 2019-09-17 | 中国人民解放军战略支援部队信息工程大学 | Radar emitter sorting recognition methods and device based on multiple simultaneous compressed transform |
| CN112751629A (en) * | 2021-01-15 | 2021-05-04 | 中国人民解放军战略支援部队信息工程大学 | Broadband specific signal detection method based on time-frequency image processing |
Non-Patent Citations (3)
| Title |
|---|
| Shared Spectrum Monitoring Using Deep Learning;Farrukh Aziz Bhatti等;《IEEE Transactions on Cognitive Communications and Networking》;20210405;第7卷(第4期);第1171-1185页 * |
| 基于深度学习的通信信号智能盲检测与识别技术研究;李润东;《中国博士学位论文全文数据库信息科技辑》;20220115;第99-130页 * |
| 通信信号调制识别综述;林冲等;《中国电子科学研究院学报》;20211120;第1074-1085页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114422049A (en) | 2022-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110197127A (en) | Wireless signal detection and electromagnetic interference categorizing system and method based on deep learning | |
| CN102148987B (en) | Compressed sensing image reconstructing method based on prior model and 10 norms | |
| US8478319B2 (en) | Feature extraction and data compression system and method for distributed sensor networks | |
| CN102984711A (en) | Multi-user collaborative spectrum sensing method based on single bit compression sensing technology | |
| CN111080105A (en) | Transformer area user-to-user relationship identification method and system based on voltage time sequence data | |
| CN108680796A (en) | Electromagnetic information leakage detecting system and method for computer display | |
| CN107392123B (en) | Radio frequency fingerprint feature extraction and identification method based on coherent accumulation noise elimination | |
| CN106301631B (en) | A kind of coprime lack sampling frequency spectrum sensing method and its device based on Subspace Decomposition | |
| CN103364646B (en) | Rapid microwave anechoic room antenna far field measurement method | |
| CN111950410B (en) | Online electromagnetic environment monitoring device and method based on radio fingerprint identification | |
| CN111709329A (en) | Unmanned aerial vehicle measurement and control signal high-speed identification method based on deep learning | |
| CN108881096B (en) | Spatial modulation MQAM base station based on phase decision | |
| CN102176779A (en) | Wireless multimedia sensing network video signal adaptive sampling and spectrum allocation method | |
| CN112867010A (en) | Radio frequency fingerprint embedded real-time identification method and system based on convolutional neural network | |
| CN114422049B (en) | Frequency spectrum monitoring big data cleaning method and system based on deep learning detection | |
| CN107491412A (en) | A kind of user power utilization load characteristic extracting method based on experience wavelet transformation | |
| CN202918327U (en) | Automatic classification system for digital modulation signal | |
| CN109085532B (en) | Large-aperture two-dimensional short wave direction finding system | |
| CN112486676A (en) | Data sharing and distributing method and device based on edge calculation | |
| CN104270210A (en) | Soft-decision spectrum sensing method based on compression non-reconstruction | |
| CN103152790B (en) | Two-graded fusion Modulation Identification method based on dependency sub-clustering | |
| CN104181508A (en) | Threatening radar signal detection method based on compressed sensing | |
| CN107341519B (en) | Support vector machine identification optimization method based on multi-resolution analysis | |
| CN101262382A (en) | A data collection system for automatic monitoring in network environment | |
| CN113705646A (en) | Radio frequency fine characteristic information extraction method and system based on semi-supervised meta learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |