CN113194011B - Automatic establishment method and device for radio electromagnetic signal environment - Google Patents
Automatic establishment method and device for radio electromagnetic signal environment Download PDFInfo
- Publication number
- CN113194011B CN113194011B CN202110476115.8A CN202110476115A CN113194011B CN 113194011 B CN113194011 B CN 113194011B CN 202110476115 A CN202110476115 A CN 202110476115A CN 113194011 B CN113194011 B CN 113194011B
- Authority
- CN
- China
- Prior art keywords
- frequency point
- data
- monitoring
- module
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
- H04L43/045—Processing captured monitoring data, e.g. for logfile generation for graphical visualisation of monitoring data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Environmental & Geological Engineering (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The invention provides a method and a device for automatically establishing a radio electromagnetic signal environment, which solve the problem that a uniform and effective radio electromagnetic signal environment image cannot be formed due to the fact that a plurality of monitoring data sources and various data are relatively independent. The invention utilizes big data cluster, radio integration platform and artificial intelligence algorithm, develops and designs a method for automatically forming radio electromagnetic signal environment, and comprehensively realizes the representation of signal environment based on equipment such as electromagnetic environment frequency point information, electromagnetic environment back noise information, electromagnetic environment monitoring signals, electromagnetic environment signal statistics and the like, thereby forming the electromagnetic signal automatic representation of monitoring areas and provinces of the province and better mastering the electromagnetic panoramic information of the areas.
Description
Technical Field
The invention belongs to the technical field of radio signals, and particularly relates to an automatic establishment method and device of a radio electromagnetic signal environment.
Background
With the rapid development of the monitoring technology of the radio monitoring equipment and the construction and the use of the radio management integrated platform in each province, a large amount of monitoring data is generated and accumulated, and the data comprises daily networking monitoring data, offline bin file data and idle-time monitoring tasks. The data can be used for providing electromagnetic environment information support for a certain venue and area in daily analysis work. However, these monitoring data are mutually independent in task dimension, and when supporting external and internal services, service personnel are required to spend a lot of time on monitoring equipment and monitoring tasks, but correct monitoring data cannot be provided, and an electromagnetic signal environment corresponding to the equipment cannot be formed according to these data.
Disclosure of Invention
The present invention is directed to solve the above technical problems, and provides a method and an apparatus for automatically establishing a radio electromagnetic signal environment.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for automatic establishment of a radio electromagnetic signal environment, comprising the steps of:
(1) the monitoring data import module monitors the networked collectors and the bin file uploading equipment through data transmission tools such as a flash Agent and an FTP Agent, and stores the newly uploaded and uploaded data to a big data platform in real time when monitoring the newly uploaded and uploaded data;
(2) the monitoring data wide table fusion module performs frame decoding on the imported data module through a corresponding frame decoding algorithm, data after frame decoding is aggregated into wide tables with corresponding dimensions by taking tasks and time as dimensions, and the wide tables are stored into large data modules such as Hbase and Hive in a specific storage mode;
(3) the monitoring data electromagnetic environment portrait module automatically forms electromagnetic environment portrait of each monitoring facility device based on the aggregated wide table, thereby automatically forming electromagnetic environment portrait of the whole area.
Preferably, in the step (1), the networking monitoring data is stored in a corresponding acquisition machine through the integrated platform, the acquisition machine monitors the directory where the data is located in real time through a transmission software flash Agent, uploads new monitoring data to the big data platform in real time, and simultaneously marks the uploaded data; and for the externally transmitted Bin file, transmitting the externally transmitted Bin file to a big data platform through an FTP Agent, and simultaneously labeling the uploaded file.
Preferably, the monitoring data wide table fusing module fuses the deframing files according to various time dimensions to form wide tables, wherein the wide tables comprise monitoring data wide tables with dimensions of 1 minute, 15 minutes and 1 hour;
the 1 minute monitoring data broad table is to aggregate deframed data at a granularity of 1 minute, and considering that the monitoring data is generally in a T level and the requirement of subsequent aggregate quick query, the part of data is stored in Hbase in the following format:
wherein rowkey is the task ID and the corresponding 1 minute time, and startfreq represents the frequency point of the start of the task; step represents the step diameter of the task; num represents all frequency points included in one period of the task scanning; avg represents the average value of each frequency point in a scanning period, wherein the level values are stored in a form of 1 byte, and the level values of the frequency points are sequentially stored according to the frequency point sequence; min represents the minimum value of each frequency point in a period, wherein the minimum value is stored in a format of 1 byte, and the minimum level values of the frequency points are sequentially stored according to the frequency point sequence; max represents the maximum value of each frequency point in a scanning period, wherein the maximum value is stored in a format of 1 byte, and the maximum level values of the frequency points are sequentially stored according to the frequency point sequence; noise represents the back noise value of each frequency point in a scanning period, wherein the back noise value is stored in a format of 1 byte, and the back noise values of the frequency points are sequentially stored according to the frequency point sequence;
the data is aggregated according to 15 minutes in a monitoring data wide table of 15 minutes, the data of 15 minutes level comprises data storage from time dimension and frequency point dimension, the data is stored in Hbase, and the time dimension storage format is as follows:
forming a row key rowkey by using the task ID and time, and storing various characteristic values in the time period according to bytes, wherein startfreq represents a starting frequency point of the task; step represents the step size of the task; num represents the number of all frequency points in one scanning period; avg represents the average value of each frequency point in the 15-minute time period, the average value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; max represents the maximum value of each frequency point within 15 minutes of cycle time, the maximum value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; noise represents the back noise of each frequency point within 15 minutes of cycle time, the back noise of each frequency point is stored by one byte, and all the frequency points are stored in sequence;
the frequency point dimension storage format is as follows:
the frequency point dimension is used for storing all levels of a certain frequency point within the 15-minute dimension and the frequency of the level, and the task ID and the frequency point are used as row keys for storage;
the hour monitoring data wide table polymerizes data according to 1 hour dimension, the 1 hour dimension data includes data storage from time dimension and frequency point dimension, the data is stored in Hbase, and the time dimension storage format is as follows:
forming a row key rowkey by using the task ID and time, and storing various characteristic values in the time period according to bytes, wherein startfreq represents a starting frequency point of the task; step represents the step size of the task; num represents the number of all frequency points in one period; avg represents the average value of each frequency point in the 1-hour time period, the average value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; max represents the maximum value of each frequency point within 1 hour period, the maximum value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; noise represents the back noise of each frequency point within 1 hour period, the back noise of each frequency point is stored by one byte, and all the frequency points are stored in sequence;
the frequency point dimension storage format is as follows:
the frequency point dimension is used for storing all levels of a certain frequency point within the dimension of 1 hour and the frequency of the level occurrence, and the task ID and the frequency point are used as row keys for storage.
Preferably, the monitoring data wide table fusion module periodically monitors a folder of an original file stored in the big data platform, when a new uploaded file is found, corresponding deframing codes are called according to the type of the uploaded file, and if the uploaded file is networking monitoring data, atomic service deframing codes are called for deframing; and if the file is the external Bin file, calling a Bin file deframing code to deframe.
Preferably, the monitoring data electromagnetic environment portrait module comprises a monitoring facility frequency point information/noise environment portrait module, a monitoring facility monitoring signal portrait module and a monitoring facility electromagnetic environment frequency band/signal statistical portrait module;
the monitoring facility frequency point information/noise environment portrait module is used for counting the frequency point level characteristic value of a certain device according to hours and generating a corresponding noise environment portrait; the monitoring facility frequency point noise environment portrait module is used for storing level information under back noise, and data storage is carried out according to the granularity of equipment in hours;
the monitoring facility monitoring signal image module is used for storing according to aggregated daily monitoring tasks and idle-time tasks by taking hour as granularity, storing data by taking equipment as granularity, and storing the stored data in a big data platform; the monitoring facility monitoring signal image module forms a level matrix of a time frequency point according to a signal extraction algorithm, and forms a gray level image through automatic back noise, so that signals are extracted according to tasks;
merging the extracted signals according to equipment and hour granularity by the electromagnetic environment frequency band/signal statistical portrait of the monitoring facility so as to form an hourly signal distribution portrait of each equipment, and finally, portraying the legal and illegal occupancy rates of the signals of each frequency band according to service frequency band division and a legal signal sample template; the monitoring facility signal statistical portrait module obtains detailed signal information in each hour by taking the hour and the equipment as granularity on the basis of the obtained signals, and performs signal statistical portrait; the electromagnetic environment frequency band portrait monitoring module of the monitoring facility compares the extracted signals with the signal samples, so that the occupancy rates of illegal signals, legal signals and unknown signals of the frequency band are obtained.
Preferably, the step of counting the frequency point level characteristic value of a certain device by the monitoring facility frequency point information/noise environment image module and generating the corresponding noise environment image at the same time comprises:
1) after the data reach a big data platform, a monitoring data wide table fusion module deframing and aggregating the data and writing the deframed and aggregated data into wide tables for 1 minute, 15 minutes and 1 hour respectively;
2) the monitoring facility frequency point information/noise environment portrait module reads the minute-level aggregation wide table and the small-level aggregation wide table, extracts data such as mean value, median value, extreme value, occupancy degree and the like, and writes the data into the monitoring facility monitoring signal portrait module;
3) and the monitoring facility noise environment portrait module reads the minute-level aggregation wide table and the small-level aggregation wide table, extracts various characteristic values and writes the characteristic values into the monitoring facility frequency point noise environment portrait module.
Preferably, the data storage format of the monitoring facility monitoring signal representation module is as follows:
the row key takes the combination of the equipment ID and the time as the ID, and startfreq represents the task starting frequency band of the time period; step represents the step size of the data; freqcontentavg represents the level average value of the monitoring frequency points, each average value is stored by 1 byte, and the average values of a plurality of frequency points are stored in sequence; freqcontent tmax, freqcontent tmin, freqcontent noise, freqcontent cccu and freqcontent tmid respectively represent the maximum value, the minimum value, the automatic back noise value, the occupancy rate value and the median value of the level of the monitoring frequency point, and the storage rule and the average value are the same;
the processing steps of the monitoring facility monitoring signal image module are as follows:
1) the monitoring facility monitoring signal portrait module reads all equipment with tasks in the past hour at the integral point time by setting a timing task, processes one by one according to the granularity of the equipment, and reads the monitoring content of the past hour of all the equipment in the minute-level aggregation wide table and the hour-level aggregation wide table;
2) the monitoring facility monitoring signal portrait module realizes the extraction of the minimum value of the frequency point, mainly extracts from a minute-level aggregation wide table, selects the data of all tasks of a certain device in the past hour, and takes the minimum value from the minimum values as the portrait of the frequency point information of the frequency point in the hour, and the minimum value extraction method of a certain specific frequency point is as follows:
wherein fi represents a certain frequency point, tifi represents the minimum value of the frequency point fi at the moment of i minutes, and the minimum value of all frequency points in the past 1 hour is calculated to obtain the minimum value of the frequency point signal;
3) the monitoring facility monitoring signal portrait module realizes the extraction of the maximum value of the frequency point, mainly extracts from a minute-level aggregation wide table, selects the data of all tasks of a certain device in the past hour, and takes the maximum value from the maximum values as the portrait of the frequency point information of the frequency point in the hour, and the specific maximum value extraction method of the frequency point is as follows:
wherein fi represents a certain frequency point, tifi represents the maximum value of the frequency point fi at the moment of i minutes, and the maximum value of all frequency points in the past 1 hour is solved to obtain the maximum value of the frequency point signal;
4) the monitoring facility monitoring signal portrait module realizes that the average value of the frequency point is mainly extracted from a minute-level aggregation broad list, selects data of all tasks of corresponding equipment in the past hour, and obtains the average value level of a certain frequency point by averaging the average values, wherein the extraction method comprises the following steps:
wherein
Means that the average levels of a certain frequency point are added;
5) the monitoring facility monitoring signal image module realizes that the frequency point back noise is mainly extracted from a minute-level aggregation wide table, automatic back noise values of all tasks of corresponding equipment in the past hour are selected, and the back noise values are obtained by averaging, wherein the calculation formula is as follows:
wherein
Representing the sum of the backnoises in the past hour of a certain frequency point;
6) the monitoring facility monitoring signal portrait module realizes acquisition of the frequency point occupancy in a main minute-level wide table and an hour-level wide table, selects data of all tasks of corresponding equipment in the past hour, and obtains the occupancy of the frequency point by comparing the data with a background noise value, wherein the calculation formula is as follows:
where, Σ n represents the number of times that all levels of the frequency point appear in the past hour in the device, Σ amp represents the number of times that all levels are greater than the back noise value (the average back noise is increased by 5dB), where the back noise value is the average back noise value of step 5;
7) the monitoring facility monitoring signal image module realizes the calculation of the median feature of the frequency points, which is mainly obtained from an hour-level aggregation wide table, selects the data of all tasks in the past hour of the corresponding equipment, obtains all level values and the occurrence times thereof, and then estimates the median, wherein the algorithm is as follows:
preferably, the monitoring facility signal statistical representation module stores the following format:
the processing steps of the monitoring facility signal statistical image module are as follows:
1) the monitoring facility signal statistics and portrait drawing module reads all signals extracted by a certain device in the past hour from the device task signal table at each integral point;
2) the monitoring facility signal statistical image module extracts a current characteristic value from each signal and associates a corresponding service frequency band number;
3) and the monitoring facility signal statistical image module divides the duration of the signal by the time of 1 hour to obtain the time occupancy rate, and finally writes the data.
Preferably, the storage format of the image module for monitoring the electromagnetic environment frequency band of the facility is as follows:
the processing steps of the monitoring facility electromagnetic environment frequency band image module are as follows:
1) the monitoring facility electromagnetic environment frequency band portrait module obtains the occupancy rate of the frequency point by dividing the signals in the frequency band and the duration of the signals by the total time of the frequency band;
2) and comparing the signals of the frequency band portrait module of the electromagnetic environment of the monitoring facility with the marked signals of the signal sample library so as to obtain the occupancy rate of illegal stations, the occupancy rate of legal stations and the occupancy rate of unidentified signals of the frequency band in the time.
The invention also provides an automatic establishment device of the radio electromagnetic signal environment, which comprises a monitoring data import module, a monitoring data wide table fusion module and a monitoring data electromagnetic environment portrait module, wherein computer programs are stored on the monitoring data import module, the monitoring data wide table fusion module and the monitoring data electromagnetic environment portrait module, and when the computer programs are executed by a processor, the automatic establishment method of the radio electromagnetic signal environment is realized.
After the technical scheme is adopted, the invention has the following advantages:
the invention provides a method and a device for automatically establishing a radio electromagnetic signal environment, which solve the problem that a uniform and effective radio electromagnetic signal environment image cannot be formed due to the fact that a plurality of monitoring data sources and various data are relatively independent. The invention utilizes big data cluster, radio integration platform and artificial intelligence algorithm to develop and design a method for automatically forming radio electromagnetic signal environment, comprehensively realizes the portrait of signal environment based on equipment such as electromagnetic environment frequency point information, electromagnetic environment background noise information, electromagnetic environment monitoring signal, electromagnetic environment signal statistics and the like, thereby forming the electromagnetic signal automatic portrait of a full monitoring area and a full province and better mastering the electromagnetic panoramic information of the area.
Drawings
FIG. 1 is an architectural diagram of an automated method of establishing a radio electromagnetic signal environment;
FIG. 2 is an architectural diagram of a surveillance data electromagnetic environment representation module;
FIG. 3 is a flow chart of the processing steps of the monitor facility frequency point information/noise environment profile module;
FIG. 4 is a flow chart of the processing steps of a monitoring facility monitoring signal imaging module;
FIG. 5 is a flow chart of the processing steps of the monitor facility signal statistics imaging module;
FIG. 6 is a flow chart of the processing steps of the monitor noise profile module of the monitoring facility;
FIG. 7 is a flow chart of the processing steps of the monitor facility signal statistics imaging module;
FIG. 8 is an architecture diagram of a module for monitoring facility frequency point information/noise environment representation;
FIG. 9 is an architectural diagram of a frequency band/signal statistical representation of an electromagnetic environment of a monitoring facility.
Detailed Description
The present invention will be described in further detail with reference to the following drawings and specific examples.
As shown in fig. 1-9, a method for automatically establishing a radio electromagnetic signal environment includes the following steps:
(1) the monitoring data import module monitors the networked collectors and the bin file uploading equipment through data transmission tools such as a flash Agent and an FTP Agent, and stores the newly uploaded and uploaded data to a big data platform in real time when monitoring the newly uploaded and uploaded data;
(2) the monitoring data wide table fusion module performs frame decoding on the imported data module through a corresponding frame decoding algorithm, data after frame decoding is aggregated into wide tables with corresponding dimensions by taking tasks and time as dimensions, and the wide tables are stored into large data modules such as Hbase and Hive in a specific storage mode;
(3) the monitoring data electromagnetic environment portrait module automatically forms electromagnetic environment portrait of each monitoring facility device based on the aggregated wide table, thereby automatically forming electromagnetic environment portrait of the whole area.
In the step (1), the networking monitoring data is stored in a corresponding acquisition machine through an integrated platform, the acquisition machine monitors the directory where the data is located in real time through a transmission software flash Agent, uploads new monitoring data to a big data platform in real time, and simultaneously marks the uploaded data; and for the externally transmitted Bin file, transmitting the externally transmitted Bin file to a big data platform through an FTP Agent, and simultaneously labeling the uploaded file.
The monitoring data wide table fusion module fuses the unframed files according to various time dimensions to form wide tables, wherein the wide tables comprise monitoring data wide tables with dimensions of 1 minute, 15 minutes and 1 hour;
the 1 minute monitoring data broad table is to aggregate deframed data according to granularity of 1 minute, and considering that the monitoring data is generally in a T level and the requirement of subsequent aggregation quick query, the part of data is stored in Hbase, and the storage format is as follows:
wherein rowkey is the task ID and the corresponding 1 minute time, and startfreq represents the frequency point of the start of the task; step represents the step diameter of the task; num represents all frequency points included in one period of the task scanning; avg represents the average value of each frequency point in a scanning period, wherein the level value is stored in a form of 1 byte, and the level values of the frequency points are sequentially stored according to the frequency point sequence; min represents the minimum value of each frequency point in a period, wherein the minimum value is stored in a format of 1 byte, and the minimum level values of the frequency points are sequentially stored according to the frequency point sequence; max represents the maximum value of each frequency point in a scanning period, wherein the maximum value is stored in a format of 1 byte, and the maximum level values of the frequency points are sequentially stored according to the frequency point sequence; noise represents the back noise value of each frequency point in a scanning period, wherein the back noise value is stored in a format of 1 byte, and the back noise values of the frequency points are sequentially stored according to the frequency point sequence;
the broad table of 15 minutes monitoring data aggregates the data according to 15 minutes, the data of 15 minutes level includes the storage of the data from time dimension and frequency point dimension, the data is stored in Hbase, and the time dimension storage format is as follows:
forming a row key rowkey by the task ID and time, and storing various characteristic values in the time period according to bytes, wherein startfreq represents a starting frequency point of the task; step represents the step size of the task; num represents the number of all frequency points in one scanning period; avg represents the average value of each frequency point in the 15-minute time period, the average value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; max represents the maximum value of each frequency point within 15 minutes of cycle time, the maximum value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; noise represents the back noise of each frequency point within 15 minutes of cycle time, the back noise of each frequency point is stored by one byte, and all frequency points are stored in sequence;
the frequency point dimension storage format is as follows:
the frequency point dimension is used for storing all levels of a certain frequency point within the 15-minute dimension and the frequency of the level, and the task ID and the frequency point are used as row keys for storage;
the hour monitoring data wide table polymerizes data according to 1 hour dimension, the 1 hour dimension data includes data storage from time dimension and frequency point dimension, the data is stored in Hbase, and the time dimension storage format is as follows:
forming a row key rowkey by using the task ID and time, and storing various characteristic values in the time period according to bytes, wherein startfreq represents a starting frequency point of the task; step represents the step size of the task; num represents the number of all frequency points in one scanning period; avg represents the average value of each frequency point in the 1-hour time period, the average value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; max represents the maximum value of each frequency point within 1 hour period, the maximum value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; noise represents the back noise of each frequency point within 1 hour period, the back noise of each frequency point is stored by one byte, and all the frequency points are stored in sequence;
the frequency point dimension storage format is as follows:
the frequency point dimension is used for storing all levels of a certain frequency point within the dimension of 1 hour and the frequency of the level occurrence, and the task ID and the frequency point are used as row keys for storage.
The monitoring data wide table fusion module periodically monitors a folder of an original file stored in the big data platform, when a new uploaded file is found, corresponding deframing codes are called according to the type of the uploaded file, and if the uploaded file is networking monitoring data, atomic service deframing codes are called for deframing; and if the file is the external Bin file, calling a Bin file deframing code to deframe.
The monitoring data electromagnetic environment portrait module comprises a monitoring facility frequency point information/noise environment portrait module, a monitoring facility monitoring signal portrait module and a monitoring facility electromagnetic environment frequency band/signal statistical portrait module;
the monitoring facility frequency point information/noise environment portrait module is used for counting the frequency point level characteristic value of a certain device according to hours and generating a corresponding noise environment portrait; the monitoring facility frequency point noise environment portrait module is used for storing level information under back noise, and data storage is carried out according to the granularity of equipment in hours;
the monitoring facility monitoring signal portrait module is used for storing hourly granularity according to aggregated daily monitoring tasks and idle-time tasks, storing data by taking equipment as granularity, and storing the stored data in a big data platform; the monitoring facility monitoring signal image module forms a level matrix of a time frequency point according to a signal extraction algorithm, and forms a gray level image through automatic back noise, so that signals are extracted according to tasks;
merging the extracted signals according to equipment and hour granularity by the monitoring facility electromagnetic environment frequency band/signal statistical portrait so as to form hourly signal distribution portrait of each equipment, and finally portrait the legal and illegal occupancy rates of the signals of each frequency band according to business frequency band division and a legal signal sample template; the monitoring facility signal statistical portrait module obtains detailed signal information in each hour by taking the hour and the equipment as granularity on the basis of the obtained signals, and performs signal statistical portrait; the electromagnetic environment frequency band portrait monitoring module of the monitoring facility compares the extracted signals with the signal samples, so that the occupancy rates of illegal signals, legal signals and unknown signals of the frequency band are obtained.
The monitoring facility frequency point information/noise environment portrait module counts the frequency point level characteristic value of a certain device, and the step of generating the corresponding noise environment portrait comprises the following steps:
1) after the data reach a big data platform, a monitoring data wide table fusion module unframes and aggregates the data and respectively writes the data into wide tables for 1 minute, 15 minutes and 1 hour;
2) the monitoring facility frequency point information/noise environment portrait module reads the minute-level aggregation wide table and the small-level aggregation wide table, extracts data such as mean value, median value, extreme value, occupancy degree and the like, and writes the data into the monitoring facility monitoring signal portrait module;
3) and the monitoring facility noise environment portrait module reads the minute-level aggregation wide table and the small-level aggregation wide table, extracts various characteristic values and writes the characteristic values into the monitoring facility frequency point noise environment portrait module.
The data storage format of the monitoring facility monitoring signal portrait module is as follows:
the row key takes the combination of the equipment ID and the time as the ID, and startfreq represents the task starting frequency band of the time period; step represents the step size of the data; freqcontentavg represents the level average value of the monitoring frequency points, each average value is stored by 1 byte, and the average values of a plurality of frequency points are stored in sequence; freqcontent tmax, freqcontent tmin, freqcontent noise, freqcontent cccu and freqcontent tmid respectively represent the maximum value, the minimum value, the automatic back noise value, the occupancy rate value and the median value of the level of the monitoring frequency point, and the storage rule and the average value are the same;
the processing steps of the monitoring facility monitoring signal image module are as follows:
1) the monitoring facility monitoring signal portrait module reads all equipment with tasks in the past hour at the integral point time by setting a timing task, processes the equipment one by one according to the granularity of the equipment, and reads the monitoring content of the past hour of all the equipment in the minute-level aggregation wide table and the hour-level aggregation wide table;
2) the monitoring facility monitoring signal portrait module realizes the extraction of the minimum value of the frequency point, mainly extracts from a minute-level aggregation wide table, selects the data of all tasks of a certain device in the past hour, and takes the minimum value from the minimum values as the portrait of the frequency point information of the frequency point in the hour, and the minimum value extraction method of a certain specific frequency point is as follows:
wherein fi represents a certain frequency point, tifi represents the minimum value of the frequency point fi at the moment of i minutes, and the minimum value of all frequency points in the past 1 hour is calculated to obtain the minimum value of the frequency point signal;
3) the monitoring facility monitoring signal portrait module realizes the extraction of the maximum value of the frequency point, mainly extracts from a minute-level aggregation wide table, selects the data of all tasks of a certain device in the past hour, and takes the maximum value from the maximum values as the portrait of the frequency point information of the frequency point in the hour, and the specific maximum value extraction method of the frequency point is as follows:
wherein fi represents a certain frequency point, tifi represents the maximum value of the frequency point fi at the moment of i minutes, and the maximum value of all frequency points in the past 1 hour is solved to obtain the maximum value of the frequency point signal;
4) the monitoring facility monitoring signal portrait module realizes that the average value of the frequency point is mainly extracted from a minute-level aggregation broad list, selects data of all tasks of corresponding equipment in the past hour, and obtains the average value level of a certain frequency point by averaging the average values, wherein the extraction method comprises the following steps:
wherein
Means that the average levels of a certain frequency point are added;
5) the monitoring facility monitoring signal image module realizes that the frequency point back noise is mainly extracted from a minute-level aggregation wide table, selects the automatic back noise values of all tasks of corresponding equipment in the past hour, and obtains the back noise values by averaging, wherein the calculation formula is as follows:
wherein
Representing the sum of the backnoises in the past hour of a certain frequency point;
6) the monitoring facility monitoring signal portrait module realizes acquisition of the frequency point occupancy in a main minute-level wide table and an hour-level wide table, selects data of all tasks of corresponding equipment in the past hour, and obtains the occupancy of the frequency point by comparing the data with a background noise value, wherein the calculation formula is as follows:
where, Σ n represents the number of times that all levels of the frequency point appear in the past hour in the device, Σ amp represents the number of times that all levels are greater than the back noise value (the average back noise is increased by 5dB), where the back noise value is the average back noise value of step 5;
7) the monitoring facility monitoring signal image module realizes the calculation of the median feature of the frequency points, which is mainly obtained from an hour-level aggregation wide table, selects the data of all tasks in the past hour of the corresponding equipment, obtains all level values and the occurrence times thereof, and then estimates the median, wherein the algorithm is as follows:
the monitoring facility signal statistics portrait module is stored in the format as follows:
the processing steps of the monitoring facility signal statistical image module are as follows:
1) the monitoring facility signal statistics and portrait drawing module reads all signals extracted by a certain device in the past hour from the device task signal table at each integral point;
2) the monitoring facility signal statistical image module extracts a current characteristic value from each signal and associates a corresponding service frequency band number;
3) and the monitoring facility signal statistical image module divides the duration of the signal by the time of 1 hour to obtain the time occupancy rate, and finally writes the data.
The storage format of the monitoring facility electromagnetic environment frequency band portrait module is as follows:
the processing steps of the monitoring facility electromagnetic environment frequency band portrait module are as follows:
1) the monitoring facility electromagnetic environment frequency band portrait module obtains the occupancy rate of the frequency point by dividing the signals in the frequency band and the duration of the signals by the total time of the frequency band;
2) and comparing the signals of the frequency band portrait module of the electromagnetic environment of the monitoring facility with the marked signals of the signal sample library so as to obtain the occupancy rate of illegal stations, the occupancy rate of legal stations and the occupancy rate of unidentified signals of the frequency band in the time.
The invention also provides an automatic establishing device of the radio electromagnetic signal environment, which comprises a monitoring data importing module, a monitoring data wide table fusing module and a monitoring data electromagnetic environment portrait module, wherein computer programs are stored on the monitoring data importing module, the monitoring data wide table fusing module and the monitoring data electromagnetic environment portrait module, and when the computer programs are executed by a processor, the automatic establishing method of the radio electromagnetic signal environment is realized.
Other embodiments of the present invention than the preferred embodiments described above will be apparent to those skilled in the art from the present invention, and various changes and modifications can be made therein without departing from the spirit of the present invention as defined in the appended claims.
Claims (5)
1. A method for automatically establishing a radio electromagnetic signal environment, comprising the steps of:
(1) the monitoring data import module monitors the networked collectors and the bin file uploading equipment through a Flume Agent and FTP Agent data transmission tool, and stores the newly uploaded and uploaded data to a big data platform in real time when monitoring the newly uploaded and uploaded data;
(2) the monitoring data wide table fusion module performs frame decoding on the imported data module through a corresponding frame decoding algorithm, data after frame decoding is aggregated into wide tables with corresponding dimensions by taking tasks and time as dimensions, and the wide tables are stored into the Hbase and Hive big data modules in a specific storage mode;
(3) the monitoring data electromagnetic environment portrait module automatically forms electromagnetic environment portrait of each monitoring facility device based on the aggregated wide table, thereby automatically forming electromagnetic environment portrait of the whole area;
the monitoring data wide table fusion module fuses the unframed files according to various time dimensions to form wide tables, wherein the wide tables comprise monitoring data wide tables with dimensions of 1 minute, 15 minutes and 1 hour;
the 1 minute monitoring data broad table is to aggregate deframed data at a granularity of 1 minute, and considering that the monitoring data is of a T level and the requirement of subsequent aggregate quick query, the part of data is stored in Hbase, and the storage format is as follows:
wherein rowkey is the task ID and the corresponding 1 minute time, and startfreq represents the frequency point of the start of the task; step represents the step diameter of the task; num represents all frequency points included in one period of the task scanning; avg represents the average value of each frequency point in a scanning period, wherein the level values are stored in a form of 1 byte, and the level values of the frequency points are sequentially stored according to the frequency point sequence; min represents the minimum value of each frequency point in a period, wherein the minimum value is stored in a format of 1 byte, and the minimum level values of the frequency points are sequentially stored according to the frequency point sequence; max represents the maximum value of each frequency point in a scanning period, wherein the maximum value is stored in a format of 1 byte, and the maximum level values of the frequency points are sequentially stored according to the frequency point sequence; noise represents the back noise value of each frequency point in a scanning period, wherein the back noise value is stored in a format of 1 byte, and the back noise values of the frequency points are sequentially stored according to the frequency point sequence;
the data is aggregated according to 15 minutes in a monitoring data wide table of 15 minutes, the data of 15 minutes level comprises data storage from time dimension and frequency point dimension, the data is stored in Hbase, and the time dimension storage format is as follows:
forming a row key rowkey by the task ID and time, and storing various characteristic values in the time period according to bytes, wherein startfreq represents a starting frequency point of the task; step represents the step size of the task; num represents the number of all frequency points in one scanning period; avg represents the average value of each frequency point in the 15-minute time period, the average value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; max represents the maximum value of each frequency point within 15 minutes of cycle time, the maximum value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; noise represents the back noise of each frequency point within 15 minutes of cycle time, the back noise of each frequency point is stored by one byte, and all frequency points are stored in sequence;
the frequency point dimension storage format is as follows:
the frequency point dimension is used for storing all levels of a certain frequency point within the 15-minute dimension and the frequency of the level, and the task ID and the frequency point are used as row keys for storage;
the hour monitoring data wide table polymerizes data according to 1 hour dimension, the 1 hour dimension data includes data storage from time dimension and frequency point dimension, the data is stored in Hbase, and the time dimension storage format is as follows:
forming a row key rowkey by using the task ID and time, and storing various characteristic values in the time period according to bytes, wherein startfreq represents a starting frequency point of the task; step represents the step size of the task; num represents the number of all frequency points in one period; avg represents the average value of each frequency point in the 1-hour time period, the average value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; max represents the maximum value of each frequency point within 1 hour period, the maximum value of each frequency point is stored by one byte, and all the frequency points are stored in sequence; noise represents the back noise of each frequency point within 1 hour period, the back noise of each frequency point is stored by one byte, and all frequency points are stored in sequence;
the frequency point dimension storage format is as follows:
the frequency point dimension is used for storing all levels of a certain frequency point within the dimension of 1 hour and the occurrence times of the levels, and the task ID and the frequency point are used as row keys for storage;
the monitoring data electromagnetic environment portrait module comprises a monitoring facility frequency point information/noise environment portrait module, a monitoring facility monitoring signal portrait module and a monitoring facility electromagnetic environment frequency band/signal statistical portrait module;
the monitoring facility frequency point information/noise environment portrait module is used for counting the frequency point level characteristic value of a certain device according to hours and generating a corresponding noise environment portrait; the monitoring facility frequency point noise environment portrait module is used for storing level information under back noise, and storing data according to the granularity of equipment in hours;
the monitoring facility monitoring signal image module is used for storing according to aggregated daily monitoring tasks and idle-time tasks by taking hour as granularity, storing data by taking equipment as granularity, and storing the stored data in a big data platform; the monitoring facility monitoring signal image module forms a level matrix of a time frequency point according to a signal extraction algorithm, and forms a gray level image through automatic back noise, so that signals are extracted according to tasks;
merging the extracted signals according to equipment and hour granularity by the monitoring facility electromagnetic environment frequency band/signal statistical portrait so as to form hourly signal distribution portrait of each equipment, and finally portrait the legal and illegal occupancy rates of the signals of each frequency band according to business frequency band division and a legal signal sample template; the monitoring facility signal statistical portrait module obtains detailed signal information in each hour by taking the hour and the equipment as granularity on the basis of the obtained signals, and performs signal statistical portrait; the monitoring facility electromagnetic environment frequency band portrait module obtains the occupancy rates of illegal signals, legal signals and unidentified signals of the frequency band by comparing the extracted signals with signal samples;
the data storage format of the monitoring facility monitoring signal portrait module is as follows:
the row key takes the combination of the equipment ID and the time as the ID, and startfreq represents the task starting frequency band of the time period; step represents the step size of the data; freqcontentavg represents the level average value of the monitoring frequency points, each average value is stored by 1 byte, and the average values of a plurality of frequency points are stored in sequence; freqcontent tmax, freqcontent tmin, freqcontent noise, freqcontent cccu and freqcontent tmid respectively represent the maximum value, the minimum value, the automatic back noise value, the occupancy rate value and the median value of the level of the monitoring frequency point, and the storage rule and the average value are the same;
the processing steps of the monitoring facility monitoring signal image module are as follows:
1) the monitoring facility monitoring signal portrait module reads all equipment with tasks in the past hour at the integral point time by setting a timing task, processes one by one according to the granularity of the equipment, and reads the monitoring content of the past hour of all the equipment in the minute-level aggregation wide table and the hour-level aggregation wide table;
2) the monitoring facility monitoring signal portrait module extracts the minimum value of the frequency point from the minute-level aggregation wide table, selects data of all tasks of a certain device in the past hour, and takes the minimum value from the minimum values as the portrait of the frequency point information of the frequency point in the hour, and the minimum value extraction method of a specific frequency point is as follows:
min(fi)＝min(t1fi，t2if，t3fi...t60fi)
wherein fi represents a certain frequency point, tifi represents the minimum value of the frequency point fi at the moment of i minutes, and the minimum value of all frequency points in the past 1 hour is calculated to obtain the minimum value of the frequency point signal;
3) the monitoring facility monitoring signal portrait module extracts the maximum value of the frequency point from the minute-level aggregation wide table, selects data of all tasks of a certain device in the past hour, and takes the maximum value from the maximum values as a portrait of the frequency point information of the frequency point in the hour, and the maximum value extraction method of a specific frequency point is as follows:
max(fi)＝max(t1fi，t2if，t3fi...t60fi)
wherein fi represents a certain frequency point, tifi represents the maximum value of the frequency point fi at the moment of i minutes, and the maximum value of all frequency points in the past 1 hour is solved to obtain the maximum value of the frequency point signal;
4) the monitoring facility monitoring signal portrait module realizes the extraction of the average value of the frequency points from the minute-level aggregation broad table, selects the data of all tasks of the corresponding equipment in the past hour, and obtains the average value level of a certain frequency point by averaging the average values, and the extraction method comprises the following steps:
wherein
Means that the average levels of a certain frequency point are added;
5) the monitoring facility monitoring signal image module extracts frequency point back noise from a minute-level aggregation wide table, selects automatic back noise values of all tasks of corresponding equipment in the past hour, and obtains the back noise values by averaging, wherein the calculation formula is as follows:
wherein
To representAdding back noises of a certain frequency point within one hour;
6) the monitoring facility monitoring signal portrait module realizes that the occupancy rate of the frequency point is obtained from a minute-level wide table and an hour-level wide table, selects data of all tasks of corresponding equipment in the past hour, and obtains the occupancy rate of the frequency point by comparing the data with a background noise value, wherein the calculation formula is as follows:
wherein, sigma n represents the number of times of all the appearing levels of the frequency point in the past hour of the device, sigma amp represents the number of times of all the levels larger than the back noise value, wherein the back noise value is the average back noise value of the step 5;
7) the monitoring facility monitoring signal image module realizes the calculation of the median feature of the frequency points and obtains the median feature from the hour-level aggregation broad table, selects the data of all tasks in the past hour of the corresponding equipment, obtains all level values and the occurrence times thereof, and estimates the median, wherein the algorithm is as follows:
the monitoring facility signal statistics portrait module is stored in the format as follows:
the processing steps of the monitoring facility signal statistical image module are as follows:
1) the monitoring facility signal statistics and portrait drawing module reads all signals extracted by a certain device in the past hour from the device task signal table at each integral point;
2) the monitoring facility signal statistical image module extracts a current characteristic value from each signal and associates a corresponding service frequency band number;
3) the monitoring facility signal statistics portrait module divides the duration of the signal by 1 hour to obtain the time occupancy rate, and finally writes data in;
the storage format of the monitoring facility electromagnetic environment frequency band portrait module is as follows:
the processing steps of the monitoring facility electromagnetic environment frequency band image module are as follows:
1) the monitoring facility electromagnetic environment frequency band portrait module obtains the occupancy rate of the frequency point by dividing the signals in the frequency band and the duration of the signals by the total time of the frequency band;
2) and comparing the signals of the frequency band portrait module of the electromagnetic environment of the monitoring facility with the marked signals of the signal sample library so as to obtain the occupancy rate of illegal stations, the occupancy rate of legal stations and the occupancy rate of unidentified signals of the frequency band in the time.
2. The method for automatically establishing the radio electromagnetic signal environment according to claim 1, wherein in the step (1), the networking monitoring data is stored in the corresponding acquisition machine through the integrated platform, the acquisition machine monitors the directory of the data in real time through the flash Agent, uploads the new monitoring data to the big data platform in real time, and marks the uploaded data at the same time; and for the externally transmitted Bin file, transmitting the externally transmitted Bin file to a big data platform through an FTP Agent, and simultaneously labeling the uploaded file.
3. The method for automatically creating an environment for radio-electromagnetic signals according to claim 1, wherein the monitoring data wide table fusion module periodically monitors folders of original files stored in the big data platform, when a new uploaded file is found, a corresponding deframing code is called according to the type of the uploaded file, and if the uploaded file is networking monitoring data, an atomic service deframing code is called for deframing; and if the file is the external Bin file, calling a Bin file deframing code to deframe.
4. The method for automatically establishing an environment for radio electromagnetic signals according to claim 1, wherein said monitoring facility frequency point information/noise environment picture module counts a frequency point level characteristic value of a certain device, and generates a corresponding noise environment picture thereof comprises:
1) after the data reach a big data platform, a monitoring data wide table fusion module deframing and aggregating the data and writing the deframed and aggregated data into wide tables for 1 minute, 15 minutes and 1 hour respectively;
2) the monitoring facility frequency point information/noise environment portrait module reads the minute-level aggregation wide table and the small-level aggregation wide table, extracts mean value, median value, extreme value and occupancy rate data and writes the data into the monitoring facility monitoring signal portrait module;
3) and the monitoring facility noise environment portrait module reads the minute-level aggregation wide table and the small-level aggregation wide table, extracts various characteristic values and writes the characteristic values into the monitoring facility frequency point noise environment portrait module.
5. An automatic building device of a radio electromagnetic signal environment, comprising a monitoring data importing module, a monitoring data wide table fusing module, and a monitoring data electromagnetic environment portrait module, wherein the monitoring data importing module, the monitoring data wide table fusing module, and the monitoring data electromagnetic environment portrait module have computer programs stored thereon, and when the computer programs are executed by a processor, the automatic building method of a radio electromagnetic signal environment according to any one of claims 1 to 4 is realized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110476115.8A CN113194011B (en) | 2021-04-29 | 2021-04-29 | Automatic establishment method and device for radio electromagnetic signal environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110476115.8A CN113194011B (en) | 2021-04-29 | 2021-04-29 | Automatic establishment method and device for radio electromagnetic signal environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113194011A CN113194011A (en) | 2021-07-30 |
| CN113194011B true CN113194011B (en) | 2022-08-26 |
Family
ID=76980677
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110476115.8A Active CN113194011B (en) | 2021-04-29 | 2021-04-29 | Automatic establishment method and device for radio electromagnetic signal environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113194011B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110350993A (en) * | 2019-08-19 | 2019-10-18 | 浙江原初数据科技有限公司 | A kind of black broadcast automatic discovering method based on networking monitoring under big data scene |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9265086B2 (en) * | 2013-10-10 | 2016-02-16 | Qualcomm Incorporated | Addressing radio link failures in wireless communication systems |
| CN106353603B (en) * | 2016-08-31 | 2018-11-06 | 成都九华圆通科技发展有限公司 | A kind of intelligent cloud monitoring method for radio |
| CN110943882B (en) * | 2019-11-12 | 2022-03-01 | 浙江原初数据科技有限公司 | Real-time black broadcast identification system based on networking monitoring and identification method thereof |
| CN112257529B (en) * | 2020-10-12 | 2023-11-28 | 浙江原初数据科技有限公司 | Automatic extraction system and method for radio signal characteristics |
| CN112527805B (en) * | 2021-01-26 | 2022-04-19 | 浙江原初数据科技有限公司 | Real-time playback method of radio mass monitoring data |
-
2021
- 2021-04-29 CN CN202110476115.8A patent/CN113194011B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110350993A (en) * | 2019-08-19 | 2019-10-18 | 浙江原初数据科技有限公司 | A kind of black broadcast automatic discovering method based on networking monitoring under big data scene |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113194011A (en) | 2021-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107634848B (en) | System and method for collecting and analyzing network equipment information | |
| CN103686237B (en) | Recommend the method and system of video resource | |
| CN109391629B (en) | Data processing method of rail transit comprehensive monitoring system | |
| CN101355686A (en) | Method and system for statistic of audience rating | |
| CN112884159B (en) | Model updating system, model updating method and related equipment | |
| CN110765134A (en) | File establishing method, equipment and storage medium | |
| CN113034010A (en) | Intelligent government affair request processing system based on cloud computing | |
| CN112817753A (en) | Task processing method and device, storage medium and electronic device | |
| CN114757639A (en) | Data processing method, device, equipment and storage medium | |
| CN113283824A (en) | Comprehensive management method and system for intelligent park data | |
| CN113194011B (en) | Automatic establishment method and device for radio electromagnetic signal environment | |
| CN113127472B (en) | Method and system for real-time deduplication statistics of number of drivers with large reporting amount | |
| WO2024066720A1 (en) | Indicator threshold determination method and apparatus, storage medium, and electronic apparatus | |
| CN117479218A (en) | Radio spectrum data compression method based on pattern recognition | |
| WO2022001480A1 (en) | Popular application identification method, network system, network device and storage medium | |
| CN113918303B (en) | Multitasking distributed scheduling system based on micro-service | |
| CN112613396B (en) | Task emergency degree processing method and system | |
| CN114266075A (en) | Internet of things data evidence storing method based on block chain | |
| CN112015726A (en) | User activity prediction method, system and readable storage medium | |
| CN112364056B (en) | Automatic fluorescence map playing system and method applied to real-time big data scene | |
| CN116633673B (en) | Data safety transmission system for comprehensive energy platform | |
| CN116150482B (en) | Distributed message pushing system based on big data cloud platform | |
| CN114138498B (en) | Rapid processing method for building energy consumption | |
| CN118171329B (en) | Real estate registration monitoring equipment data occurrence time recording method | |
| CN118413452B (en) | Method and device for discovering Torr key nodes based on time sequence network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Chen Wei Inventor after: Ma Gaofeng Inventor after: Wu Jingbo Inventor after: Wang Xinzhi Inventor after: Ma Jie Inventor before: Chen Wei Inventor before: Ma Gaofeng Inventor before: Wu Jingbo Inventor before: Wang Xinzhi |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |