CN113609081B - Method for storing radio ultrashort wave frequency band monitoring sweep frequency basic data - Google Patents
Method for storing radio ultrashort wave frequency band monitoring sweep frequency basic data Download PDFInfo
- Publication number
- CN113609081B CN113609081B CN202110854127.XA CN202110854127A CN113609081B CN 113609081 B CN113609081 B CN 113609081B CN 202110854127 A CN202110854127 A CN 202110854127A CN 113609081 B CN113609081 B CN 113609081B
- Authority
- CN
- China
- Prior art keywords
- data
- monitoring
- sweep frequency
- basic data
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention provides a method for storing radio ultrashort wave frequency band monitoring sweep frequency basic data, which utilizes a distributed storage system, a radio integrated platform and a structured database development design to comprehensively realize long-time effective storage of radio ultrashort wave monitoring sweep frequency basic data and accurate positioning reading of monitoring basic data, thereby forming a data asset foundation of full-province or ground city monitoring data and guiding a monitoring center to better manage and use ultrashort wave monitoring data results. The method solves the problems of effective storage and rapid extraction of mass sweep frequency monitoring data after networking monitoring.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to a method for storing radio ultra-short wave frequency band monitoring sweep frequency basic data.
Background
In recent years, with the wide application of new radio equipment and new technology and the construction of radio integrated platforms in each province, each frequency band scan data collected in each province is stored in a radio monitoring data platform in each province. Currently, a province monitoring station has about 150 fixed monitoring stations and 20 monitoring vehicles, and the generated data volume is huge, and the monitoring stations have about 200T of monitoring data in one year. The data are stored for 3-5 years according to the related national requirements, and meanwhile, the requirements of random extraction and playback of the data in the monitoring center business are met.
In order to meet the requirements for the storage and use of raw monitoring data, the following methods are generally adopted at present: 1. the original monitoring data are stored on the industrial personal computers corresponding to the devices, and the method can be used for fast data playback and sampling, but is limited by small storage and networking-free performance of the industrial personal computers, and the data cannot be stored in a large scale and can be called by other applications; 2. dividing the monitoring data into 255M-sized files, and storing the files in a distributed storage system; meanwhile, each file is built with a corresponding index, and the index is stored in a structured database, so that the method can realize the storage of a large amount of data and the random and rapid data query of the data among various applications, but because the current monitoring data is too huge, the index of each year of data files can reach over 300 ten thousand index entries, and the method can become a great burden and bottleneck for the structured database.
Therefore, in the process of establishing the radio integrated platform and the radio monitoring data platform, effective storage of massive monitoring data is required to be considered, the storage requirement of the data validity of each level of monitoring center is met, and meanwhile, the requirements of external each level of authorities and application on random reading of the data are met.
Disclosure of Invention
The invention aims to solve the technical problems and provides a method for storing frequency sweep basic data for monitoring a radio ultrashort wave frequency band.
In order to achieve the above purpose, the invention adopts the following technical scheme:
the data storage method is based on a radio ultrashort wave frequency band monitoring frequency sweep basic data storage system, and the radio ultrashort wave frequency band monitoring frequency sweep basic data storage system comprises a frequency sweep data import module, a frequency sweep basic data storage module, a frequency sweep basic data configuration module and a frequency sweep basic data reading module; the sweep frequency basic data importing module is used for importing networking sweep frequency data and drive test bin file data into a radio ultrashort wave monitoring sweep frequency basic data storage system, and the data are uploaded to a radio monitoring data platform through a data transmission tool and stored; the frequency sweep basic data storage module comprises a normal frequency sweep basic data storage module and a compressed frequency sweep basic data storage module, wherein the normal frequency sweep monitoring data storage module is used for storing the latest uploaded monitoring data, and the compressed frequency sweep basic data storage module is used for storing long-time unused frequency sweep data; the sweep frequency basic data configuration module is used for realizing the configuration management function of stored sweep frequency data, and comprises normal sweep frequency data management, compressed sweep frequency data management and data compression management; the sweep frequency basic data reading module is used for realizing the rapid reading function of the original monitoring data, and comprises the reading of the original monitoring file and the reading of the basic monitoring data stream;
the data storage method comprises the following steps:
step 1, uploading sweep frequency monitoring data of networking equipment to a radio monitoring data platform in real time or quasi-real time through a sweep frequency data importing module, and writing corresponding task related information into a sweep frequency basic data configuration module;
step 2, the sweep frequency basic data storage module is based on a radio monitoring data platform, and after receiving the uploaded sweep frequency data, the sweep frequency basic data storage module determines the number of backup data and the compression type of the data according to the data of the sweep frequency basic data configuration table to store the data;
step 3, the sweep frequency basic data configuration module stores, compresses and extracts various information of the recorded uploading basic monitoring data;
and 4, after the sweep frequency basic data reading module receives the requirement of external query data, determining the monitoring basic data through a positioning algorithm by querying corresponding table items in the sweep frequency basic data configuration module, and returning to related applications.
Preferably, in the step 1, the networking monitoring platform uploads the data to the radio monitoring data platform through a data acquisition machine, and the data acquisition machine uploads the data to the radio monitoring data platform through relevant tools such as transmission software Flume, FTP and the like.
Preferably, in the step 2, the types of data stored in the sweep frequency basic data storage module include original monitoring sweep frequency basic data and compressed monitoring sweep frequency basic data, the monitoring basic data storage module compresses according to idle time of task data, the monitoring basic data storage module reads last editing time of the task in the task information table in the monitoring configuration module, judges whether the time and the current time exceed one year, if so, the task is compressed, after compression, the monitoring basic data only retains one piece of data, and meanwhile, a compressed mark is marked in a task file processing field.
Preferably, the sweep frequency basic data storage module receives the sweep frequency basic original data and stores the sweep frequency basic original data to the radio data platform, the sweep frequency monitoring basic data is stored in the radio data platform according to the task dimension, and the sweep frequency basic data storage module also updates the data to the task information table stored in the sweep frequency data configuration module.
Preferably, the data updated by the sweep frequency basic data storage module is used for butting task information with the radio data platform, periodically inquiring the task ID of the uploaded data from the radio data platform and storing the corresponding information into a task information storage table; meanwhile, the sweep frequency basic data storage module automatically updates the saved task information table through the uploaded data, the sweep frequency basic data storage module compares the minimum time in each task text frame with the text starting time in the saved task information table, if the minimum time in each task text frame is smaller than the text starting time in the saved task information table, and meanwhile, compares the maximum time in each task text frame with the text ending time in the saved task information table, and if the maximum time in each task text frame is larger than the text ending time in the saved task information table, the sweep frequency basic data storage module updates; the text size of the corresponding task is updated every time the sweep frequency basic data storage module receives one text; and the sweep frequency basic data storage module selects and changes the number of backup parts of the stored original data through the backup parameters in the sweep frequency basic data configuration module.
Preferably, in the step 3, the sweep frequency basic data configuration data stored by the sweep frequency basic data configuration module is stored in a data set form, and the data set includes a corresponding save task information table and a data backup configuration table.
Preferably, in the step 4, the sweep frequency basic data reading module provides a quick and accurate positioning return of the original monitoring sweep frequency file, and supports a quick return of the original monitoring file and a byte return of the socket stream.
Preferably, in the quick original monitoring file return, the sweep frequency basic data reading module returns a corresponding monitoring file according to the time or the time period of data extraction, monitoring equipment and a monitoring task, and query parameters are as follows: < query start time, query end time, task ID >.
Preferably, after the sweep frequency basic data reading module receives the query parameters, specific file reading data is positioned according to the starting time and the task ID, and the steps are as follows:
step 1, a sweep frequency basic data reading module inquires and stores a task information table according to a task ID, wherein inquired fields comprise the starting time of a task text, the ending time of the text, the size of monitoring task data and compression state parameters of the monitoring task; if the compression state parameter of the monitoring task is displayed to be compressed, the data extraction module decompresses the monitoring task and then continues to execute the step 1;
step 2, the sweep frequency basic data reading module compares and acquires the time difference delta T between the duration delta T of the monitoring task and the inquiry starting time and the text starting time, and the calculation method is as follows:
δT=T2-T1，δt=t1-T1；
wherein T2 represents the end time of the text of the monitoring task, T1 represents the start time of the text of the monitoring task, and T1 represents the start time of the query;
if δt is less than 0, the reading module provides the saved monitoring text with the monitoring original task data numbered 1 as a starting text, and if δt is greater than 0, the reading module obtains the starting text according to the following formula:
；
wherein M represents the data size of the whole monitoring task, floor represents the whole taking-down, 255 represents each text size of 255M, and the calculated integer value is the text initial sequence number value of the original data of the taken monitoring task;
step 3, sweep frequency basic data reading module ratioComparing the time difference between the acquisition monitoring task duration delta T and the query end time and the text end timet, the calculation method is as follows:
δT=T2-T1，t=T2-t2；
wherein T2 represents the end time of the text of the monitoring task, T1 represents the start time of the text of the monitoring task, and T2 represents the end time of the query;
if it ist is less than 0, the sweep frequency basic data reading module provides a stored monitoring text with the largest monitoring original task data label as an ending text; if->And t is greater than 0, acquiring an initial text by the sweep frequency basic data reading module according to the following formula:
；
wherein M represents the data size of the whole monitoring task, ceil represents the taking integer, 255 represents that the size of each text is 255M, and the calculated integer value is the text ending sequence number value of the original data of the taken monitoring task;
and step 4, returning data by the sweep frequency basic data reading module in a sequential increasing mode, wherein the returned data comprises the following contents:
< monitor original document text URL1, access username, access password >, < monitor original document text URL2, access username, access password > … >.
Preferably, the sweep frequency basic data reading module supports continuous return of data flow in a Socket form, and the method comprises the following steps:
step 1, positioning and monitoring the file position of original data according to a file positioning algorithm;
step 2, invoking a frame decoding algorithm, carrying out frame decoding on the positioned original monitoring file, comparing the time in the frame with the inquiry starting time, if the time of the two minutes is equal, successfully positioning the data stream, and establishing a socket stream;
and step 3, returning the URL, the user name and the password corresponding to the socket stream by the sweep frequency basic data reading module.
After the technical scheme is adopted, the invention has the following advantages:
the invention utilizes a distributed storage system, a radio integrated platform and a structured database to develop and design a radio ultrashort wave frequency band monitoring sweep frequency basic data storage method, comprehensively realizes long-time effective storage of radio ultrashort wave monitoring sweep frequency basic data and accurate positioning reading of monitoring basic data, thereby forming a data asset foundation of full-province or ground city monitoring data, guiding a monitoring center to better manage and use ultrashort wave monitoring data results. The method solves the problems of effective storage and rapid extraction of mass sweep frequency monitoring data after networking monitoring.
Drawings
FIG. 1 is a schematic diagram of a radio ultra-short wave frequency band monitoring sweep frequency basic data storage system;
fig. 2 is a flowchart of a method for storing the monitoring sweep frequency basic data of the radio ultrashort wave frequency band.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples.
A data storage method for monitoring sweep frequency basic data in a radio ultrashort wave frequency band is based on a system for storing the sweep frequency basic data in the radio ultrashort wave frequency band.
As shown in fig. 1, the radio ultrashort wave frequency band monitoring sweep frequency basic data storage system comprises a sweep frequency data import module, a sweep frequency basic data storage module, a sweep frequency basic data configuration module and a sweep frequency basic data reading module.
The sweep frequency basic data importing module is used for importing networking sweep frequency data and drive test bin file data into a radio ultrashort wave monitoring sweep frequency basic data storage system, and the data are uploaded to a radio monitoring data platform through a data transmission tool and stored;
the frequency sweep basic data storage module comprises a normal frequency sweep basic data storage module and a compressed frequency sweep basic data storage module, wherein the normal frequency sweep monitoring data storage module is used for storing the latest uploaded monitoring data, and the compressed frequency sweep basic data storage module is used for storing long-time unused frequency sweep data;
the sweep frequency basic data configuration module is used for realizing the configuration management function of stored sweep frequency data, and comprises normal sweep frequency data management, compressed sweep frequency data management and data compression management;
the sweep frequency basic data reading module is used for realizing the rapid reading function of the original monitoring data, and comprises the reading of the original monitoring file and the reading of the basic monitoring data stream.
As shown in fig. 2, a method for storing frequency sweep basic data for monitoring a radio ultrashort wave frequency band includes the following steps:
step 1, uploading sweep frequency monitoring data of networking equipment to a radio monitoring data platform in real time or quasi-real time through a sweep frequency data importing module, and writing corresponding task related information into a sweep frequency basic data configuration module;
step 2, the sweep frequency basic data storage module is based on a radio monitoring data platform, and after receiving the uploaded sweep frequency data, the sweep frequency basic data storage module determines the number of backup data and the compression type of the data according to the data of the sweep frequency basic data configuration table to store the data;
step 3, the sweep frequency basic data configuration module stores, compresses and extracts various information of the recorded uploading basic monitoring data;
and 4, after the sweep frequency basic data reading module receives the requirement of external query data, determining the monitoring basic data through a positioning algorithm by querying corresponding table items in the sweep frequency basic data configuration module, and returning to related applications.
In the step 1, the networking monitoring platform uploads the data to the radio monitoring data platform through the data acquisition machine, and the data acquisition machine uploads the data to the radio monitoring data platform through relevant tools such as transmission software Flume and FTP.
In the step 2, the types of data stored in the sweep frequency basic data storage module include original monitoring sweep frequency basic data and compressed monitoring sweep frequency basic data, the monitoring basic data storage module compresses according to idle time of task data, the monitoring basic data storage module reads last editing time of tasks in a task information table in the monitoring configuration module, judges whether the time and the current time exceed one year, if so, the tasks are compressed, after the compression, the monitoring basic data only retain one data, and meanwhile, compressed marks are marked in task file processing fields.
The sweep frequency basic data storage module receives the sweep frequency basic original data and stores the sweep frequency basic original data to the radio data platform, the sweep frequency monitoring basic data is stored in the radio data platform according to the task dimension, and the sweep frequency basic data storage module also updates the data to a task information table stored in the sweep frequency data configuration module.
The fields in the save task information table are shown in table 1:
TABLE 1
| Sequence number | Field name | Chinese description | Type(s) |
| 1 | ID | Sequence number | varchar(255) |
| 2 | TASKID | Task ID | varchar（64） |
| 3 | MFID | Facility ID | varchar（64） |
| 4 | EUQID | Device ID | varchar（64） |
| 5 | TASKTYPE | Task type | varchar（64） |
| 6 | APPID | Application type | varchar（64） |
| 7 | USERID | User ID | varchar（64） |
| 8 | STATUS | Task status | varchar（64） |
| 9 | FEATURE | Service code | int |
| 10 | FUNCTIONNAME | Function name | varchar（64） |
| 11 | STEP | Stepping | int |
| 12 | STARTFREQ | Start frequency | int |
| 13 | ENDFREQ | End frequency | int |
| 14 | CREATETIME | Task creation time | time |
| 15 | STARTTIME | Task start time | time |
| 16 | ENDTIME | Task end time | time |
| 17 | RUNSTARTTIME | Data start time | time |
| 18 | RUNENDTIME | Data end time | time |
| 19 | EDITTIME | Editing time | time |
| 20 | TASKNAME | Task name | varchar（64） |
| 21 | funparam | Functional parameters | varchar（64） |
| 22 | fummodule | Functional mode | varchar（64） |
| 23 | FILESIZE | File size | int |
| 24 | DEALSTATUS | File processing status | int |
| 25 | MONITORXMLIN | Managing XML input parameters | varchar（255） |
| 26 | MONITORXMLOUT | Managing XML output parameters | varchar（255） |
| 27 | taskfile_state | Task file type | varchar(10) |
| 28 | signal_extraction | Signal extraction | int |
The data updated by the sweep frequency basic data storage module is used for butting task information with the radio data platform, periodically inquiring the task ID of the uploaded data from the radio data platform and storing the corresponding information into a task information storage table; meanwhile, the sweep frequency basic data storage module automatically updates the saved task information table through the uploaded data, the sweep frequency basic data storage module compares the minimum time in each task text frame with the text starting time in the saved task information table, if the minimum time in each task text frame is smaller than the text starting time in the saved task information table, and meanwhile, compares the maximum time in each task text frame with the text ending time in the saved task information table, and if the maximum time in each task text frame is larger than the text ending time in the saved task information table, the sweep frequency basic data storage module updates; the text size of the corresponding task is updated every time the sweep frequency basic data storage module receives one text; and the sweep frequency basic data storage module selects and changes the number of backup parts of the stored original data through the backup parameters in the sweep frequency basic data configuration module.
In the step 3, the sweep frequency basic data configuration data stored by the sweep frequency basic data configuration module is stored in a data set form, and the data set comprises a corresponding storage task information table and a data backup configuration table.
The data backup configuration table includes fields as shown in table 2:
TABLE 2
| Sequence number | Field name | Chinese description | Type(s) |
| 1 | ID | Sequence number | varchar(255) |
| 2 | MFID | Facility ID | varchar（64） |
| 3 | EUQID | Device ID | varchar（64） |
| 4 | BACKUP | Prepared parts | int |
In the step 4, the sweep frequency basic data reading module provides quick and accurate positioning return of the original monitoring sweep frequency file, and supports quick original monitoring file return and byte return of socket flow.
In the quick original monitoring file return, the sweep frequency basic data reading module returns the corresponding monitoring file according to the time or the time period of data extraction, monitoring equipment and monitoring tasks, and the query parameters are as follows: < query start time, query end time, task ID >.
After the sweep frequency basic data reading module receives the inquiry parameters, specific file reading data is positioned according to the starting time and the task ID, and the steps are as follows:
step 1, a sweep frequency basic data reading module inquires and stores a task information table according to a task ID, wherein inquired fields comprise the starting time of a task text, the ending time of the text, the size of monitoring task data and compression state parameters of the monitoring task; if the compression state parameter of the monitoring task is displayed to be compressed, the data extraction module decompresses the monitoring task and then continues to execute the step 1;
step 2, the sweep frequency basic data reading module compares and acquires the time difference delta T between the duration delta T of the monitoring task and the inquiry starting time and the text starting time, and the calculation method is as follows:
δT=T2-T1，δt=t1-T1；
wherein T2 represents the end time of the text of the monitoring task, T1 represents the start time of the text of the monitoring task, and T1 represents the start time of the query;
if δt is less than 0, the reading module provides the saved monitoring text with the monitoring original task data numbered 1 as a starting text, and if δt is greater than 0, the reading module obtains the starting text according to the following formula:
；
wherein M represents the data size of the whole monitoring task, floor represents the whole taking-down, 255 represents each text size of 255M, and the calculated integer value is the text initial sequence number value of the original data of the taken monitoring task;
step 3, the sweep frequency basic data reading module compares and acquires the time difference between the duration delta T of the monitoring task and the inquiry ending time and the text ending timet, the calculation method is as follows:
δT=T2-T1，t=T2-t2；
wherein T2 represents the end time of the text of the monitoring task, T1 represents the start time of the text of the monitoring task, and T2 represents the end time of the query;
if it ist is less than 0, the sweep frequency basic data reading module provides a stored monitoring text with the largest monitoring original task data label as an ending text; if->And t is greater than 0, acquiring an initial text by the sweep frequency basic data reading module according to the following formula:
；
wherein M represents the data size of the whole monitoring task, ceil represents the taking integer, 255 represents that the size of each text is 255M, and the calculated integer value is the text ending sequence number value of the original data of the taken monitoring task;
and step 4, returning data by the sweep frequency basic data reading module in a sequential increasing mode, wherein the returned data comprises the following contents:
< monitor original document text URL1, access username, access password >, < monitor original document text URL2, access username, access password > … >.
The sweep frequency basic data reading module supports continuous return of data flow in a Socket form, and the method comprises the following steps:
step 1, positioning and monitoring the file position of original data according to a file positioning algorithm;
step 2, invoking a frame decoding algorithm, carrying out frame decoding on the positioned original monitoring file, comparing the time in the frame with the inquiry starting time, if the time of the two minutes is equal, successfully positioning the data stream, and establishing a socket stream;
and step 3, returning the URL, the user name and the password corresponding to the socket stream by the sweep frequency basic data reading module.
The application method of the radio ultra-short wave frequency band monitoring sweep frequency basic data storage method comprises the following steps:
step 1, a sweep frequency data import module uses a data transmission tool to upload monitoring original data from a data of the block to a sweep frequency basic data storage module in a radio monitoring data platform in sequence from a collector;
step 2, after the sweep frequency basic data storage module receives the data again, inquiring task information related to the integrated management and control platform according to the task ID in the monitoring block file, if the task information does not exist in the sweep frequency basic data configuration module, writing all the information into a task information storage table, and adding the basic data storage module into fields which cannot be updated according to the uploaded file and the content of the file, wherein the fields comprise data start time, data end time, file size, file processing state, task file type and signal extraction fields; if the task information obtained after the inquiry is already in the task storage information table, only updating the data start time, the data end time, the file size, the file processing state, the task file type and the signal extraction field;
step 3, monitoring basic data is stored in a sweep frequency basic data storage module according to natural task blocks, and the sweep frequency basic data storage module is organized by adopting an HDFS file system which is easy to expand and suitable for mass data storage;
step 4, the sweep frequency basic data storage module periodically patrols each task, if the task does not generate any access record in the last year, the task is compressed, and after the data is compressed, the data is stored according to 1 storage block;
the monitoring basic data reading module realizes the function of rapidly providing detection original data for other modules and applications, and the working method is as follows:
step 1, a sweep frequency monitoring basic data reading module receives a data query request from an external application in the form of an interface, and the request information is as follows:
< task ID, query start time, query end time >;
step 2, the sweep frequency monitoring basic data reading module inquires task information corresponding to the task ID from the monitoring basic data configuration module according to the task ID, wherein the task information comprises the following information:
(data start time, data end time, monitor task size, file processing status);
the sweep monitoring basic data reading module judges file processing filling of a task, if the corresponding task is compressed, an instruction is sent out, and the compressed task in the sweep monitoring data storage module is automatically decompressed to a normal task state;
step 3, the sweep frequency monitoring basic data reading module calculates the position of a starting text according to the starting time of the query task, the starting time of the text and the current size of each monitoring block file, and splices the file name by using the task ID and the monitoring task type; the sweep frequency monitoring basic data reading module calculates the position of the ending text according to the ending time of the query task, the ending time of the text and the size of each current monitoring block file, and the total size of the monitoring task is utilized to calculate the last file returned by the splicing part of the task ID and the monitoring task type;
and 4, returning all files between the inquiry start file and the inquiry end file to the inquirer by the sweep monitoring basic data reading module in the following mode:
< task file 1, ULR1, username, password >, < task file 2, ULR, username, password > … < task file m, ULR, username, password >.
The method for storing the radio ultrashort wave frequency band monitoring sweep frequency basic data realizes effective storage of the original data which is met for 5 years after networking through national integrated platform equipment, and also has good system expansibility; meanwhile, when other external application systems or other industries need related data, the system can quickly position related data file blocks as required and provide the related data file blocks in sequence, so that the management and the use of the radio industry to the industrial data assets are greatly improved, and the system is an effective means for improving the work intellectualization, the datamation and the automation of the radio industry.
In addition to the above preferred embodiments, the present invention has other embodiments, and various changes and modifications may be made by those skilled in the art without departing from the spirit of the invention, which is defined in the appended claims.
Claims (10)
1. A method for storing the basic data of radio ultra-short wave frequency band monitoring sweep frequency is characterized in that,
the data storage method is based on a radio ultrashort wave frequency band monitoring sweep frequency basic data storage system, and the radio ultrashort wave frequency band monitoring sweep frequency basic data storage system comprises a sweep frequency data import module, a sweep frequency basic data storage module, a sweep frequency basic data configuration module and a sweep frequency basic data reading module; the sweep frequency basic data importing module is used for importing networking sweep frequency data and drive test bin file data into a radio ultrashort wave monitoring sweep frequency basic data storage system, and the data are uploaded to a radio monitoring data platform through a data transmission tool and stored; the frequency sweep basic data storage module comprises a normal frequency sweep basic data storage module and a compressed frequency sweep basic data storage module, wherein the normal frequency sweep monitoring data storage module is used for storing the latest uploaded monitoring data, and the compressed frequency sweep basic data storage module is used for storing long-time unused frequency sweep data; the sweep frequency basic data configuration module is used for realizing the configuration management function of stored sweep frequency data, and comprises normal sweep frequency data management, compressed sweep frequency data management and data compression management; the sweep frequency basic data reading module is used for realizing the rapid reading function of the original monitoring data, and comprises the reading of the original monitoring file and the reading of the basic monitoring data stream;
the data storage method comprises the following steps:
step 1, uploading sweep frequency monitoring data of networking equipment to a radio monitoring data platform in real time or quasi-real time through a sweep frequency data importing module, and writing corresponding task related information into a sweep frequency basic data configuration module;
step 2, the sweep frequency basic data storage module is based on a radio monitoring data platform, and after receiving the uploaded sweep frequency data, the sweep frequency basic data storage module determines the number of backup data and the compression type of the data according to the data of the sweep frequency basic data configuration table to store the data;
step 3, the sweep frequency basic data configuration module stores, compresses and extracts various information of the recorded uploading basic monitoring data;
and 4, after the sweep frequency basic data reading module receives the requirement of external query data, determining the monitoring basic data through a positioning algorithm by querying corresponding table items in the sweep frequency basic data configuration module, and returning to related applications.
2. The method for storing basic data of monitoring sweep frequency in ultra-short wave band according to claim 1, wherein in step 1, the networking monitoring platform uploads the data to the monitoring basic sweep frequency data through a collector, and the collector uploads the data to the radio monitoring data platform through related tools such as transmission software Flume and FTP.
3. The method according to claim 1, wherein in the step 2, the types of data stored in the sweep frequency basic data storage module include original monitoring sweep frequency basic data and compressed monitoring sweep frequency basic data, the monitoring basic data storage module compresses according to idle time of task data, the monitoring basic data storage module reads last editing time of task in task information table in the monitoring configuration module, judges whether the time and the current time are more than one year, if yes, the task is compressed, after compression, the monitoring original data only remains one data, and at the same time, the compressed mark is marked in task file processing field.
4. A method for storing basic data of monitoring sweep frequency in ultra-short wave band according to claim 1 or 3, wherein the basic data storage module of sweep frequency receives the basic data of sweep frequency and stores the basic data of sweep frequency to the radio data platform, the basic data of sweep frequency is stored in the radio data platform according to the task dimension, and the basic data storage module of sweep frequency also updates the data to the task information table stored in the configuration module of sweep frequency.
5. The method for storing the sweep frequency basic data for monitoring the ultrashort wave frequency band of the radio according to claim 4, wherein the data updated by the sweep frequency basic data storage module is used for butting task information with a radio data platform, periodically inquiring the task ID of the uploaded data from the radio data platform and storing the corresponding information into a task information storage table; meanwhile, the sweep frequency basic data storage module automatically updates the saved task information table through the uploaded data, the sweep frequency basic data storage module compares the minimum time in each task text frame with the text starting time in the saved task information table, if the minimum time in each task text frame is smaller than the text starting time in the saved task information table, and meanwhile, compares the maximum time in each task text frame with the text ending time in the saved task information table, and if the maximum time in each task text frame is larger than the text ending time in the saved task information table, the sweep frequency basic data storage module updates; the text size of the corresponding task is updated every time the sweep frequency basic data storage module receives one text; and the sweep frequency basic data storage module selects and changes the number of backup parts of the stored original data through the backup parameters in the sweep frequency basic data configuration module.
6. The method according to claim 1, wherein in step 3, the sweep frequency basic data configuration data stored by the sweep frequency basic data configuration module is stored in the form of a data set, and the data set includes a corresponding save task information table and a data backup configuration table.
7. The method according to claim 1, wherein in step 4, the sweep frequency basic data reading module provides fast and accurate positioning return of the original monitoring sweep frequency file, and supports fast original monitoring file return and byte return of the socket stream.
8. The method for storing the frequency sweep basic data for monitoring the ultrashort wave frequency band of the radio according to claim 7, wherein in the quick original monitoring file return, the frequency sweep basic data reading module returns the corresponding monitoring file according to the time or the time period of data extraction, the monitoring equipment and the monitoring task, and the query parameters are as follows: < query start time, query end time, task ID >.
9. The method for storing the frequency sweep basic data for monitoring the ultra-short wave band of radio according to claim 8, wherein,
after the sweep frequency basic data reading module receives the inquiry parameters, specific file reading data is positioned according to the starting time and the task ID, and the steps are as follows:
step 1, a sweep frequency basic data reading module inquires and stores a task information table according to a task ID, wherein inquired fields comprise the starting time of a task text, the ending time of the text, the size of monitoring task data and compression state parameters of the monitoring task; if the compression state parameter of the monitoring task is displayed to be compressed, the data extraction module decompresses the monitoring task and then continues to execute the step 1;
step 2, the sweep frequency basic data reading module compares and acquires the time difference delta T between the duration delta T of the monitoring task and the inquiry starting time and the text starting time, and the calculation method is as follows:
δT=T2-T1，δt=t1-T1；
wherein T2 represents the end time of the text of the monitoring task, T1 represents the start time of the text of the monitoring task, and T1 represents the start time of the query;
if δt is less than 0, the reading module provides the saved monitoring text with the monitoring original task data numbered 1 as a starting text, and if δt is greater than 0, the reading module obtains the starting text according to the following formula:
；
wherein M represents the data size of the whole monitoring task, floor represents the whole taking-down, 255 represents each text size of 255M, and the calculated integer value is the text initial sequence number value of the original data of the taken monitoring task;
step 3, the sweep frequency basic data reading module compares the obtained monitoring task duration delta T with the inquiry ending time and the text ending timeTime differencet, the calculation method is as follows:
δT=T2-T1，t=T2-t2；
wherein T2 represents the end time of the text of the monitoring task, T1 represents the start time of the text of the monitoring task, and T2 represents the end time of the query;
if it ist is less than 0, the sweep frequency basic data reading module provides a stored monitoring text with the largest monitoring original task data label as an ending text; if->And t is greater than 0, acquiring an initial text by the sweep frequency basic data reading module according to the following formula:
；
wherein M represents the data size of the whole monitoring task, ceil represents the taking integer, 255 represents that the size of each text is 255M, and the calculated integer value is the text ending sequence number value of the original data of the taken monitoring task;
and step 4, returning data by the sweep frequency basic data reading module in a sequential increasing mode, wherein the returned data comprises the following contents:
< monitor original document text URL1, access username, access password >, < monitor original document text URL2, access username, access password > … >.
10. The method for storing frequency sweep basic data for monitoring radio ultrashort wave frequency band according to claim 9, wherein the frequency sweep basic data reading module supports continuous return of data stream in a Socket form, and the method comprises the following steps:
step 1, positioning and monitoring the file position of original data according to a file positioning algorithm;
step 2, invoking a frame decoding algorithm, carrying out frame decoding on the positioned original monitoring file, comparing the time in the frame with the inquiry starting time, if the time of the two minutes is equal, successfully positioning the data stream, and establishing a socket stream;
and step 3, returning the URL, the user name and the password corresponding to the socket stream by the sweep frequency basic data reading module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110854127.XA CN113609081B (en) | 2021-07-28 | 2021-07-28 | Method for storing radio ultrashort wave frequency band monitoring sweep frequency basic data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110854127.XA CN113609081B (en) | 2021-07-28 | 2021-07-28 | Method for storing radio ultrashort wave frequency band monitoring sweep frequency basic data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113609081A CN113609081A (en) | 2021-11-05 |
| CN113609081B true CN113609081B (en) | 2023-07-28 |
Family
ID=78305693
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110854127.XA Active CN113609081B (en) | 2021-07-28 | 2021-07-28 | Method for storing radio ultrashort wave frequency band monitoring sweep frequency basic data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113609081B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110350993A (en) * | 2019-08-19 | 2019-10-18 | 浙江原初数据科技有限公司 | A kind of black broadcast automatic discovering method based on networking monitoring under big data scene |
| CN112527805A (en) * | 2021-01-26 | 2021-03-19 | 浙江原初数据科技有限公司 | Real-time playback method of radio mass monitoring data |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080036601A1 (en) * | 2006-08-11 | 2008-02-14 | Thompson Louis H | Radio frequency integration platform and network therefor |
-
2021
- 2021-07-28 CN CN202110854127.XA patent/CN113609081B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110350993A (en) * | 2019-08-19 | 2019-10-18 | 浙江原初数据科技有限公司 | A kind of black broadcast automatic discovering method based on networking monitoring under big data scene |
| CN112527805A (en) * | 2021-01-26 | 2021-03-19 | 浙江原初数据科技有限公司 | Real-time playback method of radio mass monitoring data |
Non-Patent Citations (1)
| Title |
|---|
| 在超短波监测一体化体系中数据流压缩技术的应用;王善道;刘会明;;中国无线电(08);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113609081A (en) | 2021-11-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103246595B (en) | Application management method, device, server and terminating unit | |
| KR102511271B1 (en) | Method and device for storing and querying time series data, and server and storage medium therefor | |
| US7809752B1 (en) | Representing user behavior information | |
| CN102446326B (en) | A kind of method of information pushing, system and equipment | |
| CN111125089A (en) | Time sequence data storage method, device, server and storage medium | |
| CN102244758A (en) | Video-recording-file-based data acquisition method and equipment | |
| CN104317800A (en) | Hybrid storage system and method for mass intelligent power utilization data | |
| CN105069048A (en) | Small file storage method, query method and device | |
| CN103873503A (en) | Data block backup system and method | |
| CN102799598A (en) | Data recovery method for deleting repeated data | |
| CN102346751B (en) | Information transmitting method and equipment | |
| CN107741889A (en) | Document recovery method, system, readable storage medium storing program for executing and terminal | |
| CN1924854B (en) | Desktop searching method for intelligent mobile terminal | |
| CN103823807A (en) | Data de-duplication method, device and system | |
| CN107491458B (en) | Method, device and system for storing time series data | |
| CN112925757A (en) | Method, equipment and storage medium for tracking operation log of intelligent equipment | |
| CN111008183A (en) | Storage method and system for business wind control log data | |
| CN101645062B (en) | Report form generation method and system | |
| CN113609081B (en) | Method for storing radio ultrashort wave frequency band monitoring sweep frequency basic data | |
| CN104899702A (en) | Big data-based decoration budget quota management system and method | |
| CN100561936C (en) | Equipment inspection system and method | |
| KR101666440B1 (en) | Data processing method in In-memory Database System based on Circle-Queue | |
| US7337029B2 (en) | Design data management system and trace system | |
| CN111427845B (en) | Interactive modeling analysis operator data exchange method | |
| CN109993576B (en) | Method and system for sensing service quality, acquiring, processing and analyzing data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Ma Gaofeng Inventor after: Cui Hong Inventor after: Tu Yongsheng Inventor after: Kang Hongjie Inventor after: Li Jiangmin Inventor after: Huang Lei Inventor before: Ma Gaofeng Inventor before: Tu Yongsheng Inventor before: Kang Hongjie Inventor before: Li Jiangmin Inventor before: Huang Lei |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |