CN112527805B - Real-time playback method of radio mass monitoring data - Google Patents

Real-time playback method of radio mass monitoring data Download PDF

Info

Publication number
CN112527805B
CN112527805B CN202110101620.4A CN202110101620A CN112527805B CN 112527805 B CN112527805 B CN 112527805B CN 202110101620 A CN202110101620 A CN 202110101620A CN 112527805 B CN112527805 B CN 112527805B
Authority
CN
China
Prior art keywords
playback
data
time
frequency point
distributed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110101620.4A
Other languages
Chinese (zh)
Other versions
CN112527805A (en
Inventor
李江敏
马高峰
陈伟
张鹏程
涂永胜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Yuanchu Data Technology Co ltd
Original Assignee
Zhejiang Yuanchu Data Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Yuanchu Data Technology Co ltd filed Critical Zhejiang Yuanchu Data Technology Co ltd
Priority to CN202110101620.4A priority Critical patent/CN112527805B/en
Publication of CN112527805A publication Critical patent/CN112527805A/en
Application granted granted Critical
Publication of CN112527805B publication Critical patent/CN112527805B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/221Column-oriented storage; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/215Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24552Database cache management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/248Presentation of query results
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computing Systems (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)

Abstract

The invention provides a real-time playback method of radio mass monitoring data, which comprises a distributed storage system, a data cleaning and loading module, a distributed playback storage module and a distributed playback control module which are sequentially connected; the applied acquisition machine transmits the acquired frequency spectrum data to the distributed storage system; the distributed storage system is a storage module of a big data platform, and original monitoring is uploaded to the distributed storage system and then waits for a subsequent module to process; the data cleaning and loading module extracts data from the distributed storage system, frames the data according to an atomic protocol, frames the monitoring file of each task, stores the monitoring file and writes the monitoring file into an Hbase database; the distributed playback storage module stores the played back data in a column-type storage mode and is implemented on a column-type storage Hbase database; the distributed playback control module is used for controlling playback, and comprises the steps of receiving external query parameters, performing playback rate control, fault-tolerant management and stream pushing.

Description

Real-time playback method of radio mass monitoring data
Technical Field
The invention belongs to the technical field of playback of radio historical monitoring data, and particularly relates to a real-time playback method of radio mass monitoring data.
Background
With the construction and data convergence of a large radio monitoring data platform, radio monitoring centers in various provinces accumulate large-scale networking fixed station monitoring data which are precious data assets in the radio industry, the requirements of the monitoring centers and various radio applications on fine-grained playback of massive historical data are increasing day by day, and many applications need to query the data according to certain conditions and perform playback processing according to a specific playback sequence, such as multidimensional waterfall playback, occupancy rate trend, signal simulation and the like of the monitoring data. Since the playback data amount involved in the monitoring data playback is huge, generally the monitoring data amount of one year in one province is about 50T, the upper layer application cannot simply query the storage result data from the database and then play back the storage result. Currently, the playback of radio monitoring data provides the data in the following way:
original stream file mode: after receiving a request of the application for the original data playback, the center sends the monitoring file containing the time interval to the application by searching, and after receiving the reply message, the application acquires the original monitoring file from the corresponding address according to the download information of the reply message. The application unframes the part of the monitoring files according to the atomic service, acquires the data of the dimension required by the application and then applies the data
The coarse grain approach provides: because the amount of the played back data is huge, the center clusters the monitoring data according to days, hours and the like, keeps the average value of the levels of the monitoring data, stores the data in a database, and returns the data to an application after inquiring the data through corresponding inquiry conditions when the application needs to play back. And when the amount of data is large, it is difficult for the conventional database to guarantee the performance of data playback.
These playback methods and playback techniques all suffer from different drawbacks and cannot support large-scale playback services in an efficient and easy-to-use manner. On one hand, the application is provided with the original file, because the file needs to be downloaded and unframed, the playback speed is extremely slow, and because the file is the original file, the application needs to be unframed and positioned, the application pressure is caused; on one hand, the coarse-grained provision method cannot meet the granularity requirement of the application, and the specific query is executed on the static data of the database system, and the query result is returned to the user in a batch processing mode rather than a stream processing mode, so that the speed cannot be controlled; finally, the defects of long time delay of batch processing, incapability of controlling the playing speed and the like exist respectively in a database system or a stream calculation mode. Therefore, the existing playback modes cannot be directly used as a playback system to process static data.
Disclosure of Invention
The present invention aims to solve the above technical problems and provide a real-time playback method for radio massive monitoring data.
In order to achieve the purpose, the invention adopts the following technical scheme:
a real-time playback method of radio mass monitoring data comprises a distributed storage system, a data cleaning and loading module, a distributed playback storage module and a distributed playback control module which are connected in sequence; the applied acquisition machine transmits the acquired frequency spectrum data to a distributed storage system through data such as FTP (file transfer protocol), Flume and the like; the distributed storage system is a storage module of a big data platform, and original monitoring is uploaded to the distributed storage system and then waits for a subsequent module to process; the data cleaning and loading module extracts data from the distributed storage system, deframes the data according to an atomic protocol, deframes the monitoring file of each task, stores the deframed monitoring file according to a format of < time, frequency point and level >, and writes the deframed monitoring file into an Hbase database; the distributed playback storage module stores the played back data in a column-type storage mode and is implemented on a column-type storage Hbase database; the distributed playback control module is used for controlling playback, and comprises receiving external query parameters, performing playback rate control, fault-tolerant management and stream pushing; the distributed playback storage module is divided into a time sequence level data storage method and a frequency point detail storage method according to a time domain and a frequency domain.
Preferably, the time-series level data storage method stores data according to a time domain, the time-series level data completes the storage of the minute-level, fifteen-minute-level and hour-level data of all monitoring data, the storage of the characteristic values of each frequency point is completed in each time dimension, the characteristic values include the average value, the maximum value and the minimum value of the level of the frequency point at the moment, and the corresponding storage modes are as follows:
Figure 100002_DEST_PATH_IMAGE002
wherein, the row key of the time sequence level can be used as a row key to quickly inquire and position historical data, a 16 byte is used as the value of the row key, wherein, the task ID of 6 bits is mapped by 32 bit task IDs according to the sequence of numbers;
timestamp represents a timestamp, the time value at which data was written for this purpose; in the column names, startfreq represents the value of the starting frequency point at the moment, step represents the monitoring step length at the moment, and num represents the number of the frequency points at the moment; avg represents the average value of the level at the moment, each frequency point has an average level value, two bytes are occupied, and the average level values of all the frequency points are arranged according to the frequency point sequence and are stored according to the byte array; min and max represent the minimum and maximum levels, which are organized and stored in the same way as the average level.
Preferably, the frequency point details are mainly stored in a frequency domain, the frequency point details are mainly aggregated in aggregation granularity of fifteen minutes and hour, and the corresponding storage mode is as follows:
Figure DEST_PATH_IMAGE004
wherein, rowkey of frequency point details is used as row key to quickly inquire and position historical data, wherein, 15 bytes are selected as row key value, wherein, the 6-bit task ID is mapped by 32-bit task ID according to the sequence of numbers;
timestamp represents a timestamp, the time value at which data was written for this purpose; in the column name, { time1} indicates a specific time, the corresponding values are all level values of a specific frequency point at that time and the number of times the level occurs, and the level values are arranged in descending or ascending order of the level values.
Preferably, the distributed playback control module comprises a playback controller, the playback controller is used for playing back data streams, and the playback controller comprises a frequency point controller, an occupancy rate controller and a level controller;
the real-time playback method of the radio mass monitoring data comprises a playback job execution method, and the playback job execution method comprises the following steps:
s1. the playback client sends a playback request to the playback server, the playback request mainly supports the query of frequency point trend, level and occupancy rate, the minimum supported time granularity is 1 minute;
s2, after receiving the request, the playback server sends the request to corresponding controllers according to the type of the request, and the various request controllers decompose, dilute and control the speed of the playback request;
s3. decomposing and mapping the requests by various controllers, changing the requests into a string of continuous rowkeys, and inquiring corresponding records from Hbase by the controllers by taking the rowkeys as keywords;
s4. each request controller sends the received data to the playback server;
s5. the playback server returns data to the playback client in turn according to the connection socket streaming interface.
Preferably, the playback controller comprises a time sequence level playback and a frequency point detail playback;
the time sequence level playback shows the characteristic conditions of all frequency points at a certain moment, such as the average level, the median, the maximum and the minimum of the frequency points, and the input parameters are as follows: < start time, end time, start frequency point, end frequency point, rarefaction pace, playback rate, device ID >;
and the frequency point detail playback shows the full situation of the frequency point at each moment in a period of time, such as occupancy rate, trend and the like, and the input parameters are as follows: < start time, end time, frequency point, playback rate, device ID >;
according to the parameters, the playback system sequentially acquires all data corresponding to time, frequency points and equipment from the column storage, the data are sequentially sent to the application to meet the real-time data requirement of the application, the playback of all the data supports the specified playback rate, and the playback controller controls the sending rate through the playback rate in the parameters.
Preferably, after receiving a playback request of a playback client, a playback controller decomposes the request according to request parameters to generate a series of rowkeys required by the query, the playback controller queries from an Hbase database according to the rowkeys, a part of query results are put into a cache of the playback, a speed controller of the playback controller reads data from the cache according to the speed and sends the data to an application client, if the data in the cache is consumed completely, the rest of the rowkeys are used for continuously querying the data from the Hbase database, and the playback controller sends the results to the playback client one by one in a socket data stream mode.
After the technical scheme is adopted, the invention has the following advantages:
in order to realize that the radio data playback needs to read the data with self-defined, rapid and coarse granularity, and transmit the data to data display software and algorithm to carry out diversified display such as numbers, curves, drawings, algorithms and the like, thereby facilitating the reproduction of upper-layer application.
Drawings
Fig. 1 is a schematic structural diagram of a real-time playback method of radio mass monitoring data;
FIG. 2 is a flow chart illustrating the steps of a real-time playback method of radio mass monitoring data;
fig. 3 is a flowchart illustrating steps of a playback operation execution method in a real-time playback method of radio mass monitoring data.
Detailed Description
The present invention will be described in further detail with reference to the following drawings and specific examples.
As shown in fig. 1 to 3, a real-time playback method for radio mass monitoring data includes a distributed storage system, a data cleaning and loading module, a distributed playback storage module, and a distributed playback control module, which are connected in sequence.
The applied acquisition machine transmits the acquired frequency spectrum data to a distributed storage system through data such as FTP (file transfer protocol), Flume and the like; the distributed storage system is a storage module of a big data platform, and original monitoring is uploaded to the distributed storage system and then waits for a subsequent module to process; the data cleaning and loading module extracts data from the distributed storage system, deframes the data according to an atomic protocol, deframes the monitoring file of each task, stores the deframed monitoring file according to a format of < time, frequency point and level >, and writes the deframed monitoring file into an Hbase database; the distributed playback storage module stores the played back data in a column-type storage mode and is implemented on a column-type storage Hbase database; the distributed playback control module is used for controlling playback, and comprises the steps of receiving external query parameters, performing playback rate control, fault-tolerant management and stream pushing.
The real-time playback method of the radio mass monitoring data specifically comprises the following steps:
1. the networking monitoring application issues a monitoring task, monitoring data are stored in a monitoring application acquisition machine, and the acquisition machine transmits the data to a distributed storage system through data transmission software (FTP, FLUME) at intervals;
2. the distributed storage system performs compressed storage according to tasks and equipment;
3. the data cleaning and loading module detects the uploaded data, deframes the newly uploaded data according to an atomic protocol, and stores the deframed data into a distributed playback storage;
4. distributed playback storage uses columnar storage playback data (Hbase), including storage of time series level data and frequency point detail data;
5. the playback controller converts the playback request parameters into a row key group, fetches data from the distributed playback storage through the row keys, and finally sends the data to the upper application according to the designated rate.
The distributed playback storage module is divided into a time sequence level data storage method and a frequency point detail storage method according to a time domain and a frequency domain.
The time sequence level data storage method stores data according to a time domain, the time sequence level data completes the storage of minute-level, fifteen-minute-level and hour-level data of all monitoring data, the storage of characteristic values of each frequency point is completed under each time dimension, the characteristic values comprise the average value, the maximum value and the minimum value of the level of the frequency point at the moment, and the corresponding storage modes are as follows:
Figure DEST_PATH_IMAGE002A
wherein, the row key of the time sequence level can be used as a row key to quickly inquire and position historical data, a 16 byte is used as the value of the row key, wherein, the task ID of 6 bits is mapped by 32 bit task IDs according to the sequence of numbers;
timestamp represents a timestamp, the time value at which data was written for this purpose; in the column names, startfreq represents the value of the starting frequency point at the moment, step represents the monitoring step length at the moment, and num represents the number of the frequency points at the moment; avg represents the average value of the level at the moment, each frequency point has an average level value, two bytes are occupied, and the average level values of all the frequency points are arranged according to the frequency point sequence and are stored according to the byte array; min and max represent the minimum and maximum levels, which are organized and stored in the same way as the average level.
The frequency point details are mainly stored in a frequency domain, the frequency point details are mainly aggregated on aggregation granularity of fifteen minutes and hour, and the corresponding storage modes are as follows:
Figure DEST_PATH_IMAGE004A
wherein, rowkey of frequency point details is used as row key to quickly inquire and position historical data, wherein, 15 bytes are selected as row key value, wherein, the 6-bit task ID is mapped by 32-bit task ID according to the sequence of numbers;
timestamp represents a timestamp, the time value at which data was written for this purpose; in the column name, { time1} indicates a specific time, the corresponding values are all level values of a specific frequency point at that time and the number of times the level occurs, and the level values are arranged in descending or ascending order of the level values.
The real-time playback process of the radio mass monitoring data comprises the following steps:
1. the data cleaning and loading module calls a wireless data atomic protocol to unframe data, and the data is written into an Hbase database after being aggregated according to the requirement of a design table in the Hbase database;
2. the playback client sends a playback request to a playback server, the playback request mainly supports the query of frequency point trend, level and occupancy rate, and the minimum supported time granularity is 1 minute;
3. after receiving the request, the playback server sends the request to corresponding controllers according to the type of the request, and the various request controllers decompose, dilute and control the speed of the playback request;
4. decomposing and mapping the requests by various controllers, changing the requests into a string of continuous rowkeys, and inquiring corresponding records from the Hbase by the controllers by taking the rowkeys as keywords;
5. the various request controllers send the received data to a playback server;
6. and the playback server sequentially returns the data to the playback client according to the connection socket stream interface.
The distributed playback control module comprises a playback controller, the playback controller is used for playing back data streams, and the playback controller comprises a frequency point controller, an occupancy rate controller and a level controller;
the real-time playback method of the radio mass monitoring data comprises a playback job execution method, and the playback job execution method comprises the following steps:
s1. the playback client sends a playback request to the playback server, the playback request mainly supports the query of frequency point trend, level and occupancy rate, the minimum supported time granularity is 1 minute;
s2, after receiving the request, the playback server sends the request to corresponding controllers according to the type of the request, and the various request controllers decompose, dilute and control the speed of the playback request;
s3. decomposing and mapping the requests by various controllers, changing the requests into a string of continuous rowkeys, and inquiring corresponding records from Hbase by the controllers by taking the rowkeys as keywords;
s4. each request controller sends the received data to the playback server;
s5. the playback server returns data to the playback client in turn according to the connection socket streaming interface.
The playback controller comprises time sequence level playback and frequency point detail playback;
the time sequence level playback shows the characteristic conditions of all frequency points at a certain moment, such as the average level, the median, the maximum and the minimum of the frequency points, and the input parameters are as follows: < start time, end time, start frequency point, end frequency point, rarefaction pace, playback rate, device ID >;
and the frequency point detail playback shows the full situation of the frequency point at each moment in a period of time, such as occupancy rate, trend and the like, and the input parameters are as follows: < start time, end time, frequency point, playback rate, device ID >;
according to the parameters, the playback system sequentially acquires all data corresponding to time, frequency points and equipment from the column storage, the data are sequentially sent to the application to meet the real-time data requirement of the application, the playback of all the data supports the specified playback rate, and the playback controller controls the sending rate through the playback rate in the parameters.
The playback control of the real-time playback of the radio mass monitoring data mainly realizes the control of the playback rate so as to ensure the stability of the playback operation, and the realization process comprises the following steps:
1. after receiving a playback request of a playback client, the playback controller decomposes the request according to the request parameters to generate a series of rowkeys required by the query;
2. the playback controller inquires from Hbase according to the rowkeys, and a part of inquiry results are put into a cache of the playback;
3. the speed controller of the playback controller reads data from the cache according to the speed and sends the data to the application client, and if the data in the cache is completely consumed, the data is continuously inquired from the Hbase by using the rest Rowkey;
4. the playback controller sends the results to the client end one by one in the form of a socket data stream.
To this end, the data playback logic can meet all the requirements of a radio playback service.
Other embodiments of the present invention than the preferred embodiments described above will be apparent to those skilled in the art from the present invention, and various changes and modifications can be made therein without departing from the spirit of the present invention as defined in the appended claims.

Claims (3)

1. A real-time playback method of radio massive monitoring data is characterized by comprising a distributed storage system, a data cleaning and loading module, a distributed playback storage module and a distributed playback control module which are sequentially connected;
the applied acquisition machine transmits the acquired frequency spectrum data to a distributed storage system through FTP and Flume data; the distributed storage system is a storage module of a big data platform, and original monitoring is uploaded to the distributed storage system and then waits for a subsequent module to process; the data cleaning and loading module extracts data from the distributed storage system, deframes the data according to an atomic protocol, deframes the monitoring file of each task, stores the deframed monitoring file according to a format of < time, frequency point and level >, and writes the deframed monitoring file into an Hbase database; the distributed playback storage module stores the played back data in a column-type storage mode and is implemented on a column-type storage Hbase database; the distributed playback control module is used for controlling playback, and comprises receiving external query parameters, performing playback rate control, fault-tolerant management and stream pushing; the distributed playback storage module is divided into a time sequence level data storage method and a frequency point detail storage method according to a time domain and a frequency domain;
the frequency point details are stored in a frequency domain, and the frequency point details are aggregated on aggregation granularity of fifteen minutes and hour, and the corresponding storage mode is as follows:
Figure DEST_PATH_IMAGE001
wherein, rowkey of frequency point details is used as row key to quickly inquire and position historical data, wherein, 15 bytes are selected as row key value, wherein, the 6-bit task ID is mapped by 32-bit task ID according to the sequence of numbers;
timestamp represents a timestamp, the time value at which data was written for this purpose; amp represents that the data is stored according to column family; in the column name, { time1} represents a specific time, the corresponding values are all level values of specific frequency points at that time and the occurrence times of the levels, and the level values are arranged in descending order or ascending order of the level values;
the distributed playback control module comprises a playback controller, the playback controller is used for playing back data streams, and the playback controller comprises a frequency point controller, an occupancy rate controller and a level controller;
the real-time playback method of the radio mass monitoring data comprises a playback job execution method, and the playback job execution method comprises the following steps:
s1. the playback client sends a playback request to the playback server, the playback request supports the query of frequency point trend, level and occupancy rate, the minimum supported time granularity is 1 minute;
s2, after receiving the request, the playback server sends the request to corresponding controllers according to the type of the request, and the various request controllers decompose, dilute and control the speed of the playback request;
s3. decomposing and mapping the requests by various controllers, changing the requests into a string of continuous rowkeys, and inquiring corresponding records from Hbase by the controllers by taking the rowkeys as keywords;
s4. each request controller sends the received data to the playback server;
s5. the playback server returns data to the playback client in turn according to the connection socket stream interface;
the playback controller comprises time sequence level playback and frequency point detail playback;
the time sequence level playback shows the characteristic conditions of all frequency points at a certain moment, such as the average level, the median and the maximum and minimum values of the frequency points, and the input parameters are as follows: < start time, end time, start frequency point, end frequency point, rarefaction pace, playback rate, device ID >;
and the frequency point detail playback shows the full condition of the frequency point at each moment in a period of time, and the input parameters are as follows: < start time, end time, frequency point, playback rate, device ID >;
according to the parameters, the playback system sequentially acquires all data corresponding to time, frequency points and equipment from the column storage, the data are sequentially sent to the application to meet the real-time data requirement of the application, the playback of all the data supports the specified playback rate, and the playback controller controls the sending rate through the playback rate in the parameters.
2. The method for real-time playback of radio massive monitoring data according to claim 1, wherein the time sequence level data storage method stores data in time domain, the time sequence level data completes the storage of data of all the monitoring data in minute, fifteen minute and hour levels, and completes the storage of characteristic values of each frequency point in each time dimension, the characteristic values include the average value, the maximum value and the minimum value of the level of the frequency point at the time, and the corresponding storage modes are as follows:
Figure DEST_PATH_IMAGE002
wherein, the row key of the time sequence level can be used as a row key to quickly inquire and position historical data, a 16 byte is used as the value of the row key, wherein, the task ID of 6 bits is mapped by 32 bit task IDs according to the sequence of numbers;
timestamp represents a timestamp, the time value at which data was written for this purpose; amp represents that the data is stored according to column family; in the column names, startfreq represents the value of the starting frequency point at the moment, step represents the monitoring step length at the moment, and num represents the number of the frequency points at the moment; avg represents the average value of the level at the moment, each frequency point has an average level value, two bytes are occupied, and the average level values of all the frequency points are arranged according to the frequency point sequence and are stored according to the byte array; min and max represent the minimum and maximum levels, which are organized and stored in the same way as the average level.
3. The method as claimed in claim 1, wherein the playback controller, after receiving a playback request from the playback client, decomposes the request according to the request parameters to generate a series of rowkeys required for the query, queries the Hbase database according to the rowkeys, puts a portion of the query results into the buffer for the playback, reads data from the buffer according to the rate, and sends the data to the application client, if the data in the buffer is consumed, continues to query the Hbase database with the remaining rowkeys, and sends the results to the playback client one by one in the form of socket data stream.
CN202110101620.4A 2021-01-26 2021-01-26 Real-time playback method of radio mass monitoring data Active CN112527805B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110101620.4A CN112527805B (en) 2021-01-26 2021-01-26 Real-time playback method of radio mass monitoring data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110101620.4A CN112527805B (en) 2021-01-26 2021-01-26 Real-time playback method of radio mass monitoring data

Publications (2)

Publication Number Publication Date
CN112527805A CN112527805A (en) 2021-03-19
CN112527805B true CN112527805B (en) 2022-04-19

Family

ID=74975450

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110101620.4A Active CN112527805B (en) 2021-01-26 2021-01-26 Real-time playback method of radio mass monitoring data

Country Status (1)

Country Link
CN (1) CN112527805B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113194011B (en) * 2021-04-29 2022-08-26 浙江原初数据科技有限公司 Automatic establishment method and device for radio electromagnetic signal environment
CN113609081B (en) * 2021-07-28 2023-07-28 浙江原初数据科技有限公司 Method for storing radio ultrashort wave frequency band monitoring sweep frequency basic data

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103999090A (en) * 2011-12-14 2014-08-20 奈飞公司 Improving startup times of streaming digital media playback
CN105893596A (en) * 2016-04-18 2016-08-24 华信咨询设计研究院有限公司 Radio monitoring data replay method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9904718B2 (en) * 2013-03-13 2018-02-27 Wal-Mart Stores, Inc. System and method for streaming events in a transaction-based system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103999090A (en) * 2011-12-14 2014-08-20 奈飞公司 Improving startup times of streaming digital media playback
CN105893596A (en) * 2016-04-18 2016-08-24 华信咨询设计研究院有限公司 Radio monitoring data replay method

Also Published As

Publication number Publication date
CN112527805A (en) 2021-03-19

Similar Documents

Publication Publication Date Title
CN112527805B (en) Real-time playback method of radio mass monitoring data
CN109710678B (en) Real-time data system for industrial big data monitoring management and operation method
CN103688259B (en) For the method by compressing and file storage carries out automaticdata placement
CN100438625C (en) Video monitoring data storage managing method and system
CN102541990B (en) Database redistribution method and system utilizing virtual partitions
CN101160908A (en) Data synchronization processing method and client terminal, server and data synchronization system
EP1256896B1 (en) &#34;A plant information management system&#34;
EP3285186B1 (en) Methods and procedures for timestamp-based indexing of items in real-time storage
US20180165313A1 (en) Distributing and processing streams over one or more networks for on-the-fly schema evolution
CN110609813A (en) Data storage system and method
JPH0883232A (en) File server system
CN112035563A (en) Real-time database system based on shared storage
CN114189455A (en) Container network flow monitoring and counting method and system based on ebpf technology
CN1308852C (en) Data transmission method, computer-readable medium, and data transmission apparatus
CN112269726A (en) Data processing method and device
WO2010043138A1 (en) Method for accessing magnanimity data of intelligent network service database and system and device thereof
JP2010068279A (en) Flow information processor and network system
WO2017049488A1 (en) Cache management method and apparatus
CN112307102A (en) Ship edge intelligent data acquisition module based on time sequence database
JPH11149403A (en) Data sharing system and data shared network system
CN101217437A (en) A data acquisition monitoring system based on SMS module
KR100375365B1 (en) A system for providing stock-related information and trading stocks by using mobile communication devices
CN118132295B (en) Parameter updating method and device and related equipment
CN108694251A (en) A kind of Log Administration System
CN112395344A (en) PLC data acquisition management system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant