CN112528097A - Historical trend query method and device for monitoring data of online equipment - Google Patents
Historical trend query method and device for monitoring data of online equipment Download PDFInfo
- Publication number
- CN112528097A CN112528097A CN202011495738.1A CN202011495738A CN112528097A CN 112528097 A CN112528097 A CN 112528097A CN 202011495738 A CN202011495738 A CN 202011495738A CN 112528097 A CN112528097 A CN 112528097A
- Authority
- CN
- China
- Prior art keywords
- query
- data
- data points
- time interval
- minimum time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
- G06F16/90348—Query processing by searching ordered data, e.g. alpha-numerically ordered data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9035—Filtering based on additional data, e.g. user or group profiles
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a historical trend query method and a historical trend query device for monitoring data of online equipment, wherein the method comprises the following steps: s1: acquiring query information of data to be queried; s2: inquiring according to the number M of the inquiry data points of the inquiry information: if the number M of the query data points is less than or equal to the number N of the preset fixed data points, directly querying to obtain M data and returning; if the number M of the query data points is larger than the number N of the fixed data points, compressing the data to be queried and returning; determining a minimum time interval according to a query time period of query information, and sequentially extracting characteristic data points by taking the minimum time interval as a unit, wherein the characteristic data points comprise a maximum value, a minimum value and an initial value; and combining the acquired characteristic data points according to a time sequence to obtain a compressed query result. The invention not only effectively compresses the data quantity of the query and improves the query speed, but also keeps the trend of the data points to the maximum extent by the extracted characteristic points.
Description
Technical Field
The invention belongs to the technical field of data lossy compression, and particularly relates to a historical trend query method and a historical trend query device for monitoring data of online equipment.
Background
As the demand of users for data monitoring becomes higher and higher, more and more online devices start to support second-level uploading of monitoring data. How to quickly and correctly perform historical trend query of a large time span also becomes a technical problem.
In historical trend query, the number of data points returned by the query is increased as the time span of the query (the time difference between the start time and the end time of the query) is increased. When the number of data points returned by the query is too large, the query speed is influenced, and the efficiency and the attractiveness of interface data display are also influenced. Such as: for certain monitoring data, if the data is sent every second, the data of one day of inquiry may have 86400 data points, the transmitted data amount may be more than 2MB, if the time span is further expanded by one month, the original data point may be 2592000 data points, and the transmitted data amount may be more than 60 MB. It is clear that communicating the raw data directly is very inefficient.
A patent with a prior art patent application number of "CN 201110252901" patent name "a process data lossy compression method based on linearity" discloses a process data lossy compression method based on linearity, which includes taking a straight line type, a parabolic type and an inverse parabolic type as optional fitting functions, and dynamically adjusting the form of an expression of the fitting function during operation according to the variation trend of actual data.
The patent with the prior art patent application number of 'CN 201210392281' patent name 'spatio-temporal data lossy compression method based on Fourier transform' discloses a spatio-temporal data lossy compression method based on Fourier transform, which is characterized in that any group of spatio-temporal data is decomposed into three functions of x, y and z for spatial centering of time parameters, Fourier change is respectively carried out on data points in each dimension, after data are compressed, if data points exceeding the preset error requirement range are found, and repairing the data after the data point is restored, comparing the difference value between the data point after being compressed and the original data, finding out the data with the maximum difference value and replacing the compressed data with the original data, if the data point still exceeds the preset error requirement range after being replaced, sequentially comparing the rest difference values, and replacing the compressed data with the original data until the data point meets the preset error requirement range or is restored to the original data.
However, the above patents do not solve the technical problems of fast data compression and accurate data feedback in online query of large data amount data. In order to solve the problem, data compression is generally required to be performed on data returned by query, and the compressed data is directly returned, however, in the existing lossy compression technology, such as revolving door compression, periodic sampling, an average value method, and the like, the change trend of the feedback data is inaccurate.
Disclosure of Invention
The invention provides a historical trend query method and a historical trend query device for online equipment monitoring data, aiming at solving the technical problems.
In order to solve the problems, the technical scheme of the invention is as follows:
a historical trend query method for monitoring data of online equipment comprises the following steps:
s1: acquiring query information of data to be queried;
s2: inquiring according to the number M of the inquiry data points of the inquiry information:
if the number M of the query data points is less than or equal to the number N of the preset fixed data points, directly querying to obtain M data and returning;
if the number M of the query data points is larger than the number N of the fixed data points, compressing the data to be queried and returning;
wherein, compressing the data to be queried further comprises:
determining a minimum time interval according to a query time period of query information, and sequentially extracting characteristic data points by taking the minimum time interval as a unit, wherein the characteristic data points comprise a maximum value, a minimum value and an initial value in data in the minimum time interval;
and combining the acquired characteristic data points according to a time sequence to obtain a compressed query result.
In one embodiment, determining the minimum time interval according to the query time period of the query information further comprises:
the minimum time interval Dt is (T1-T0)/(N/3), where T1 is the end time of the query period and T0 is the start time of the query period.
In one embodiment, sequentially extracting the feature data points in units of a minimum time interval further comprises:
dividing data in the query time period into a plurality of characteristic segments by taking a minimum time interval as a unit;
and extracting and acquiring the characteristic data points of each characteristic segment.
An apparatus for querying historical trend of monitoring data of online equipment, comprising:
the device comprises an acquisition unit, a query unit and a query unit, wherein the acquisition unit is used for acquiring query information of data to be queried;
the query unit is used for querying according to the query data point number M of the query information:
if the number M of the query data points is less than or equal to the number N of the preset fixed data points, directly querying to obtain M data and returning;
if the number M of the query data points is larger than the number N of the fixed data points, compressing the data to be queried and returning;
the query unit comprises a compression extraction subunit, which is used for determining a minimum time interval according to a query time period of query information, sequentially extracting characteristic data points by taking the minimum time interval as a unit, wherein the characteristic data points comprise a maximum value, a minimum value and an initial value in data in the minimum time interval, and combining the acquired characteristic data points according to a time sequence to obtain a compressed query result.
A computer device comprising a memory and a processor, the memory having stored therein computer readable instructions which, when executed by the processor, cause the processor to perform a method of historical trend querying of online device monitoring data as any one of the above.
A storage medium storing computer readable instructions which, when executed by one or more processors, cause the one or more processors to perform a method of historical trend querying of online device monitoring data as any one of above.
Compared with the prior art, the invention has the following advantages and positive effects:
the invention inquires the historical trend of the online equipment monitoring data through the fixed data point number, if the inquired data point number is smaller than the fixed data point number, a direct inquiry mode is adopted to obtain the real historical trend, if the inquired data point number exceeds the fixed data point number, loss compression is adopted to improve the data inquiry speed and simultaneously maintain the trend of the actual data point to the maximum extent, wherein, the fixed data point number is used for determining the minimum time interval, and the minimum time interval is used as a unit to carry out sectional extraction on the inquired data point to extract the characteristic data points such as the maximum value, the minimum value, the initial value and the like, thereby effectively compressing the inquired data amount and improving the inquiry speed, and the extracted characteristic points retain the trend of the data point to the maximum extent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention.
FIG. 1 is a flow chart illustrating a historical trend query method for online device monitoring data according to the present invention;
FIG. 2 is a trend graph of raw data points to be queried according to an embodiment of the present invention;
FIG. 3 is a data trend graph of the raw data of FIG. 2 after mean sampling compression;
fig. 4 is a data trend graph of the raw data shown in fig. 2 after the historical trend query method of the online device monitoring data is performed.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will be made with reference to the accompanying drawings. It is obvious that the drawings in the following description are only some examples of the invention, and that for a person skilled in the art, other drawings and embodiments can be derived from them without inventive effort.
For the sake of simplicity, the drawings only schematically show the parts relevant to the present invention, and they do not represent the actual structure as a product. In addition, in order to make the drawings concise and understandable, components having the same structure or function in some of the drawings are only schematically illustrated or only labeled. In this document, "one" means not only "only one" but also a case of "more than one".
The method and the device for querying historical trend of monitoring data of online equipment provided by the invention are further described in detail with reference to the accompanying drawings and specific embodiments.
Referring to fig. 1, the present application provides a historical trend query method for monitoring data of an online device, including the following steps:
s1: acquiring query information of data to be queried;
s2: inquiring according to the number M of the inquiry data points of the inquiry information:
if the number M of the query data points is less than or equal to the number N of the preset fixed data points, directly querying to obtain M data and returning;
if the number M of the query data points is larger than the number N of the fixed data points, compressing the data to be queried and returning;
wherein, compressing the data to be queried further comprises:
determining a minimum time interval according to a query time period of query information, and sequentially extracting characteristic data points by taking the minimum time interval as a unit, wherein the characteristic data points comprise a maximum value, a minimum value and an initial value in data in the minimum time interval;
and combining the acquired characteristic data points according to a time sequence to obtain a compressed query result.
The present embodiment will now be described in detail, but is not limited thereto.
The method is suitable for historical trend query of monitoring data of online equipment, and is particularly suitable for historical trend query of big data so as to quickly and accurately determine data trends.
In step S1 of this embodiment, the query information of the data to be queried is obtained, which at least includes a time period of the data to be queried and a content of the data to be queried, where the content of the data to be queried may be defined by a keyword of the data to be queried, and a required number M of query data points, that is, a data amount to be queried, is determined.
In step S2 of this embodiment, an inquiry is performed according to an inquiry data point number M and a fixed data point number N, where the fixed data point number N is a preset inquiry parameter, and the setting of the fixed data point number N is not specifically limited in this embodiment, and may be actually set according to a hardware processing speed of the device.
Specifically, if the number M of query data points is less than or equal to the preset number N of fixed data points, that is, the query time corresponding to the query data volume is within the reasonable acceptance range, in this embodiment, the M data are obtained and returned by direct query, and if the number M of query data points is greater than the number N of fixed data points, that is, the query time corresponding to the query data volume exceeds the reasonable acceptance range, in this embodiment, the data to be queried are compressed and returned, so as to reduce the transmission volume of the data and improve the query response speed.
Further, in this embodiment, the data to be queried is compressed, and specifically, a minimum time interval is determined according to a query time period of the query information, where the minimum time interval Dt is:
Dt＝(T1-T0)/(N/3)
in the present embodiment, the query time period is divided into a plurality of time segments by the above formula, where T1 is an end time of the query time period, and T0 is a start time of the query time period, and T0 to T0+ Dt, T0+ Dt to T0+2Dt, T0+2Dt to T0+3Dt … … T0+ nDt to T1, so as to perform feature data point extraction respectively.
In this embodiment, the characteristic data points of each time segment are extracted by taking the minimum time interval as a unit, wherein the characteristic data points of this embodiment include a maximum value, a minimum value, and an initial value in the data in the minimum time interval, so that the variation trend of each time segment, that is, the fluctuation condition of the data, can be fed back accurately. And then combining the acquired characteristic data points according to a time sequence to obtain a compressed query result and returning the compressed query result.
Referring to fig. 2, the original data actually required to be queried has two fluctuations, referring to fig. 3, the data trend of the original data after mean value sampling compression in the prior art obviously has defects, and referring to fig. 4, the original data still retains the historical change trend after the historical trend query method of the online device monitoring data, and well retains the data characteristics.
In the embodiment, the historical trend of the online equipment monitoring data is inquired through the fixed data point number, if the inquired data point number is smaller than the fixed data point number, a direct inquiry mode is adopted to obtain a real historical trend, and if the inquired data point number exceeds the fixed data point number, loss compression is adopted to improve the data inquiry speed and simultaneously maintain the trend of an actual data point to the maximum extent, wherein the fixed data point number is used for determining the minimum time interval, and the minimum time interval is used as a unit to perform sectional extraction on the inquired data point to obtain the characteristic data points such as the maximum value, the minimum value, the initial value and the like, so that the inquired data amount is effectively compressed, the inquiry speed is improved, and the extracted characteristic point retains the trend of the.
In another embodiment of the present application, based on the foregoing embodiment, a historical trend query device for monitoring data of online equipment is further provided, including:
the device comprises an acquisition unit, a query unit and a query unit, wherein the acquisition unit is used for acquiring query information of data to be queried;
the query unit is used for querying according to the query data point number M of the query information:
if the number M of the query data points is less than or equal to the number N of the preset fixed data points, directly querying to obtain M data and returning;
if the number M of the query data points is larger than the number N of the fixed data points, compressing the data to be queried and returning;
the query unit comprises a compression extraction subunit, which is used for determining a minimum time interval according to a query time period of query information, sequentially extracting characteristic data points by taking the minimum time interval as a unit, wherein the characteristic data points comprise a maximum value, a minimum value and an initial value in data in the minimum time interval, and combining the acquired characteristic data points according to a time sequence to obtain a compressed query result.
The application also provides a computer device, which comprises a memory and a processor, wherein the memory stores computer readable instructions, and the computer readable instructions, when executed by the processor, cause the processor to execute the historical trend query method of the online device monitoring data as mentioned in the above embodiment.
The present application further proposes a storage medium storing computer-readable instructions which, when executed by one or more processors, cause the one or more processors to perform the method for historical trend query of online device monitoring data as mentioned in the above embodiments.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, it is still within the scope of the present invention if they fall within the scope of the claims of the present invention and their equivalents.
Claims (6)
1. A historical trend query method for monitoring data of online equipment is characterized by comprising the following steps:
s1: acquiring query information of data to be queried;
s2: inquiring according to the inquiry data point number M of the inquiry information:
if the number M of the query data points is less than or equal to the number N of the preset fixed data points, directly querying to obtain M data and returning;
if the number M of the query data points is larger than the number N of the fixed data points, compressing the data to be queried and returning;
wherein the compressing the data to be queried further comprises:
determining a minimum time interval according to the query time period of the query information, and sequentially extracting characteristic data points by taking the minimum time interval as a unit, wherein the characteristic data points comprise a maximum value, a minimum value and an initial value in data in the minimum time interval;
and combining the acquired feature data points according to a time sequence to obtain a compressed query result.
2. The method of claim 1, wherein determining the minimum time interval according to the query time period of the query information further comprises:
the minimum time interval Dt is (T1-T0)/(N/3), where T1 is the end time of the query period and T0 is the start time of the query period.
3. The method of claim 1, wherein the sequentially extracting the characteristic data points in units of a minimum time interval further comprises:
dividing data in the query time period into a plurality of characteristic segments by taking a minimum time interval as a unit;
and extracting the characteristic data points of each characteristic segment.
4. An on-line equipment monitoring data historical trend inquiry unit is characterized by comprising:
the device comprises an acquisition unit, a query unit and a query unit, wherein the acquisition unit is used for acquiring query information of data to be queried;
the query unit is used for querying according to the query data point number M of the query information:
if the number M of the query data points is less than or equal to the number N of the preset fixed data points, directly querying to obtain M data and returning;
if the number M of the query data points is larger than the number N of the fixed data points, compressing the data to be queried and returning;
the query unit comprises a compression extraction subunit, which is used for determining a minimum time interval according to the query time period of the query information, sequentially extracting feature data points by taking the minimum time interval as a unit, wherein the feature data points comprise a maximum value, a minimum value and an initial value in data in the minimum time interval, and combining the obtained feature data points according to a time sequence to obtain a compressed query result.
5. A computer device comprising a memory and a processor, the memory having stored therein computer-readable instructions that, when executed by the processor, cause the processor to perform the method of historical trend querying of online device monitoring data as claimed in any one of claims 1 to 3.
6. A storage medium storing computer readable instructions which, when executed by one or more processors, cause the one or more processors to perform the method of historical trend querying of online device monitoring data as claimed in any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011495738.1A CN112528097A (en) | 2020-12-17 | 2020-12-17 | Historical trend query method and device for monitoring data of online equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011495738.1A CN112528097A (en) | 2020-12-17 | 2020-12-17 | Historical trend query method and device for monitoring data of online equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112528097A true CN112528097A (en) | 2021-03-19 |
Family
ID=75000945
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011495738.1A Pending CN112528097A (en) | 2020-12-17 | 2020-12-17 | Historical trend query method and device for monitoring data of online equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112528097A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114547144A (en) * | 2022-01-30 | 2022-05-27 | 清华大学 | Time sequence data range query method, device and equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102693361A (en) * | 2012-05-07 | 2012-09-26 | 北京航空航天大学 | Drawing method for trend curve of large dataset |
| CN111143405A (en) * | 2019-12-23 | 2020-05-12 | 江苏源普控制设备有限公司 | Method for drawing trend graph of water pump high-voltage motor temperature acquisition based on optimized search |
| CN111538728A (en) * | 2020-04-27 | 2020-08-14 | 中国科学技术大学 | Method for archiving and querying historical data of large scientific device |
| CN111952959A (en) * | 2020-07-14 | 2020-11-17 | 北京科东电力控制系统有限责任公司 | Method and device for compressing power grid process simulation time and storage medium |
-
2020
- 2020-12-17 CN CN202011495738.1A patent/CN112528097A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102693361A (en) * | 2012-05-07 | 2012-09-26 | 北京航空航天大学 | Drawing method for trend curve of large dataset |
| CN111143405A (en) * | 2019-12-23 | 2020-05-12 | 江苏源普控制设备有限公司 | Method for drawing trend graph of water pump high-voltage motor temperature acquisition based on optimized search |
| CN111538728A (en) * | 2020-04-27 | 2020-08-14 | 中国科学技术大学 | Method for archiving and querying historical data of large scientific device |
| CN111952959A (en) * | 2020-07-14 | 2020-11-17 | 北京科东电力控制系统有限责任公司 | Method and device for compressing power grid process simulation time and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 李重文等: "基于分段极值的时间序列数据查询显示方法", 《计算机工程》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114547144A (en) * | 2022-01-30 | 2022-05-27 | 清华大学 | Time sequence data range query method, device and equipment |
| CN114547144B (en) * | 2022-01-30 | 2023-03-24 | 清华大学 | Time sequence data range query method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107329983B (en) | Machine data distributed storage and reading method and system | |
| KR102511271B1 (en) | Method and device for storing and querying time series data, and server and storage medium therefor | |
| CN108932257B (en) | Multi-dimensional data query method and device | |
| CN106502868B (en) | Dynamic monitoring frequency adjusting method suitable for cloud computing | |
| WO2016141584A1 (en) | Method and apparatus for compaction of data received over a network | |
| CN102073712B (en) | Holographic process data archiving and inverting method based on dynamic change frame | |
| CN104978324B (en) | Data processing method and device | |
| CN103326958A (en) | Method and system for monitoring data cache | |
| CN106970935B (en) | Real-time data storage structure, data writing method and data reading method | |
| CN111090705B (en) | Multidimensional data processing method, device and equipment and storage medium | |
| WO2019114754A1 (en) | Join query method and system for multiple time sequences under columnar storage | |
| CN114528231A (en) | Data dynamic storage method and device, electronic equipment and storage medium | |
| CN112528097A (en) | Historical trend query method and device for monitoring data of online equipment | |
| CN112613271A (en) | Data paging method and device, computer equipment and storage medium | |
| CN110334118B (en) | Data query method and device | |
| CN115380267A (en) | Data compression method and device, data compression equipment and readable storage medium | |
| CN114398520A (en) | Data retrieval method, system, device, electronic equipment and storage medium | |
| CN117278046B (en) | Time sequence data compression storage method and device, electronic equipment and storage medium | |
| CN108614721A (en) | Application system configures processing method, device and the storage medium of parameter | |
| US10983888B1 (en) | System and method for generating dynamic sparse exponential histograms | |
| CN116932470A (en) | Method, system and storage medium capable of calculating and storing time sequence data of Internet of things | |
| CN110515952A (en) | A kind of storage method of electric power data | |
| CN118118553B (en) | Intelligent sensor data caching method and system based on edge calculation | |
| CN111666286A (en) | Method and device for detecting sub-warehouse and sub-table, computer equipment and storage medium | |
| US11449514B2 (en) | Approximate aggregation queries |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |