CN113360538B - Space-time convergence and query method for energy data - Google Patents
Space-time convergence and query method for energy data Download PDFInfo
- Publication number
- CN113360538B CN113360538B CN202110797362.8A CN202110797362A CN113360538B CN 113360538 B CN113360538 B CN 113360538B CN 202110797362 A CN202110797362 A CN 202110797362A CN 113360538 B CN113360538 B CN 113360538B
- Authority
- CN
- China
- Prior art keywords
- time
- data
- granularity
- target
- data structure
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000002776 aggregation Effects 0.000 claims abstract description 18
- 238000004220 aggregation Methods 0.000 claims abstract description 18
- 238000005265 energy consumption Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 4
- 230000004044 response Effects 0.000 abstract description 3
- 235000019580 granularity Nutrition 0.000 description 72
- 238000003780 insertion Methods 0.000 description 3
- 230000037431 insertion Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000012731 temporal analysis Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- Marketing (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Mathematical Physics (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a space-time convergence method of energy utilization data, which comprises the following steps: s1, constructing a hierarchical space-time data structure; s2, at the minimum position granularity, the energy utilization data are sequentially converged according to the sequence from small to large of the time granularity, and converged data with the minimum position granularity at the maximum time granularity are obtained; s3, inserting the converged data with the minimum position granularity at the maximum time granularity into the energy utilization data with the position granularity at the minimum time granularity at the upper stage to form the converged data with the position granularity at the minimum time granularity at the upper stage; s4, starting from the position granularity of the previous stage by analogy of the steps S2-S3, sequentially converging according to the sequence from small to large of the position granularity to obtain converging data with the maximum position granularity at the maximum time granularity; thereby ensuring the accuracy and the execution efficiency of data aggregation; the query method can realize the space-time query of the converged data by adopting a specially designed query language through the query engine, and has high efficiency and quick response.
Description
Technical Field
The invention relates to the field of intelligent power distribution networks, in particular to a space-time aggregation and query method of energy consumption data.
Background
The intelligent power grid can not only improve efficiency and increase value, but also provide data-centered service for clients to realize the digital transformation target by analyzing mass energy data acquired by the distributed monitoring equipment. The amount of data generated by smart grids is enormous and grows at an explosive rate each year. In 2020, the holding quantity of the European intelligent ammeter reaches 2.4 hundred million, and China reaches more than 4 hundred million. Not only does the massive energy data place a heavy burden on the data store, but it makes data scalability analysis very difficult. The data generated by the smart grid arrives in a continuous and unpredictable data stream form, requiring a data structure that can support fast updating and efficient querying of the data stream. Taking metering data as an example, while the data generated by smart meters is typically smooth (terminal ID, installation location, metering readings), many decision functions require aggregating and analyzing data generated by a large number of smart meters at different levels of time and space (e.g., energy per hour for all smart meters at a particular bay).
The existing big data aggregation and query method cannot be directly applied to the application energy data. Firstly, the data stream processing methods such as Spark, flink and Storm mainly adopt frequent item mining technology, only important data need to be reserved when data are gathered, the rest data are discarded, and any data are not allowed to be discarded for energy bill and power grid running state estimation (used for analyzing the power characteristics of a power grid). Second, the above method cannot capture the spatiotemporal nature of the data. For example, the energy consumption data is usually from smart meters located at the client side, and in order to ensure smooth operation of the smart grid, it is required to aggregate and analyze different periods (minutes, hours, days, weeks, years), different locations (smart meters, bays, power stations, branches, head office), or both, and such special data hierarchy analysis and temporal analysis cannot be supported by conventional database management systems, time series database management systems, big data and data stream processing tools, etc.
Disclosure of Invention
Therefore, the invention aims to overcome the defects in the prior art, and provides a space-time convergence and query method for data, which can ensure the accuracy and the execution efficiency of data convergence, reduce the storage space and select proper granularity according to convergence requirements. The query engine can realize the space-time query of the converged data, and has high efficiency and quick response.
The energy utilization data space-time aggregation method comprises the following steps:
s1, constructing a hierarchical space-time data structure; the hierarchical spatiotemporal data structure comprises a time dimension structure and a position dimension structure;
s2, at the minimum position granularity of the position dimension structure, the energy utilization data are sequentially converged according to the sequence from small time granularity to large time granularity in the time dimension structure, so that converged data with the minimum position granularity at the maximum time granularity are obtained;
s3, inserting the converged data with the minimum position granularity at the maximum time granularity into the energy utilization data with the position granularity at the minimum time granularity at the first stage above the minimum position granularity to form the converged data with the position granularity at the first stage above the minimum position granularity at the minimum time granularity;
s4, starting from the position granularity of the upper level of the minimum position granularity by analogy in the steps S2-S3, sequentially converging according to the sequence from small to large of the position granularity in the position dimension structure, and obtaining converging data with the maximum position granularity at the maximum time granularity.
Further, the hierarchical spatiotemporal data structure is composed of node sets arranged in a grid form; the nodes adopt cylindrical data structures.
Further, the cylindrical data structure includes N independent time queue data structures.
Further, the time queue data structure is a first-in first-out circular buffer, and the number of cache units of the circular buffer is C.
Further, the number N of the independent time queue data structures is determined according to the number of the intelligent electric meters with the target position granularity.
Further, the step S2 specifically includes:
s21, setting the number of time queue data structures according to the position of the target position granularity, and setting the number of cache units of each time queue data structure of the target position granularity according to the size of the time granularity;
s22, acquiring energy data acquired by all intelligent electric meters, and calculating the variation of the energy data of all the intelligent electric meters;
s23, starting from the minimum time granularity, inserting the variable quantity into a target cache unit of a target time queue data structure;
s24, if the number of the buffer units with the variable data stored in the target time queue data structure in the step S23 reaches the set number, inserting all the variable data in the target time queue data structure into the target buffer units of the time queue data structure corresponding to the last time granularity of the minimum time granularity;
s25, completing the aggregation of the energy utilization data from the small time granularity to the large time granularity according to the analogy of the steps S23-S24, and obtaining the aggregated data with the target position granularity at the maximum time granularity.
Further, in step S23, the target cache unit of the target time queue data structure is determined according to the following steps:
a. setting the data flow tau generated by the intelligent ammeter as tau=<e,t,△ e >The method comprises the steps of carrying out a first treatment on the surface of the Wherein e is an identifier of the intelligent ammeter, t is a time stamp, and delta e The variation of the energy consumption data;
b. taking a time queue data structure with the number of e as a target time queue data structure in a plurality of time queue data structures;
c. selecting a corresponding numerical value from the time stamp t according to the target time granularity, and taking the numerical value as a target numerical value;
d. among the plurality of cache units of the target time queue data structure, the cache unit numbered as the target value is taken as a target cache unit.
A method for querying energy consumption data comprises the following steps:
A1. constructing a query language model;
A2. adjusting each parameter value in the query language model to enable the query language model to generate a query language meeting the query requirement, and taking the query language as a target query language;
A3. target data is queried from the hierarchical spatiotemporal data structure using a target query language.
Further, a query language model is determined according to the following formula:
f(Q)[{+,-,*,/}]；
wherein f is an operation function, Q is a parameter of f, [ { +, -,/} ] is an optional query syntax for performing f (Q) operations; the Q= [ Time, tr ], [ Loc, lr ], time is the attribute value of the Time hierarchy, loc is the attribute value of the position hierarchy, tr is the specific Time, and lr is the specific position.
The beneficial effects of the invention are as follows: the space-time aggregation and query method for the energy utilization data adopts the grid-shaped hierarchical space-time data structure to ensure the accuracy and the execution efficiency of data aggregation, realizes the gradual aggregation of data at a high level of a hierarchical structure, stores the latest data in a granularity form at a low level, not only can reduce the storage space, but also can select proper granularity according to the aggregation requirement. By adopting a specially designed query language, the query engine can realize the space-time query of the converged data, and has high efficiency and quick response.
Drawings
The invention is further described below with reference to the accompanying drawings and examples:
FIG. 1 is a schematic flow chart of a space-time convergence method of the invention;
FIG. 2 is a schematic diagram of a hierarchical spatiotemporal data structure of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings, as shown in fig. 1:
the space-time convergence method of the energy utilization data comprises the following steps:
s1, constructing a hierarchical space-time data structure; the hierarchical spatiotemporal data structure comprises a time dimension structure and a position dimension structure;
s2, at the minimum position granularity of the position dimension structure, the energy utilization data are sequentially converged according to the sequence from small time granularity to large time granularity in the time dimension structure, so that converged data with the minimum position granularity at the maximum time granularity are obtained;
s3, inserting the converged data with the minimum position granularity at the maximum time granularity into the energy utilization data with the position granularity at the minimum time granularity at the first stage above the minimum position granularity to form the converged data with the position granularity at the first stage above the minimum position granularity at the minimum time granularity;
s4, starting from the position granularity of the upper level of the minimum position granularity by analogy in the steps S2-S3, sequentially converging according to the sequence from small to large of the position granularity in the position dimension structure, and obtaining converging data with the maximum position granularity at the maximum time granularity.
The time granularity can be divided into Minutes (MI), hours (HR), days (DY), weeks (WE) and Years (YE) according to the order of the granularity from small to large, and a time hierarchy relation is formed among the time granularities; the position granularity can be divided into a Smart Meter (SM), a station area (SD), a power supply station (PS), a branch company (BO) and a main company (HO) according to the order of the granularity from small to large, the position granularity forms a position hierarchy relation, the station area is used as the upper stage of the smart meter, the power supply station is used as the upper stage of the station area, and the like. The intelligent ammeter belongs to a terminal.
The grid-shaped hierarchical space-time data structure (Spatiotemporal Data Structure, STDS) is adopted to ensure the accuracy and the execution efficiency of data aggregation, so that the data is gradually aggregated at a high level of the hierarchical structure, the latest data is stored in a granularity form at a low level, the storage space can be reduced, and the proper granularity can be selected according to the aggregation requirement.
In this embodiment, the grid-like hierarchical spatiotemporal data structure is composed of node sets arranged in a grid form; the nodes store and aggregate energy data from the smart meter in the form of cylindrical data structures (cylindrical shape data structure, CSDS).
In this embodiment, the cylindrical data structure includes N independent time queue data structures (time queue data structure, TQDS).
In this embodiment, the time queue data structure is a first-in first-out circular buffer, and the number of cache units of the circular buffer is set to be C. By storing N data streams in each CSDS of the current level, each TQDS sums the latest C data aggregates and then sends the latest C data aggregates and sums to the CSDS of the previous level, and simultaneously deletes the C data. This aggregation approach not only stores data in an aggregated form, but also ensures that no data is discarded or lost.
As shown in FIG. 2, nodes [0,0], [1,0], …, [4,0] are smart meter hierarchies distributed along the time granularity, but are all located, and these nodes are referred to as the 0 th level of the STDS. Similarly, nodes [0,1], [1,1], [4,1] are referred to as stage 1 of the STDS, and so on. It can be seen that the higher level nodes of the STDS contain aggregate data from the lower level nodes. For example, a level 1 node contains data for a level 0 node, a level 2 node contains data for a level 1 node, and so on.
If more hours of data need to be stored in the HR-SD node, such as the last 100 hours of data, c=100 can be set. Since the upper layer of the "HR-SD" node is the "DA-SD" node (along the time dimension indicated by the solid line), when the "HR-SD" node sets c=24, the TQDS aggregates the 24 data and sends the aggregated data to a buffer unit of the "DA-SD" node TQDS. During the queuing operation, a non-zero sum is returned when C insertions occur, otherwise a 0 is returned. The returned non-zero sums are the aggregate data sent to the upper level node.
In this embodiment, the number N of independent time queue data structures is determined according to the number of smart meters with the granularity of the target location. If the number of the smart meters arranged at the terminal level is 10, the number N of the time queue data structures in each CSDS at the terminal position level is 10.
In this embodiment, the step S2 specifically includes:
s21, setting the number of time queue data structures according to the position of the target position granularity, and setting the number of cache units of each time queue data structure at the target position granularity according to the size of the time granularity;
wherein, a position vector Φ= {16,8,4,2} and a time vector ψ= {60,24,7,52,10} are set; the location vector is used to determine the number N of independent TQDS in the CSDS, the location vector set described above means that the node CSDS labeled SM has 16 TQDS; the node CSDS labeled SD has 8 TQDSs, the node CSDS labeled PS has 4 TQDSs, the node CSDS labeled BO has 2 TQDSs, and the node CSDS labeled HO has 1 TQDS.
The time vector is used to determine the number of cache units of the TQDS of different levels. The above set time vector ψ= {60,24,7,52,10} means that the number of cache units of the TQDS in each level of nodes is: the node marked MI has 60 cache units per TQDS, the node marked HR has 24 cache units per TQDS, the node marked DA has 7 cache units per TQDS, the node marked WE has 52 cache units per TQDS, and the node marked YE has 10 cache units per TQDS.
S22, acquiring energy data acquired by all intelligent electric meters, and calculating the variation of the energy data of all the intelligent electric meters; wherein, the intelligent ammeter continuously transmits the data v, and then the STDS stores the variation delta of the energy data collected twice continuously e ＝v t-1 -v t I.e. energy used since the previous reading.
S23, starting from the minimum time granularity, inserting the variable data into a target cache unit of a target time queue data structure;
s24, if the number of the buffer units with the variable data stored in the target time queue data structure in the step S23 reaches the set number, inserting all the variable data in the target time queue data structure into the target buffer units of the time queue data structure corresponding to the last time granularity of the minimum time granularity; wherein, the node marked MI has 60 cache units per TQDS, and the number of the cache units set at the moment is 60.
S25, completing the aggregation of the energy utilization data from the small time granularity to the large time granularity according to the analogy of the steps S23-S24, and obtaining the aggregated data with the target position granularity at the maximum time granularity.
In this embodiment, in step S23, the target cache unit of the target time queue data structure is determined according to the following steps:
a. setting the data flow tau generated by the intelligent ammeter as tau=<e,t,Δ e >The method comprises the steps of carrying out a first treatment on the surface of the Wherein e is an identifier of the intelligent ammeter, t is a time stamp, and delta e The variation of the energy consumption data;
b. taking a time queue data structure with the number of e as a target time queue data structure in a plurality of time queue data structures;
c. selecting a corresponding numerical value from the time stamp t according to the target time granularity, and taking the numerical value as a target numerical value;
d. among the plurality of cache units of the target time queue data structure, the cache unit numbered as the target value is taken as a target cache unit.
Wherein each node of the STDS is associated with a location SM, DS, PS, BO and HO, i.e., each location is mapped to a positive integer j using a one-to-one mapping, and the data for that location is inserted into the TQDS located at index j of the CSDS. For example, the smart meter collection data numbered 9 is inserted into the 9 th TQDS of the CSDS labeled SM node. Specifically, when the received data stream is τ=<e＝5,t＝13/05/20200845,Δ e ＝152>The smart meter with number 5 (corresponding to the identifier) collects data 152 at 45 minutes at 8 am on 5/13/2020, e is used to locate the TQDS, and t is used to determine the cache unit in the TQDS. Data 152 is inserted in the 45 th cache location of the 5 th TQDS of the node marked MI (the lowest level node of the STDS), and when more than 15 arrays are received (up to 1 hour), the 60 data are aggregated and inserted in the 8 th cache location of the 5 th TQDS of the node marked HR (this batch of 60 data belongs to 8 hours in the morning).
By the above space-time convergence method, the time complexity of the storage resource requirement and the data update is analyzed as follows:
storage resource requirements: when the time level vector and the position level vector set by the STDS are respectively psi and phi, C epsilon psi and N epsilon phi, and the storage resource required by the CSDS is C multiplied by N. Specifically, for a certain node n in the STDS ij The memory resource required by CSDS is C i ×N j Then the total memory resources required by STDS areIt shows that the storage space required for layering the STDS is independent of the data size and depends only on the set time hierarchy vector ψ and the position hierarchy vector Φ. For example, the time hierarchy vector ψ= {60,24,7,52,10} and the position hierarchy vector Φ= {8239,2000,200,50,1} are set respectively, then the storage resources required for STDS are:
{60,24,7,52,10}×{8239,2000,200,50,1}＝1601370
meaning that the total number of cache units required for STDS is 1601370, if the amount of data storage required for each cache unit is 100 bytes, then the total storage space required for STDS is 160137000 bytes = 152.72MB.
Time complexity of data update: for each piece of energy data generated by the smart meter, the data in the circular buffer needs to be updated. The circular buffer adopts a data structure based on hash, one hash operation is needed to update data, and 60 arrays v are stored in the circular buffer after 60 times of data update 0 ,…,v 59 Calculate Δ=v 59 -v 0 Insert it into MI-node [0,0] of grid right diagonal line in FIG. 2],[0,1],[0,2],[03],[0,4]Is a kind of medium. The calculation Δ and the insertion node respectively need to perform one operation, the loop buffer receives 60 groups of data, which need to perform (60+5× (1 (calculation Δ) +1 (insertion data))) =70 operations, and the time complexity of performing one data update is 70/60=o (1).
A method for querying energy data based on a space-time convergence method of the energy data comprises the following steps:
A1. constructing a query language model;
A2. adjusting each parameter value in the query language model to enable the query language model to generate a query language meeting the query requirement, and taking the query language as a target query language;
A3. target data is queried from the hierarchical spatiotemporal data structure using a target query language.
In this embodiment, the query language model is determined according to the following formula:
f(Q)[{+,-,*,/}]；
wherein f is an operation function, Q is a parameter of f, [ { +, -,/} ] is an optional query syntax for performing f (Q) operations; the Q= [ Time, tr ], [ Loc, lr ], time is the attribute value of the Time hierarchy, loc is the attribute value of the position hierarchy, tr is the specific Time, and lr is the specific position.
Wherein the query language supports a wide range of advanced user queries. For example, the query operation in the form of "how much energy is per each smart meter recorded" is All ([ WE ], [ SM ]). Similarly, the query operation performed by "which station has higher energy demand in the last 5 weeks" is Max ([ WE,1-5], [ SD ]); the query operation of "average daily energy per smart meter in the last year" is All ([ YEs, 1], [ SM ])/365.
By the above-mentioned query method, the time complexity of the data query is analyzed as follows:
1) Single point query: in the single-point query process, the CSDS for locating the STDS node needs to be operated once, and the TQDS for locating the CSDS needs to be operated once. Thus, querying the data in the STSD requires two positioning operations, with a time complexity of O (1).
2) Range query: the number of positioning operations for a range query depends on the time and location range tr, lr. For the time range tr, tr single-point queries need to be initiated; for a location range lr, lr single point queries need to be initiated, i.e., the range query has a temporal complexity O (tr×lr).
Finally, it is noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made thereto without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered by the scope of the claims of the present invention.
Claims (5)
1. A space-time convergence method of energy-consumption data is characterized in that: the method comprises the following steps:
s1, constructing a hierarchical space-time data structure; the hierarchical spatiotemporal data structure comprises a time dimension structure and a position dimension structure; the hierarchical space-time data structure consists of node sets arranged in a grid form; the nodes adopt cylindrical data structures; the cylindrical data structure includes N independent time queue data structures;
s2, at the minimum position granularity of the position dimension structure, the energy utilization data are sequentially converged according to the sequence from small time granularity to large time granularity in the time dimension structure, so that converged data with the minimum position granularity at the maximum time granularity are obtained; the step S2 specifically includes:
s21, setting the number of time queue data structures according to the position of the target position granularity, and setting the number of cache units of each time queue data structure of the target position granularity according to the size of the time granularity;
s22, acquiring energy data acquired by all intelligent electric meters, and calculating the variation of the energy data of all the intelligent electric meters;
s23, starting from the minimum time granularity, inserting the variable quantity into a target cache unit of a target time queue data structure;
s24, if the number of the buffer units with the variable data stored in the target time queue data structure in the step S23 reaches the set number, inserting all the variable data in the target time queue data structure into the target buffer units of the time queue data structure corresponding to the last time granularity of the minimum time granularity;
s25, completing the aggregation of the energy utilization data from the small time granularity to the large time granularity according to the analogy of the steps S23-S24, and obtaining the aggregated data with the target position granularity at the maximum time granularity;
s3, inserting the converged data with the minimum position granularity at the maximum time granularity into the energy utilization data with the position granularity at the minimum time granularity at the first stage above the minimum position granularity to form the converged data with the position granularity at the first stage above the minimum position granularity at the minimum time granularity;
s4, starting from the position granularity of the upper level of the minimum position granularity by analogy in the steps S2-S3, sequentially converging according to the sequence from small to large of the position granularity in the position dimension structure, and obtaining converging data with the maximum position granularity at the maximum time granularity.
2. The method for space-time aggregation of energy consumption data according to claim 1, wherein: the time queue data structure is a first-in first-out circular buffer, and the number of cache units of the circular buffer is C.
3. The method for space-time aggregation of energy consumption data according to claim 1, wherein: and the number N of the independent time queue data structures is determined according to the number of the intelligent electric meters with the granularity of the target positions.
4. The method for space-time aggregation of energy consumption data according to claim 1, wherein: in step S23, the target cache unit of the target time queue data structure is determined according to the following steps:
a. setting the data flow tau generated by the intelligent ammeter as tau= < e, t and delta e >; wherein e is an identifier of the intelligent ammeter, t is a time stamp, and delta e The variation of the energy consumption data;
b. taking a time queue data structure with the number of e as a target time queue data structure in a plurality of time queue data structures;
c. selecting a corresponding numerical value from the time stamp t according to the target time granularity, and taking the numerical value as a target numerical value;
d. among the plurality of cache units of the target time queue data structure, the cache unit numbered as the target value is taken as a target cache unit.
5. A method for querying aggregated data based on the space-time aggregation method of energy data according to any one of claims 1 to 4, characterized in that: the method comprises the following steps:
A1. constructing a query language model;
determining a query language model according to the following formula:
f(Q)[{+,-,*,/}]；
wherein f is an operation function, Q is a parameter of f, [ { +, -,/} ] is an optional query syntax for performing f (Q) operations; the Q= [ Time, tr ], [ Loc, lr ], time is the attribute value of the Time hierarchy, loc is the attribute value of the position hierarchy, tr is specific Time, and lr is specific position;
A2. adjusting each parameter value in the query language model to enable the query language model to generate a query language meeting the query requirement, and taking the query language as a target query language;
A3. target data is queried from the hierarchical spatiotemporal data structure using a target query language.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110797362.8A CN113360538B (en) | 2021-07-14 | 2021-07-14 | Space-time convergence and query method for energy data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110797362.8A CN113360538B (en) | 2021-07-14 | 2021-07-14 | Space-time convergence and query method for energy data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113360538A CN113360538A (en) | 2021-09-07 |
| CN113360538B true CN113360538B (en) | 2024-04-09 |
Family
ID=77539539
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110797362.8A Active CN113360538B (en) | 2021-07-14 | 2021-07-14 | Space-time convergence and query method for energy data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113360538B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105608222A (en) * | 2016-01-12 | 2016-05-25 | 中国人民解放军国防科学技术大学 | Rapid building method of tile pyramid for large-scale raster data set |
| CN106844664A (en) * | 2017-01-20 | 2017-06-13 | 北京理工大学 | A kind of time series data index structuring method based on summary |
| CN111797174A (en) * | 2019-04-08 | 2020-10-20 | 华为技术有限公司 | Method and apparatus for managing spatiotemporal data |
| CN112100130A (en) * | 2020-09-09 | 2020-12-18 | 陕西师范大学 | Massive remote sensing variable multi-dimensional aggregation information calculation method based on data cube model |
-
2021
- 2021-07-14 CN CN202110797362.8A patent/CN113360538B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105608222A (en) * | 2016-01-12 | 2016-05-25 | 中国人民解放军国防科学技术大学 | Rapid building method of tile pyramid for large-scale raster data set |
| CN106844664A (en) * | 2017-01-20 | 2017-06-13 | 北京理工大学 | A kind of time series data index structuring method based on summary |
| CN111797174A (en) * | 2019-04-08 | 2020-10-20 | 华为技术有限公司 | Method and apparatus for managing spatiotemporal data |
| CN112100130A (en) * | 2020-09-09 | 2020-12-18 | 陕西师范大学 | Massive remote sensing variable multi-dimensional aggregation information calculation method based on data cube model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113360538A (en) | 2021-09-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105069703B (en) | A kind of electrical network mass data management method | |
| CN110047014A (en) | A kind of user's electricity data restorative procedure based on load curve and history electricity | |
| CN110225074B (en) | Communication message distribution system and method based on equipment address domain | |
| Šikšnys et al. | Aggregating and disaggregating flexibility objects | |
| CN104731799B (en) | Main memory DBM device | |
| CN104317800A (en) | Hybrid storage system and method for mass intelligent power utilization data | |
| CN103235820B (en) | Date storage method and device in a kind of group system | |
| EP3039541B1 (en) | Meter reading data validation | |
| CN103618644A (en) | Distributed monitoring system based on hadoop cluster and method thereof | |
| CN103955509A (en) | Quick search method for massive electric power metering data | |
| CN104599032A (en) | Distributed memory power grid construction method and system for resource management | |
| CN108052634A (en) | A kind of power grid production control great Qu and the integrated approach of asset management great Qu multi information systems | |
| CN105354247A (en) | Geographical video data organization management method supporting storage and calculation linkage | |
| CN105183371A (en) | Migration balancing policy based electricity-consuming information distributed file storage method and apparatus | |
| Shah et al. | A spatiotemporal data summarization approach for real-time operation of smart grid | |
| CN103530335B (en) | In-stockroom operation method and device of electric power measurement acquisition system | |
| CN111008245A (en) | WAMS data sharing method for scheduling mechanism above provincial level based on regulation cloud | |
| CN113360538B (en) | Space-time convergence and query method for energy data | |
| CN113947498A (en) | Data storage and retrieval method for converged terminal data center | |
| CN107682180B (en) | Method for collecting performance index data of communication network equipment | |
| CN112182499A (en) | Low-voltage distribution network topological structure identification method based on time sequence electric quantity data | |
| CN113868465B (en) | Locatable video retrieval method based on two-dimensional time grid | |
| CN110674235A (en) | Marketing and distribution topology file synchronous management system and method | |
| CN107707487A (en) | The real-time retrieval system and real-time search method of a kind of network service traffic | |
| Da Silva et al. | Clusmaster: A clustering approach for sampling data streams in sensor networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |